Category: Uncategorized

Script to help review Exercism.io Elixir

~ chriseyre2000 ~ Leave a comment


@rem fix a.test.exs
@rem removes the pending tag from the test and adds _2 to filename
@echo %1 | sed -e "s/.exs/_2.exs/" > temp.txt
@set /p Filename=<temp.txt
@cat %1 | grep -v "@tag :pending" > %Filename%
@rm temp.txt
elixir %Filename%

view raw

fix_test.bat

hosted with ❤ by GitHub

Or on a mac/linux

grep -v @tag *_test.exs > test.exs && elixir test.exs

Amazon Documentation Also Lies

~ chriseyre2000 ~ Leave a comment

I have spent the last couple of days fighting with oauth 2.

This is based upon Cognito.

The basis of the oauth authentication is a multi step  dance. First you get the user to log in and are redirected back with a code. You need to send back the code with some shared secrets and it responds with a set of tokens.

These tokens do include the refresh token despite the docs saying that they don’t.

The next step is to fetch some user details. This has not yet been implemented yet the docs clearly state that it is there.

I have yet to find the renewal endpoint…

Update…

I have found out why cognito does not implement the ouath/userinfo endpoint.

It’s because the tokens endpoint returns a JWT token that includes the userinfo. I have yet to see if this is the most sane Bearer authentication token.

Testing Promises in Jest

~ chriseyre2000 ~ Leave a comment

I have finally found a decent pattern to test promises.

You can mock a service to return Promise.resolve(x) or Promise.reject(“reason”).

Sample is from contentful-to-neo4j

test(“If contentful is empty then nothing is sent to the db”, (done) => {
  const contentfulService=mockContentfulServiceFactory();
  const neo4jService=mockNeo4jServiceFactory();
  const log=mockLogFactory();
  const systemService=mockSystemServiceFactory();
  contentfulService.getAssets.mockReturnValue(     Promise.resolve(contentfulService.emptyResult));
 contentfulService.getEntries.mockReturnValue(     Promise.resolve(contentfulService.emptyResult));
 const transformService=transformServiceFactory(contentfulService, neo4jService,   contentfulBatchSize, log, systemService);
 transformService.copyContentfulSpaceToNeo4j();
  setTimeout( () => {
     expect(contentfulService.getAssets.mock.calls.length).toEqual(1);
     expect(contentfulService.getEntries.mock.calls.length).toEqual(1);
     expect(neo4jService.cypherCommand.mock.calls.length).toEqual(0);
     done();
    },

  1);
});
The trick is to use setTimeout of 1 to happen after the promises have been resolved.
Since the promises are already complete they all resolve before the next millisecond. The done holds the test up for the timeout.

Sample neo4j queries for the contentful-to-neo4j graph

~ chriseyre2000 ~ Leave a comment

I have now stabilised the contentful-to-neo4j project. It should be able to handle most contentful spaces. I still need to tune the transactions for very large datasets – but have no errors yet.

It has gone from being a simple script to an application that has 100% code coverage. I have learnt a lot about javascript Promises and how to test in Jest.

Here are some useful neo4j queries:

Find out how many of each content type that you have published:

MATCH (n) WHERE n.contenttype IS NOT NULL

RETURN n.contenttype, COUNT(n)

ORDER BY n.contenttype

You need to remember that nodes are circular and need to be surrounded by ()

Find pages that share slugs:

MATCH (a {slug:’common-url’}) RETURN a

Find nodes that have a slug with a specific ending:

MATCH (n) WHERE n.slug ENDS WITH ‘-kr’ RETURN DISTINCT n

Find assets that are not referenced by a content type:

MATCH (image {cmstype: ‘Asset’})
WHERE NOT (image)-[]-()
RETURN image

Simple search for orphans:

MATCH (a) WHERE NOT (a)-[]-() RETURN a

These are simple queries in Neo4j that would be hard to do in the Contentful UI.

Analysing Contentful Spaces in Graphene hosted in Heroku

~ chriseyre2000 ~ Leave a comment

I have just updated my contentful-to-neo4j project so that it will by default work on a graphene database hosted in Heroku.

Steps to get this working:

Clone the repo:

git clone https://github.com/chriseyre2000/contentful-to-neo4j

Sign up to heroku, add a credit card, create an empty heroku app.

Add the free graphene db addon (assuming that you have less than 1000 content types).

Install the heroku cli

Open a terminal in the directory that you checked out the repo.

This may require you to login to heroku on the cli.

heroku git:remote -a YOUR_APP_NAME

set the two keys that identify the contentful space:

SPACE_ID

CONTENTFUL_ACCESS_TOKEN

Once the app has finished restarting:

heroku run loader -a YOUR_APP_NAME

You should now have a full populate graphene database.

You may need to move to a paid tier big enough for your contentful space…

This is surprisingly painless. I only had to add environment variable fallbacks and a Procfile.