This is a simple powershell script that can be used to get the frequency of the first letter from a sample file.

gc ‘./sample’ | %{ $_.substring(0,1) } | group

Running this over say the FTSE 100 symbol list returns:

Count Name                      Group
----- ----                      -----
   10 A                         {A, A, A, A...}
   11 B                         {B, B, B, B...}
    4 C                         {C, C, C, C}
    2 D                         {D, D}
    2 E                         {E, E}
    3 F                         {F, F, F}
    3 G                         {G, G, G}
    5 H                         {H, H, H, H...}
    8 I                         {I, I, I, I...}
    3 J                         {J, J, J}
    1 K                         {K}
    4 L                         {L, L, L, L}
    4 M                         {M, M, M, M}
    3 N                         {N, N, N}
    1 O                         {O}
    6 P                         {P, P, P, P...}
    8 R                         {R, R, R, R...}
   15 S                         {S, S, S, S...}
    2 T                         {T, T}
    2 U                         {U, U}
    1 V                         {V}
    2 W                         {W, W}

This highlights that the symbols are not uniformly spread across the alphabet.

A-F has 1/3 of the market as does P-Z

I found out this once when trying to use the ticker symbol to load balance market data across 3 servers.

Check the distribution of the data before you use a simple key.

Oddly the second letter is a better key:

    6 A                         {A, A, A, A...}
    3 B                         {B, B, B}
    4 C                         {C, C, C, C}
    5 D                         {D, D, D, D...}
    3 E                         {E, E, E}
    6 G                         {G, G, G, G...}
    4 H                         {H, H, H, H}
    3 I                         {I, I, I}
    2 K                         {K, K}
    8 L                         {L, L, L, L...}
    6 M                         {M, M, M, M...}
    7 N                         {N, N, N, N...}
    2 O                         {O, O}
    4 P                         {P, P, P, P}
    8 R                         {R, R, R, R...}
    9 S                         {S, S, S, S...}
    7 T                         {T, T, T, T...}
    2 U                         {U, U}
    6 V                         {V, V, V, V...}
    2 W                         {W, W}
    2 X                         {X, X}
    1 Z                         {Z}

This is an interesting article on testing in Elixir: http://blog.plataformatec.com.br/2020/01/elixir-what-about-tests/

This video really demonstrates the power of Elixir: https://t.co/dXljW61NYt?amp=1

Here is the source code for this:

https://github.com/sasa1977/demo_system

It requires a really old version of node (8.3.0).

The demo mostly works with the latest version of Elixir. One of the buttons to set the number of processes does not work, however calling the function in the iex session does work.

I managed to get 10k processes running on a MacBook.

This is a useful article on building an NPM Module using typescript.

I am using the technique here to move lie.js to typescript.

The documentation is now a little old. Have a look at github.com/chriseyre2000/lie.js for a more recent version.

The practical advantage of moving to Typescript are the type definitions which the IDE can choose to use even if the module is used from a Javascript app.

Also note that I have CircleCI setup to publish to NPM upon receipt of a tag.

This is how to push tags to github:

git tag -a v1.1.4 -m "my version 1.1.4"
git push origin --tags

I am amazed by the sheer number of open source projects that a typical application depends upon. For Node this can be thousands or Java hundreds.

One of the key rights is the ability to fork. Forking projects allows a project that is no longer supported by its authors to continue. Forking an active project is rare – it splits the community. Forking a dead project is essential if you need it.

Several of the projects that I maintain uses http-builder. This has not seen an update in 6 years. This can mean that the project has reached a level of maturity where there is nothing to add.

Http-builder has a dependency upon one of the core groovy libraries. In particular there is a static method called DefaultGroovyMethods.leftShift that has been dropped from Groovy 3. This breaks the library at runtime.

There is an alternative library called http-builder-ng, but this has different call signatures which requires significant rework.

I have started work on a fork of http-request. The fix for the known issue is fairly easy – copy in two methods from the Groovy Libraries. Getting the project to build is more difficult. Some of the tests depend upon real world endpoints that have changed over time. I have ignored these for now, but intend to replace the tests with wiremock. The code also has coverage tests for documentation which is missing and some mininum test coverage requirements that I have now broken.

I now have the build working locally but now need to get it to build on circleci. Some of the dependencies failed to resolve. Once I have them working again I can get this published to either github packages or maven central.

This (https://devblogs.microsoft.com/scripting/grabbing-excel-xlsx-values-with-powershell/) is a great article on how to use powershell to extract data from an xlsx spreadsheet without having excel installed.

This is going to be very useful

Some time ago I wrote about an alternative to migrations to set up contentful spaces. This was based upon desired state configuration. I no longer work at the company that had that utility so can’t ask to open source it. These are the notes that I am going to use to recreate the utility from scratch. These notes are based from my memory of how Contentful works. Desired State Configuration is far easier to manage than migrations. You only need to edit one file that is kept in line with source control. If there are multiple spaces using the same configuration it will bring them into alignment.

The basic idea is that you have a declaration file that shows how the contentful space is meant to be defined. This covers the content types in the content model. When the utility is run it attempts to upgrade the space to match the declaration.

It will add types that don’t exist first. These will be a single field version of the type. This is sufficient to use them as a reference for other types.

Next it will ensure that the fields on each of the types match the definition given in the configuration. Fields that don’t exists will be added. Fields that already exist will be updated if it is possible to upgrade. Contentful has rules about how a type can change: you can’t turn a string into a number or a float into an integer. Fields that have been removed will be marked as disabled. It can’t work around the hard limit of 50 fields per content type so you may need to manually delete existing disabled fields.

The third part is to apply the “Editor Interface” to any field that has it specified. This will allow the specific UI rules to be applied.

Contentful is eventually consistent with the type updates so you need to wait after making an update to ensure that the read-side will have applied the update. This also means that the migration itself will need to carefully rate-limit it’s actions.

To get started you need a contentful space and a cma token. These need to be generated manually from: https://app.contentful.com/spaces/{insert space id here}/api/cma_tokens

For this example I’ll start using the sample blog space that you can generate with a free account.

This is the curl command that I use to extract the details:

curl https://api.contentful.com/spaces/{insert space id here}/environments/master/content_types -H "Authorization: Bearer {insert token here}"

This returns some json (I may have manually adapted this space from the defaults):

{
  "sys": {
    "type": "Array"
  },
  "total": 3,
  "skip": 0,
  "limit": 100,
  "items": [
    {
      "sys": {
        "space": {
          "sys": {
            "type": "Link",
            "linkType": "Space",
            "id": "e6p9dxi1frd5"
          }
        },
        "id": "blogPost",
        "type": "ContentType",
        "createdAt": "2019-05-22T13:33:53.326Z",
        "updatedAt": "2020-05-14T17:28:43.402Z",
        "environment": {
          "sys": {
            "id": "master",
            "type": "Link",
            "linkType": "Environment"
          }
        },
        "publishedVersion": 3,
        "publishedAt": "2020-05-14T17:28:43.402Z",
        "firstPublishedAt": "2019-05-22T13:33:54.668Z",
        "createdBy": {
          "sys": {
            "type": "Link",
            "linkType": "User",
            "id": "4akz83hNhoZtJH86B6riKE"
          }
        },
        "updatedBy": {
          "sys": {
            "type": "Link",
            "linkType": "User",
            "id": "4akz83hNhoZtJH86B6riKE"
          }
        },
        "publishedCounter": 2,
        "version": 4,
        "publishedBy": {
          "sys": {
            "type": "Link",
            "linkType": "User",
            "id": "4akz83hNhoZtJH86B6riKE"
          }
        }
      },
      "displayField": "title",
      "name": "Blog Post",
      "description": null,
      "fields": [
        {
          "id": "title",
          "name": "Title",
          "type": "Symbol",
          "localized": false,
          "required": true,
          "validations": [],
          "disabled": false,
          "omitted": false
        },
        {
          "id": "slug",
          "name": "Slug",
          "type": "Symbol",
          "localized": false,
          "required": true,
          "validations": [],
          "disabled": false,
          "omitted": false
        },
        {
          "id": "heroImage",
          "name": "Hero Image",
          "type": "Link",
          "localized": false,
          "required": true,
          "validations": [],
          "disabled": false,
          "omitted": false,
          "linkType": "Asset"
        },
        {
          "id": "description",
          "name": "Description",
          "type": "Text",
          "localized": false,
          "required": true,
          "validations": [],
          "disabled": false,
          "omitted": false
        },
        {
          "id": "body",
          "name": "Body",
          "type": "Text",
          "localized": false,
          "required": true,
          "validations": [],
          "disabled": false,
          "omitted": false
        },
        {
          "id": "author",
          "name": "Author",
          "type": "Link",
          "localized": false,
          "required": false,
          "validations": [],
          "disabled": false,
          "omitted": false,
          "linkType": "Entry"
        },
        {
          "id": "publishDate",
          "name": "Publish Date",
          "type": "Date",
          "localized": false,
          "required": true,
          "validations": [],
          "disabled": false,
          "omitted": false
        },
        {
          "id": "tags",
          "name": "Tags",
          "type": "Array",
          "localized": false,
          "required": false,
          "validations": [],
          "disabled": false,
          "omitted": false,
          "items": {
            "type": "Symbol",
            "validations": [
              {
                "in": [
                  "general",
                  "javascript",
                  "static-sites"
                ]
              }
            ]
          }
        },
        {
          "id": "backgroundColour",
          "name": "BackgroundColour",
          "type": "Symbol",
          "localized": false,
          "required": false,
          "validations": [],
          "disabled": false,
          "omitted": false
        }
      ]
    },
    {
      "sys": {
        "space": {
          "sys": {
            "type": "Link",
            "linkType": "Space",
            "id": "e6p9dxi1frd5"
          }
        },
        "id": "person",
        "type": "ContentType",
        "createdAt": "2019-05-22T13:33:53.330Z",
        "updatedAt": "2019-05-23T08:46:50.110Z",
        "environment": {
          "sys": {
            "id": "master",
            "type": "Link",
            "linkType": "Environment"
          }
        },
        "publishedVersion": 5,
        "publishedAt": "2019-05-23T08:46:50.110Z",
        "firstPublishedAt": "2019-05-22T13:33:54.691Z",
        "createdBy": {
          "sys": {
            "type": "Link",
            "linkType": "User",
            "id": "4akz83hNhoZtJH86B6riKE"
          }
        },
        "updatedBy": {
          "sys": {
            "type": "Link",
            "linkType": "User",
            "id": "4akz83hNhoZtJH86B6riKE"
          }
        },
        "publishedCounter": 3,
        "version": 6,
        "publishedBy": {
          "sys": {
            "type": "Link",
            "linkType": "User",
            "id": "4akz83hNhoZtJH86B6riKE"
          }
        }
      },
      "displayField": "name",
      "name": "Person",
      "description": null,
      "fields": [
        {
          "id": "name",
          "name": "Name",
          "type": "Symbol",
          "localized": false,
          "required": true,
          "validations": [],
          "disabled": false,
          "omitted": false
        },
        {
          "id": "title",
          "name": "Title",
          "type": "Symbol",
          "localized": false,
          "required": true,
          "validations": [],
          "disabled": false,
          "omitted": false
        },
        {
          "id": "company",
          "name": "Company",
          "type": "Symbol",
          "localized": false,
          "required": true,
          "validations": [],
          "disabled": false,
          "omitted": false
        },
        {
          "id": "shortBio",
          "name": "Short Bio",
          "type": "Text",
          "localized": false,
          "required": true,
          "validations": [],
          "disabled": false,
          "omitted": false
        },
        {
          "id": "email",
          "name": "Email",
          "type": "Symbol",
          "localized": false,
          "required": false,
          "validations": [],
          "disabled": false,
          "omitted": false
        },
        {
          "id": "phone",
          "name": "Phone",
          "type": "Symbol",
          "localized": false,
          "required": false,
          "validations": [],
          "disabled": false,
          "omitted": false
        },
        {
          "id": "facebook",
          "name": "Facebook",
          "type": "Symbol",
          "localized": false,
          "required": false,
          "validations": [],
          "disabled": false,
          "omitted": false
        },
        {
          "id": "twitter",
          "name": "Twitter",
          "type": "Symbol",
          "localized": false,
          "required": false,
          "validations": [],
          "disabled": false,
          "omitted": false
        },
        {
          "id": "github",
          "name": "Github",
          "type": "Symbol",
          "localized": false,
          "required": false,
          "validations": [],
          "disabled": false,
          "omitted": false
        },
        {
          "id": "image",
          "name": "Image",
          "type": "Link",
          "localized": false,
          "required": false,
          "validations": [],
          "disabled": false,
          "omitted": false,
          "linkType": "Asset"
        },
        {
          "id": "favoriteIceCream",
          "name": "favorite ice cream",
          "type": "Symbol",
          "localized": false,
          "required": false,
          "validations": [],
          "disabled": false,
          "omitted": false
        },
        {
          "id": "petName",
          "name": "pet name",
          "type": "Symbol",
          "localized": false,
          "required": false,
          "validations": [],
          "disabled": false,
          "omitted": false
        }
      ]
    },
    {
      "sys": {
        "space": {
          "sys": {
            "type": "Link",
            "linkType": "Space",
            "id": "e6p9dxi1frd5"
          }
        },
        "id": "update",
        "type": "ContentType",
        "createdAt": "2019-05-22T13:46:35.343Z",
        "updatedAt": "2019-05-22T13:46:38.843Z",
        "environment": {
          "sys": {
            "id": "master",
            "type": "Link",
            "linkType": "Environment"
          }
        },
        "publishedVersion": 3,
        "publishedAt": "2019-05-22T13:46:38.843Z",
        "firstPublishedAt": "2019-05-22T13:46:35.942Z",
        "createdBy": {
          "sys": {
            "type": "Link",
            "linkType": "User",
            "id": "4akz83hNhoZtJH86B6riKE"
          }
        },
        "updatedBy": {
          "sys": {
            "type": "Link",
            "linkType": "User",
            "id": "4akz83hNhoZtJH86B6riKE"
          }
        },
        "publishedCounter": 2,
        "version": 4,
        "publishedBy": {
          "sys": {
            "type": "Link",
            "linkType": "User",
            "id": "4akz83hNhoZtJH86B6riKE"
          }
        }
      },
      "displayField": null,
      "name": "Update",
      "description": "Latest update",
      "fields": [
        {
          "id": "update",
          "name": "Update",
          "type": "RichText",
          "localized": false,
          "required": false,
          "validations": [],
          "disabled": false,
          "omitted": false
        }
      ]
    }
  ]
}

It is unlikely that you will need to add paging to the content type analysis, but I will make the utility handle it.

The key information here is the list of items each with a list of fields and a specified displayField.

I’ll return to this topic as I develop this utility. At time of writing there is no code for this project.

Note that this content type uses readable identifiers for the fields. I would always recommend doing this as it greatly simplifies the use of this data.

I have recently updated https://github.com/chriseyre2000/package_compare to correctly handle package.json with no devDependency section.

I have just published v1.1.2 of lie.js

The library now uses more fluent names and matches the default used for timeouts in Javascript.

This is now being built and published to npm using CircleCI. The minimal test dependencies are kept upto date using dependabot.

Thanks to @ll782 for suggesting improvements.

The point of the library is to introduce timeouts into javascript Promises. It’s only by making timeouts explicit can you build a reliable application. Currently it allows rejects and success with a default value.

I plan to add typescript declarations to the project.

I have just published my first open source npm package, lie.js.

yarn add lie.js

It provides two useful functions creating a Promise that times out with a given error and another that will succeed with a default after a timeout.

The practical use of these is that you can use them with Promise.race to create functions with a bounded response time. It makes it easier to keep within a time budget if you can say go get a value or return a default if too slow.