Minimalist Clustering Demo

This is based on code from a video that I have previously linked to.
However Elixir has made a few breaking changes since then.

This is going to demonstrate how to cluster two local Elixir nodes, deploy code from one to the other and then upgrade it while it is running.

You need to put the following code in a file called blabber.ex

defmodule Blabber do

  def start do

    spawn(fn -> loop(0) end)

  end

  def loop(uptime) do

    receive do

      :stop ->

        IO.puts "Shutting blabber down ..."

        exit(:normal)

      after 1000 ->

        IO.puts "Nice! #{uptime} seconds of bgu-free uptime on #{node()}."

    end

    Blabber.loop(uptime + 1)

  end

end

Next you need to start iex with a name and a cookie (in the same folder as blabber.ex):

iex --sname cat --cookie super-secret

In a second terminal start another iex session

iex --sname dog --cookie super-secret

From the first terminal connect the two nodes together:

Node.connect :"dog@MacBook-Pro"

Compile the code in the cat node:

c ("blabber.ex", ".")

We need to use this version of the compile function as by default since Elixir 1.5 the beam file is not output by default.

The process can be started on the cat node with:

Blabber.start

Now you can transfer the compiled beam file to all linked nodes using:

nl Blabber

Now from the dog node you can start the process:

Blabber.start

At this point we notice the typo in the code. It say bgu-free rather than bug-free.
Edit the source file save the change.

On the cat node recompile this:

c ("blabber.ex", ".")

This will immediately fix the problem on the cat node (and leave the count updating).

If you use:

nl Blabber

then the solution will be moved to the other node as well.

This demonstrates the simplest possible Clustering arrangement. It is simple to extend this to a move sophisticated service.

Twitter For Tech Support

Sometimes a companies official Tech Support channels can be a little slow.

If the official channels don’t work then you can turn to twitter. Raising a support issue on a social media site can cause the company to take action. Sometimes a DM to them will enable action that a support ticket does not.

Frequencies

This is a simple powershell script that can be used to get the frequency of the first letter from a sample file.

gc ‘./sample’ | %{ $_.substring(0,1) } | group

Running this over say the FTSE 100 symbol list returns:

Count Name                      Group
----- ----                      -----
   10 A                         {A, A, A, A...}
   11 B                         {B, B, B, B...}
    4 C                         {C, C, C, C}
    2 D                         {D, D}
    2 E                         {E, E}
    3 F                         {F, F, F}
    3 G                         {G, G, G}
    5 H                         {H, H, H, H...}
    8 I                         {I, I, I, I...}
    3 J                         {J, J, J}
    1 K                         {K}
    4 L                         {L, L, L, L}
    4 M                         {M, M, M, M}
    3 N                         {N, N, N}
    1 O                         {O}
    6 P                         {P, P, P, P...}
    8 R                         {R, R, R, R...}
   15 S                         {S, S, S, S...}
    2 T                         {T, T}
    2 U                         {U, U}
    1 V                         {V}
    2 W                         {W, W}

This highlights that the symbols are not uniformly spread across the alphabet.

A-F has 1/3 of the market as does P-Z

I found out this once when trying to use the ticker symbol to load balance market data across 3 servers.

Check the distribution of the data before you use a simple key.

Oddly the second letter is a better key:

    6 A                         {A, A, A, A...}
    3 B                         {B, B, B}
    4 C                         {C, C, C, C}
    5 D                         {D, D, D, D...}
    3 E                         {E, E, E}
    6 G                         {G, G, G, G...}
    4 H                         {H, H, H, H}
    3 I                         {I, I, I}
    2 K                         {K, K}
    8 L                         {L, L, L, L...}
    6 M                         {M, M, M, M...}
    7 N                         {N, N, N, N...}
    2 O                         {O, O}
    4 P                         {P, P, P, P}
    8 R                         {R, R, R, R...}
    9 S                         {S, S, S, S...}
    7 T                         {T, T, T, T...}
    2 U                         {U, U}
    6 V                         {V, V, V, V...}
    2 W                         {W, W}
    2 X                         {X, X}
    1 Z                         {Z}

Elixir Demo

This video really demonstrates the power of Elixir: https://t.co/dXljW61NYt?amp=1

Here is the source code for this:

https://github.com/sasa1977/demo_system

It requires a really old version of node (8.3.0).

The demo mostly works with the latest version of Elixir. One of the buttons to set the number of processes does not work, however calling the function in the iex session does work.

I managed to get 10k processes running on a MacBook.

Using Typescript for an NPM Module

This is a useful article on building an NPM Module using typescript.

I am using the technique here to move lie.js to typescript.

The documentation is now a little old. Have a look at github.com/chriseyre2000/lie.js for a more recent version.

The practical advantage of moving to Typescript are the type definitions which the IDE can choose to use even if the module is used from a Javascript app.

Also note that I have CircleCI setup to publish to NPM upon receipt of a tag.

This is how to push tags to github:

git tag -a v1.1.4 -m "my version 1.1.4"
git push origin --tags

Open Source Rights

I am amazed by the sheer number of open source projects that a typical application depends upon. For Node this can be thousands or Java hundreds.

One of the key rights is the ability to fork. Forking projects allows a project that is no longer supported by its authors to continue. Forking an active project is rare – it splits the community. Forking a dead project is essential if you need it.

Several of the projects that I maintain uses http-builder. This has not seen an update in 6 years. This can mean that the project has reached a level of maturity where there is nothing to add.

Http-builder has a dependency upon one of the core groovy libraries. In particular there is a static method called DefaultGroovyMethods.leftShift that has been dropped from Groovy 3. This breaks the library at runtime.

There is an alternative library called http-builder-ng, but this has different call signatures which requires significant rework.

I have started work on a fork of http-request. The fix for the known issue is fairly easy – copy in two methods from the Groovy Libraries. Getting the project to build is more difficult. Some of the tests depend upon real world endpoints that have changed over time. I have ignored these for now, but intend to replace the tests with wiremock. The code also has coverage tests for documentation which is missing and some mininum test coverage requirements that I have now broken.

I now have the build working locally but now need to get it to build on circleci. Some of the dependencies failed to resolve. Once I have them working again I can get this published to either github packages or maven central.