Migrating from Distillery to mix release

Mix release became the offical way to release an Elixir project some years ago. Distillery had been the previous one.

Here are the steps needed to move across:

1. Move any config that uses environment variables into runtime exs.

2. Add a release section in your mix.exs

3. If you were using rel/config.exs to copy files into the release add this to a custom function in the mix.exs step

4. Remove distillery from your mix.exs and clean up the lock file.

5. Adjust your build process to call mix release instead of distillery.

This requires a lot of careful testing!

Livebooks and Wardley Maps

Livebooks are the Elixir answer to Jupyter notebooks. They are very easy to share extend and version. Given how easy it is to install livebook as an application this makes it possible to have share buttons that open a copy of a livebook on your own machine.

This is where you can install livebook: https://livebook.dev/

There is a button on this readme in github that can demonstrate opening a livebook from a URL: https://hex.pm/packages/kino_wardley

Livebooks were initially developed to aid in machine learning. However given that they can contain a mix of text and executable content (machine learning models, charts fetched from websites, or now wardley maps) they can be used to share utilities, research summaries and complex information. This is in a form that is easy to version and share!

The practical use of this is that you can run a workshop and send all of the attendees a copy of the results! You can have multiple Wardley Maps mixed with whatever additional text or information that you need.

kino_wardley now has custom x-axis labels

I have just released version 0.4.0 of kino_wardley.

It now has custom evolution axis labels.

This allows the generation of Wardley Maps such as this:

Wardley Map generated with 0.4.0

Here is the code for the above:

KinoWardley.Output.new("""
id myid8
height 400
width 800
title Tea Shop
anchor Business [0.95, 0.63]
anchor Public [0.95, 0.78]
component Cup of Tea [0.79, 0.61] label [19, -4]
component Cup [0.73, 0.78]
component Tea [0.63, 0.81]
component Hot Water [0.52, 0.80]
component Water [0.38, 0.82]
component Kettle [0.43, 0.35] label [-57, 4]
evolve Kettle [0.43, 0.62] label [16, 7]
component Power [0.1, 0.7] label [-27, 20]
evolve Power [0.1, 0.89] label [-12, 21]
Business->Cup of Tea
Public->Cup of Tea
Cup of Tea->Cup
Cup of Tea->Tea
Cup of Tea->Hot Water
Hot Water->Water
Hot Water->Kettle 
Kettle->Power
evolution Experiment->Prototype->Production->Product
""")

I have also found that onlinewardleymaps.com support a lot more features than this currently does.
It will take a while to catch up.

Adding d3 to Livebook

This is a livebook file that adds d3.js to livebook: https://github.com/chriseyre2000/livebooks/blob/main/d3js.livemd

This opens up livebook to be able to use visualisations that go beyond what can be achieved with VegaLite.

VegaLite is a great charting library. d3.js is a lower level visualisation library. There are a whole host of visualisations that go beyond the standard xy plots.

I am planning to write a Kino.D3 library with livecells included.

Using Livebook as a Model

I have been having a quick experiment with livebook.

This is an interactive website that can be used to easily share Elixir code.
Think of it as Juypter Notebooks but with the backing system being Elixir rather than Python.

It has a very neat docker image:

docker run -p 8080:8080 livebook/livebook

This will stand up a local website running livebook. The console will give you a link (including a token) to allow you to access it.

You can create notebooks and add sections in Markdown or Elixir.
The great thing is that these can be interactively edited by multiple concurrent users.
You can also export a notebook as a file, publish the details and import it into another livebook.

I have been thinking about using this to recreate an old mac simulation tool. This tool allowed you to build a model with data flowing through it and you could use it to see bottlenecks.

I have three code blocks:

The first is the initialisation block


Agent.start_link(fn -> 0 end, name: :complete)
Agent.start_link(fn -> 0 end, name: :iteration)
Agent.start_link(fn -> 0 end, name: :qa_backlog)

The second is the update block

Agent.update(:iteration, fn x -> x + 1 end)
Agent.update(:complete, fn x -> x + min(10, Agent.get(:qa_backlog, & &1)) end)
Agent.update(:qa_backlog, fn value -> max(value - 10, 0) end )
Agent.update(:qa_backlog, fn value -> value + 20 end)

The third is the output block

:ok = IO.puts "Iteration #{Agent.get(:iteration, & &1)} Backlog #{Agent.get(:qa_backlog, & &1)} Complete #{Agent.get(:complete, & &1)}"

If you put the three blocks into livebook then you have the start of a model.

This is a very simplistic model. It attempts to show that the QA team will never be able to keep up with the two dev teams feeding it work. This could become far more sophisticated (with a random percentage of QA items failing and being returned to the teams). The idea here is to show how you can use an Agent in livebook.

I am going to see if you can add visualisations.

Announcing About

Last weekend I wrote and released the Elixir package called “about”.

This adds About topics to the iex environment.

It is installed in the usual way:

Add {:about, "~> 0.0.1"} to your deps list in mix.exs

Run mix deps.get

Run iex -S mix

You can now use help topics like:

iex(1)> h About.pipeline

I am looking for suggestions/contributions to the project.

It’s hosted on https://github.com/chriseyre2000/about and available on hex.pm and hexdocs.

Getting Started With Phoenix 1.4 (Part 3)

Continuing the way through the book.

Now have reached the end of Chapter 6.

This adds video to use associations. It includes some of the clever techniques that Phoenix uses to allow the templates to be customised.

By adding an action method you can customise the parameters passed to the various utility methods:

def action(conn, _) do args = [conn, conn.params, conn.assigns.current_user] apply(__MODULE__, action_name(conn), args) end

This allows details from the conn to be decomposed and passed as a parameter.

https://github.com/chriseyre2000/rumbl_1_4

Contentful to Neo4j in Elixir

I just finished an initial port of my neo4j to contentful library from Node to Elixir.

https://github.com/chriseyre2000/contentful_to_neo4j_ex

This is the same utility but in a very different language. It’s not quite as polished as the Node version (but won’t take long to catch up).

Issues that I have had during the port:

  • My machine had a very old erlang implementation (I had installed this when reading the Erlang chapter of 7 languages in 7 weeks – over 7 years ago) which broke HTTPoison, a fairly common HTTP library. The errors pointed to HTTPoison not working on a windows machine (which I have found to not be true).
  • The Contentful Elixir bindings are not very advanced. They don’t return the total number of items that you are paging through. It was not difficult to use the api directly.
  • There are lots of Elixir bindings for Neo4j. Very few of them are clearly documented for writing to Neo4j. Eventually I landed on bolt_sips. Bolt is too primitive, neo4j_sips uses a very old version of HTTPoison.

The Elixir error messages are incredibly clear.

Credo is great for ensuring best practices are applied.

Functional programming allows you to test real code without mocks or spend time fighting promises.

iex is a great REPL environment. You can recompile a module and carry on without restarting everything.

The code is self documenting. This is the top level method:

read_all_assets()
|> read_all_entries
|> process_contentful
|> write_to_neo4j
VSCode is a great editor for elixir. It’s great from the command line
code .
The above will open vscode on the project in the current directory.