Experimenting With Elixir in Docker

I am working on my book Elixir Function Guide. Having noticed that there are a lot of mix tasks that I did not know about (and some that my exercism.io students don’t) then I thought that I would add some details at the start of the book.

I started using mix help to list the available functions:

mix                   # Runs the default task (current: "mix run")
mix app.start         # Starts all registered apps
mix app.tree          # Prints the application tree
mix archive           # Lists installed archives
mix archive.build     # Archives this project into a .ez file
mix archive.install   # Installs an archive locally
mix archive.uninstall # Uninstalls archives
mix clean             # Deletes generated application files
mix cmd               # Executes the given command
mix compile           # Compiles source files
mix credo             # Run code analysis (use `--help` for options)
mix credo.gen.check   # Generate a new custom check for Credo
mix credo.gen.config  # Generate a new config for Credo
mix deps              # Lists dependencies and their status
mix deps.clean        # Deletes the given dependencies' files
mix deps.compile      # Compiles dependencies
mix deps.get          # Gets all out of date dependencies
mix deps.tree         # Prints the dependency tree
mix deps.unlock       # Unlocks the given dependencies
mix deps.update       # Updates the given dependencies
mix do                # Executes the tasks separated by comma
mix escript           # Lists installed escripts
mix escript.build     # Builds an escript for the project
mix escript.install   # Installs an escript locally
mix escript.uninstall # Uninstalls escripts
mix format            # Formats the given files/patterns
mix help              # Prints help information for tasks
mix hex               # Prints Hex help information
mix hex.audit         # Shows retired Hex deps for the current project
mix hex.build         # Builds a new package version locally
mix hex.config        # Reads, updates or deletes local Hex config
mix hex.docs          # Fetches or opens documentation of a package
mix hex.info          # Prints Hex information
mix hex.organization  # Manages Hex.pm organizations
mix hex.outdated      # Shows outdated Hex deps for the current project
mix hex.owner         # Manages Hex package ownership
mix hex.publish       # Publishes a new package version
mix hex.repo          # Manages Hex repositories
mix hex.retire        # Retires a package version
mix hex.search        # Searches for package names
mix hex.user          # Manages your Hex user account
mix loadconfig        # Loads and persists the given configuration
mix local             # Lists local tasks
mix local.hex         # Installs Hex locally
mix local.phx         # Updates the Phoenix project generator locally
mix local.public_keys # Manages public keys
mix local.rebar       # Installs Rebar locally
mix new               # Creates a new Elixir project
mix phx.new           # Creates a new Phoenix v1.4.0 application
mix phx.new.ecto      # Creates a new Ecto project within an umbrella project
mix phx.new.web       # Creates a new Phoenix web project within an umbrella project
mix profile.cprof     # Profiles the given file or expression with cprof
mix profile.eprof     # Profiles the given file or expression with eprof
mix profile.fprof     # Profiles the given file or expression with fprof
mix release           # Assembles a self-contained release
mix release.init      # Generates sample files for releases
mix run               # Starts and runs the current application
mix test              # Runs a project's tests
mix xref              # Prints cross reference information
iex -S mix            # Starts IEx and runs the default task

It was only then that I realised that I have a number of mix extensions installed.

I need to start with what comes out of the box.
At this point I have three choices:

  • Uninstall the extensions (these are usefull .)
  • Manually edit out the extensions (error prone)
  • Use docker

The docker option is appealing as it will allow me to test using different versions of Elixir.

Here is the simple option that allows the use of docker:

docker run -it elixir:1.10 /bin/bash

This downloads the image and runs up the container. Note that this starts with bash, rather than the default iex.

From this mix help gives the more useful list:

mix                   # Runs the default task (current: "mix run")
mix app.start         # Starts all registered apps
mix app.tree          # Prints the application tree
mix archive           # Lists installed archives
mix archive.build     # Archives this project into a .ez file
mix archive.install   # Installs an archive locally
mix archive.uninstall # Uninstalls archives
mix clean             # Deletes generated application files
mix cmd               # Executes the given command
mix compile           # Compiles source files
mix deps              # Lists dependencies and their status
mix deps.clean        # Deletes the given dependencies' files
mix deps.compile      # Compiles dependencies
mix deps.get          # Gets all out of date dependencies
mix deps.tree         # Prints the dependency tree
mix deps.unlock       # Unlocks the given dependencies
mix deps.update       # Updates the given dependencies
mix do                # Executes the tasks separated by comma
mix escript           # Lists installed escripts
mix escript.build     # Builds an escript for the project
mix escript.install   # Installs an escript locally
mix escript.uninstall # Uninstalls escripts
mix format            # Formats the given files/patterns
mix help              # Prints help information for tasks
mix loadconfig        # Loads and persists the given configuration
mix local             # Lists local tasks
mix local.hex         # Installs Hex locally
mix local.public_keys # Manages public keys
mix local.rebar       # Installs Rebar locally
mix new               # Creates a new Elixir project
mix profile.cprof     # Profiles the given file or expression with cprof
mix profile.eprof     # Profiles the given file or expression with eprof
mix profile.fprof     # Profiles the given file or expression with fprof
mix release           # Assembles a self-contained release
mix release.init      # Generates sample files for releases
mix run               # Starts and runs the current application
mix test              # Runs a project's tests
mix xref              # Prints cross reference information
iex -S mix            # Starts IEx and runs the default task

Minimalist Clustering Demo

This is based on code from a video that I have previously linked to.
However Elixir has made a few breaking changes since then.

This is going to demonstrate how to cluster two local Elixir nodes, deploy code from one to the other and then upgrade it while it is running.

You need to put the following code in a file called blabber.ex

defmodule Blabber do

  def start do

    spawn(fn -> loop(0) end)

  end

  def loop(uptime) do

    receive do

      :stop ->

        IO.puts "Shutting blabber down ..."

        exit(:normal)

      after 1000 ->

        IO.puts "Nice! #{uptime} seconds of bgu-free uptime on #{node()}."

    end

    Blabber.loop(uptime + 1)

  end

end

Next you need to start iex with a name and a cookie (in the same folder as blabber.ex):

iex --sname cat --cookie super-secret

In a second terminal start another iex session

iex --sname dog --cookie super-secret

From the first terminal connect the two nodes together:

Node.connect :"dog@MacBook-Pro"

Compile the code in the cat node:

c ("blabber.ex", ".")

We need to use this version of the compile function as by default since Elixir 1.5 the beam file is not output by default.

The process can be started on the cat node with:

Blabber.start

Now you can transfer the compiled beam file to all linked nodes using:

nl Blabber

Now from the dog node you can start the process:

Blabber.start

At this point we notice the typo in the code. It say bgu-free rather than bug-free.
Edit the source file save the change.

On the cat node recompile this:

c ("blabber.ex", ".")

This will immediately fix the problem on the cat node (and leave the count updating).

If you use:

nl Blabber

then the solution will be moved to the other node as well.

This demonstrates the simplest possible Clustering arrangement. It is simple to extend this to a move sophisticated service.

Investigating NPM Dependencies

I have been working with a number of Node projects recently.

Keeping dependencies upto date is a big time sink. I use Dependabot to help with these.

Here is a utility that I have written that allows visualisation of module dependencies: https://github.com/chriseyre2000/package_compare

It loads node_modules into a Neo4j graph database.

To use this you need to install neo4j, create a database user with a password and the Erlang OTP runtime.

Here are the important details:

Once you have run mix escript.build then you can use the following:

./package_compare path-to-the/package.json localhost neo4j_username neo4j_password

This can be run across multiple projects to compare the dependencies. Once you have loaded multiple applications you can you the simple query:

MATCH (a) RETURN a

This will allow you to find the core set of dependencies that your applications are using. If two projects have a large core then there may be a common library waiting to be extracted.

This is an example of an Elixir escript application. This takes an unusual approach with the Sips library, it uses start link itself so that the database configuration can be supplied on the command line. Normally this would be started as a dependent application and the config found from a config file.

Designing Elixir Systems With OTP – Part Three

The B2 edition of this book is out. The new chapter is dedicated to Supervisors. These are correctly described as Lifecycle components.

This is the first time that I have seen the new DynamicSupervisors being used.

Minor note the parameter in

Lifecycle/lib/mastery/boundary/quiz_session.ex for select_session/1 has changed to name from session.

Similarly for answer_question/2

The code samples now clearly state where the code is to be added.

There was a minor typo and some missing code that breaks the final iex session.

Current code can be found at: https://github.com:chriseyre2000/designing_elixir

Elixir iex documentation : C

This is the third part of an exploration of the Elixir base libraries

Here is an introduction to what you get out of the box with iex:

LetterModuleDescription
CCalendarDate, time and calendar functions
CaseClauseErrorThis is a struct used for errors.
CodeUtilities for managing code compilation, code
evaluation, and code loading.
CollectableProtocol to traverse data structures.
Used for putting data into things.
CompileErrorThis is a struct used for errors.
CondClauseErrorThis is a struct used for errors.

Doing the same with :c gives the erlang modules

LetterModuleDescription
ccErlang compiler.
calendarDate, time and calendar functions
cerlCore Erlang abstract syntax trees.
cerl_clausesUtility functions for Core Erlang
case/receive clauses.
cerl_inline???
cerl_setsSet functions.
cerl_treesBasic functions on Core Erlang abstract syntax trees.
codeUtilities for managing code compilation, code
evaluation, and code loading.
code_serverProcess used for live code reloading
compileErlang compiler interface
core_lib???
core_lint
core_parse
core_pp
core_scan
countersLibrary of mutable counters with no locking.

Elixir iex documentation : B

This is the second part of an exploration of the Elixir base libraries

Here is an introduction to what you get out of the box with iex:

LetterModuleDescription
BBadArityErrorThis is a struct used for errors.
BadBooleanErrorThis is a struct used for errors.
BadFunctionErrorThis is a struct used for errors.
BadMapErrorThis is a struct used for errors.
BadStructErrorThis is a struct used for errors.
BaseEncoding and decoding functions
for 16, 32, 64, hex, url
BehaviourMechanism for handling behaviours
(deprecated)
BitwisePerforms bit calculations.

Doing the same with :b gives the erlang modules

LetterModuleDescription
bbase64Base 64 encode and decode
beam_aUsed after code generation before
optimisation. Normalises the code.
beam_asmAssembler for the beam
beam_blockErlang compiler
beam_bsErlang compiler
beam_bsmErlang compiler
beam_cleanErlang compiler
beam_deadErlang compiler
beam_dictErlang compiler
beam_disamErlang compiler
beam_exceptErlang compiler
beam_flattenErlang compiler
beam_jumpErlang compiler
beam_libErlang compiler
beam_listingErlang compiler
beam_opcodesErlang compiler
beam_peepErlang compiler
beam_receiveErlang compiler
beam_recordErlang compiler
beam_reorderErlang compiler
beam_splitErlang compiler
beam_trimErlang compiler
beam_typeErlang compiler
beam_utilsErlang compiler
beam_validatorErlang compiler
beam_zErlang compiler
binaryHandles binary data.
This is the Erlang string type

Elixir iex documentation : A

I recently found out how powerful the iex tool is for documentation.

This is the general form of getting detailed documentation:

This is the start of a long series about the modules that you get out of the box.
1> h Module.function_name/arity

This returns the documentation on this specific function in the specific module.

I also found that typing a capital letter and tab expands the known modules.

Here is an introduction to what you get out of the box with iex:

LetterModuleDescription
AAccessKey based access to structures.
AgentSimple abstraction around state
ApplicationWorking with applications and their callbacks
ArgumentErrorThis is a struct used for errors.
ArithmeticError This is a struct used for errors.
AtomConvenience functions for working with atoms.

Translating Elixir/Erlang terms:

An application is equivalent of a windows DLL, something that you compose a system out of. A typical application would be a logging system.

Atoms are a non-garbage collected resource. They make convenient aliases to global things.

Doing the same with :a gives the erlang modules

LetterModuleDescription
aapplicationGeneric OTP Application
application_controllerKernel supplied application controller
application_masterKernel supplied application master.
Responsible for knowing the topmost
supervisor
application_starterResponsible for starting applications in
a defined order
arrayArray structure, fixed or extensible
atomicsProvides atomic operations using
hardware instructions
authErlang network authentication server
(deprecated)

Designing Elixir Systems With OTP – Part Two

I have continued working through this beta book and have updated the github repo: github.com/chriseyre2000/designing_elixir with the samples.

So far this book is easier to follow than say Programming Phoenix since it does not keep changing the code that you are working on. The testing code is normally close to where you have typed it, so finding mistakes does not require too much effort. I would recommend using mix test after each code sample has been typed in as fixing errors can be tricky.

Note the only difference between my code and the supplied sample is the use of the british name Maths rather than Math for the sample code. My degree is in Mathematics so I can’t let a typo like that stand!

The main points from the second half of the book are the very clean validation techniques used. I like that they called the process returned by start_link a session.

The book currently stops at the end of chapter 6, so I will have to resume work on this series once it has been updated.