This is a great intro to both Elixir and Docker: https://www.poeticoding.com/running-elixir-in-docker-containers/
It covers networking and running distributed elixir across docker containers.
Replacing a MongoDB with a Lambda
The project that I currently work on uses a number of microservices. Several of these microservices use MongoDB. The platform provider that we are using to host these databases (compose.io) has announced that they are going to stop supporting Mongo in about two months time.
This has resulted in a thorough investigation of the mongo databases that we are using.
Several of the services use the following architecture:
This has three instances of an application that periodically poll a website before performing sending the result to rabbitmq. They use the mongodb as a lock so that only one of the three perform the operation.
The alternative design was to add an external scheduler to trigger the nodes. The load balancer would pick one of the nodes to perform the work. One of my colleagues pointed out that we could use a lambda to do this. Lambdas are (typically) javascript functions that are cloud hosted.
It it is possible to deploy a lambda with a schedule of how frequently it should be invoked, either with a given frequency or via a CRON-like specification. The lambdas invoke methods on the nodes that asynchronously trigger the process.
Here is a diagram of the replacement:
This has allowed us to replace three MongoDB’s in each of four environments with a much simpler lambda function.
In addition we have been able to remove the mongodb database driver and the internal cron scheduling components. This greatly simplifies these services. There is now less to maintain (less PR’s from dependabot). It is also far easier to test code that is called from an endpoint than one from a scheduled event.
We do still need to replace the remaining MongoDB’s which contain real data but this is a much more manageable job.
We also learned that AWS still uses Node 8 LTS for Lambda.
This is a great starting point for serverless functions.
Loading package.json into Neo4j
I have written a small utility to take a node package.json file and load it into a neo4jdatabase:
https://github.com/chriseyre2000/package_compare
It’s a simple Elixir escript that creates a command line tool.
It can be run across multiple projects (or multiple versions of a project) to visualise and compare the dependencies.
Here is the displayed output of a single project loaded into neo4j:
Announcing About
Last weekend I wrote and released the Elixir package called “about”.
This adds About topics to the iex environment.
It is installed in the usual way:
Add {:about, "~> 0.0.1"} to your deps list in mix.exs
Run mix deps.get
Run iex -S mix
You can now use help topics like:
iex(1)> h About.pipeline
I am looking for suggestions/contributions to the project.
It’s hosted on https://github.com/chriseyre2000/about and available on hex.pm and hexdocs.
Designing Elixir Systems With OTP – Part Three
The B2 edition of this book is out. The new chapter is dedicated to Supervisors. These are correctly described as Lifecycle components.
This is the first time that I have seen the new DynamicSupervisors being used.
Minor note the parameter in
Lifecycle/lib/mastery/boundary/quiz_session.ex for select_session/1 has changed to name from session.
Similarly for answer_question/2
The code samples now clearly state where the code is to be added.
There was a minor typo and some missing code that breaks the final iex session.
Current code can be found at: https://github.com:chriseyre2000/designing_elixir
Elixir iex documentation : C
This is the third part of an exploration of the Elixir base libraries
Here is an introduction to what you get out of the box with iex:
| Letter | Module | Description |
| C | Calendar | Date, time and calendar functions |
| CaseClauseError | This is a struct used for errors. | |
| Code | Utilities for managing code compilation, code evaluation, and code loading. | |
| Collectable | Protocol to traverse data structures. Used for putting data into things. | |
| CompileError | This is a struct used for errors. | |
| CondClauseError | This is a struct used for errors. |
Doing the same with :c gives the erlang modules
| Letter | Module | Description |
| c | c | Erlang compiler. |
| calendar | Date, time and calendar functions | |
| cerl | Core Erlang abstract syntax trees. | |
| cerl_clauses | Utility functions for Core Erlang case/receive clauses. | |
| cerl_inline | ??? | |
| cerl_sets | Set functions. | |
| cerl_trees | Basic functions on Core Erlang abstract syntax trees. | |
| code | Utilities for managing code compilation, code evaluation, and code loading. | |
| code_server | Process used for live code reloading | |
| compile | Erlang compiler interface | |
| core_lib | ??? | |
| core_lint | ||
| core_parse | ||
| core_pp | ||
| core_scan | ||
| counters | Library of mutable counters with no locking. |
Elixir iex documentation : B
This is the second part of an exploration of the Elixir base libraries
Here is an introduction to what you get out of the box with iex:
| Letter | Module | Description |
| B | BadArityError | This is a struct used for errors. |
| BadBooleanError | This is a struct used for errors. | |
| BadFunctionError | This is a struct used for errors. | |
| BadMapError | This is a struct used for errors. | |
| BadStructError | This is a struct used for errors. | |
| Base | Encoding and decoding functions for 16, 32, 64, hex, url | |
| Behaviour | Mechanism for handling behaviours (deprecated) | |
| Bitwise | Performs bit calculations. |
Doing the same with :b gives the erlang modules
| Letter | Module | Description |
| b | base64 | Base 64 encode and decode |
| beam_a | Used after code generation before optimisation. Normalises the code. | |
| beam_asm | Assembler for the beam | |
| beam_block | Erlang compiler | |
| beam_bs | Erlang compiler | |
| beam_bsm | Erlang compiler | |
| beam_clean | Erlang compiler | |
| beam_dead | Erlang compiler | |
| beam_dict | Erlang compiler | |
| beam_disam | Erlang compiler | |
| beam_except | Erlang compiler | |
| beam_flatten | Erlang compiler | |
| beam_jump | Erlang compiler | |
| beam_lib | Erlang compiler | |
| beam_listing | Erlang compiler | |
| beam_opcodes | Erlang compiler | |
| beam_peep | Erlang compiler | |
| beam_receive | Erlang compiler | |
| beam_record | Erlang compiler | |
| beam_reorder | Erlang compiler | |
| beam_split | Erlang compiler | |
| beam_trim | Erlang compiler | |
| beam_type | Erlang compiler | |
| beam_utils | Erlang compiler | |
| beam_validator | Erlang compiler | |
| beam_z | Erlang compiler | |
| binary | Handles binary data. This is the Erlang string type |
Elixir iex documentation : A
I recently found out how powerful the iex tool is for documentation.
This is the general form of getting detailed documentation:
This is the start of a long series about the modules that you get out of the box.
1> h Module.function_name/arity
This returns the documentation on this specific function in the specific module.
I also found that typing a capital letter and tab expands the known modules.
Here is an introduction to what you get out of the box with iex:
| Letter | Module | Description |
| A | Access | Key based access to structures. |
| Agent | Simple abstraction around state | |
| Application | Working with applications and their callbacks | |
| ArgumentError | This is a struct used for errors. | |
| ArithmeticError | This is a struct used for errors. | |
| Atom | Convenience functions for working with atoms. |
Translating Elixir/Erlang terms:
An application is equivalent of a windows DLL, something that you compose a system out of. A typical application would be a logging system.
Atoms are a non-garbage collected resource. They make convenient aliases to global things.
Doing the same with :a gives the erlang modules
| Letter | Module | Description |
| a | application | Generic OTP Application |
| application_controller | Kernel supplied application controller | |
| application_master | Kernel supplied application master. Responsible for knowing the topmost supervisor | |
| application_starter | Responsible for starting applications in a defined order | |
| array | Array structure, fixed or extensible | |
| atomics | Provides atomic operations using hardware instructions | |
| auth | Erlang network authentication server (deprecated) |
Working through “Property Based Testing in PropEr with Erlang and Elixir” in Elixir – Part 2
I have returned to this book now that the book is out of beta.
I also started using a different repo: https://github.com/chriseyre2000/property_testing
The only difference between my code and the samples is that I am using the pipe syntax where possible. My Execism.IO background in Elixir is also forcing me to make style changes to keep the code in Elixir format.
Part 1 chapter 4 does use the erlang queue library. It would be good to be able to get Elixir documentation on erlang packages.
The part 2 sample code is broken but I am working on fixing in my repo.
It turned out to be easier to download the code from the pragprog site.
Designing Elixir Systems With OTP – Part Two
I have continued working through this beta book and have updated the github repo: github.com/chriseyre2000/designing_elixir with the samples.
So far this book is easier to follow than say Programming Phoenix since it does not keep changing the code that you are working on. The testing code is normally close to where you have typed it, so finding mistakes does not require too much effort. I would recommend using mix test after each code sample has been typed in as fixing errors can be tricky.
Note the only difference between my code and the supplied sample is the use of the british name Maths rather than Math for the sample code. My degree is in Mathematics so I can’t let a typo like that stand!
The main points from the second half of the book are the very clean validation techniques used. I like that they called the process returned by start_link a session.
The book currently stops at the end of chapter 6, so I will have to resume work on this series once it has been updated.
Developer Rants 