Random outpourings of a software developer
Releases are the most complete deployment option for an Elixir project.
There are several ways to deploy an Elixir application:
The last of these is the most robust solution which will permit clean upgrades.
The term Application in Elixir is akin to a library in other languages.
An Elixir solution is more of a mini operating system with processes.
We have a Conservative leadership election in the wake of the EU election results.
Here are the facts:
The EU will not reopen negotiations.
Our current extension will run out at Halloween.
There are three options for parliament:
The first has been repeatedly rejected by parliament.
The third will not be able to win over the house of commons, but will be the default should nothing be agreed.
The Brexit party would like to be part of negotiations but since the EU are not interested that won’t help.
Something needs to move, and that will either take a fresh election or a referendum.
We have a weak opposition (unable to decide if it is pro or anti Brexit) and a minority government unable to act.
The Conservatives will consider a Step to the Right (timewarp anyone?) to appeal to the Brexit party voters.
The Brexit party had no manifesto so its impossible to know what form of Brexit that they want. The best guess would be the No Deal option.
The UK faces the prospect of a new Prime Minister selected from a list that is packed with entirely unsuitable candidates:
| Candidate | Reason |
| Boris Johnson | Mr Johnson is facing a private prosecution over claims he deliberately lied during the Referendum campaign. Mr Johnson is reported to have replied: “Fuck business.” |
| Michael Gove | I think the people in this country have had enough of experts. Stabbed Boris Johnson in the front during the previous leadership campaign. |
| Jeremy Hunt | Messed up security contracts at the Olympics. Hated by the NHS. Can’t remember which country his wife is from. |
| Andrea Leadsom | “Being a mother gives me an edge on May” Withdrew from ballot versus May |
| Ester McVey | Misled parliament over the new Universal Credit scheme |
| Dominic Raab | Surprised that Britain is an island |
There are other candidates (and more will appear over the next few days).
This is a general outline for using a headless CMS (such as Contentful).
There are other ways to do this but this is fast given a suitable size delta, and will recover from failure better than webhooks:
This has the headless CMS being polled by a Static Site Generator.
The Static Site Generator knows the current state of the site and only regenerates changed items. This may require it to keep a map of dependencies between nodes. Sometimes you may need to go 3 or 4 levels deep. Store the output in S3. Serve the website from S3, which allows some customisation of the data. Use a CDN to back the website to reduce load. Purge the pages of the site that have changed after they have been written.
We managed to use a single Heroku node for the SSG and another one for the webserver.
Some of my team have been impressed with Groovy’s support for closures:
Here is a simple closure:
def time(String message, Closure body) {
def start = Instant.now()
def result = body.call()
def finish = Instant.now()
Duration duration = Duration.between(start, finish)
log.info("$message took $duration")
result
}
This would allow you to go from:
def value = complexDatabaseQuery(1, 2, 3)
to
def msg = "Complex query 1, 2, 3"
def value = time(msg, {complexDatabaseQuery(1, 2, 3)})
This provides a simple decorating wrapper that records the time that the database call took without making the code less readable.
This could be made more sophisticated using:
def warnSlow(String message, def max_duration_in_seconds = 5, Closure body) {
def start = Instant.now()
def result = body.call()
def finish = Instant.now()
Duration duration = Duration.between(start, finish)
if (duration.getSeconds() > max_duration_in_seconds) {
warn “$message is slow, it took $duration which is longer than the threshold of $max_duration_in_seconds”
}
result
}
This permits simple performance checks to be added to code with minimal disruption.
This is a great intro to both Elixir and Docker: https://www.poeticoding.com/running-elixir-in-docker-containers/
It covers networking and running distributed elixir across docker containers.
The project that I currently work on uses a number of microservices. Several of these microservices use MongoDB. The platform provider that we are using to host these databases (compose.io) has announced that they are going to stop supporting Mongo in about two months time.
This has resulted in a thorough investigation of the mongo databases that we are using.
Several of the services use the following architecture:
This has three instances of an application that periodically poll a website before performing sending the result to rabbitmq. They use the mongodb as a lock so that only one of the three perform the operation.
The alternative design was to add an external scheduler to trigger the nodes. The load balancer would pick one of the nodes to perform the work. One of my colleagues pointed out that we could use a lambda to do this. Lambdas are (typically) javascript functions that are cloud hosted.
It it is possible to deploy a lambda with a schedule of how frequently it should be invoked, either with a given frequency or via a CRON-like specification. The lambdas invoke methods on the nodes that asynchronously trigger the process.
Here is a diagram of the replacement:
This has allowed us to replace three MongoDB’s in each of four environments with a much simpler lambda function.
In addition we have been able to remove the mongodb database driver and the internal cron scheduling components. This greatly simplifies these services. There is now less to maintain (less PR’s from dependabot). It is also far easier to test code that is called from an endpoint than one from a scheduled event.
We do still need to replace the remaining MongoDB’s which contain real data but this is a much more manageable job.
We also learned that AWS still uses Node 8 LTS for Lambda.
This is a great starting point for serverless functions.
I have written a small utility to take a node package.json file and load it into a neo4jdatabase:
https://github.com/chriseyre2000/package_compare
It’s a simple Elixir escript that creates a command line tool.
It can be run across multiple projects (or multiple versions of a project) to visualise and compare the dependencies.
Here is the displayed output of a single project loaded into neo4j:
Last weekend I wrote and released the Elixir package called “about”.
This adds About topics to the iex environment.
It is installed in the usual way:
Add {:about, "~> 0.0.1"} to your deps list in mix.exs
Run mix deps.get
Run iex -S mix
You can now use help topics like:
iex(1)> h About.pipeline
I am looking for suggestions/contributions to the project.
It’s hosted on https://github.com/chriseyre2000/about and available on hex.pm and hexdocs.
The B2 edition of this book is out. The new chapter is dedicated to Supervisors. These are correctly described as Lifecycle components.
This is the first time that I have seen the new DynamicSupervisors being used.
Minor note the parameter in
Lifecycle/lib/mastery/boundary/quiz_session.ex for select_session/1 has changed to name from session.
Similarly for answer_question/2
The code samples now clearly state where the code is to be added.
There was a minor typo and some missing code that breaks the final iex session.
Current code can be found at: https://github.com:chriseyre2000/designing_elixir
Developer Rants 