Handling a Headless CMS

This is a general outline for using a headless CMS (such as Contentful).

There are other ways to do this but this is fast given a suitable size delta, and will recover from failure better than webhooks:

This has the headless CMS being polled by a Static Site Generator.

The Static Site Generator knows the current state of the site and only regenerates changed items. This may require it to keep a map of dependencies between nodes. Sometimes you may need to go 3 or 4 levels deep. Store the output in S3. Serve the website from S3, which allows some customisation of the data. Use a CDN to back the website to reduce load. Purge the pages of the site that have changed after they have been written.

We managed to use a single Heroku node for the SSG and another one for the webserver.

Groovy Closures

Some of my team have been impressed with Groovy’s support for closures:

Here is a simple closure:

def time(String message, Closure body) {
  def start = Instant.now()
  def result = body.call()
  def finish = Instant.now()
  Duration duration = Duration.between(start, finish)
  log.info("$message took $duration")
  result
}

This would allow you to go from:

def value = complexDatabaseQuery(1, 2, 3)

to

def msg = "Complex query 1, 2, 3"
def value = time(msg, {complexDatabaseQuery(1, 2, 3)})

This provides a simple decorating wrapper that records the time that the database call took without making the code less readable.

This could be made more sophisticated using:

def warnSlow(String message, def max_duration_in_seconds = 5, Closure body) {
def start = Instant.now()
def result = body.call()
def finish = Instant.now()
Duration duration = Duration.between(start, finish)
if (duration.getSeconds() > max_duration_in_seconds) {
warn “$message is slow, it took $duration which is longer than the threshold of $max_duration_in_seconds”
}
result
}

This permits simple performance checks to be added to code with minimal disruption.

Elixir iex documentation : C

This is the third part of an exploration of the Elixir base libraries

Here is an introduction to what you get out of the box with iex:

LetterModuleDescription
CCalendarDate, time and calendar functions
CaseClauseErrorThis is a struct used for errors.
CodeUtilities for managing code compilation, code
evaluation, and code loading.
CollectableProtocol to traverse data structures.
Used for putting data into things.
CompileErrorThis is a struct used for errors.
CondClauseErrorThis is a struct used for errors.

Doing the same with :c gives the erlang modules

LetterModuleDescription
ccErlang compiler.
calendarDate, time and calendar functions
cerlCore Erlang abstract syntax trees.
cerl_clausesUtility functions for Core Erlang
case/receive clauses.
cerl_inline???
cerl_setsSet functions.
cerl_treesBasic functions on Core Erlang abstract syntax trees.
codeUtilities for managing code compilation, code
evaluation, and code loading.
code_serverProcess used for live code reloading
compileErlang compiler interface
core_lib???
core_lint
core_parse
core_pp
core_scan
countersLibrary of mutable counters with no locking.

Elixir iex documentation : B

This is the second part of an exploration of the Elixir base libraries

Here is an introduction to what you get out of the box with iex:

LetterModuleDescription
BBadArityErrorThis is a struct used for errors.
BadBooleanErrorThis is a struct used for errors.
BadFunctionErrorThis is a struct used for errors.
BadMapErrorThis is a struct used for errors.
BadStructErrorThis is a struct used for errors.
BaseEncoding and decoding functions
for 16, 32, 64, hex, url
BehaviourMechanism for handling behaviours
(deprecated)
BitwisePerforms bit calculations.

Doing the same with :b gives the erlang modules

LetterModuleDescription
bbase64Base 64 encode and decode
beam_aUsed after code generation before
optimisation. Normalises the code.
beam_asmAssembler for the beam
beam_blockErlang compiler
beam_bsErlang compiler
beam_bsmErlang compiler
beam_cleanErlang compiler
beam_deadErlang compiler
beam_dictErlang compiler
beam_disamErlang compiler
beam_exceptErlang compiler
beam_flattenErlang compiler
beam_jumpErlang compiler
beam_libErlang compiler
beam_listingErlang compiler
beam_opcodesErlang compiler
beam_peepErlang compiler
beam_receiveErlang compiler
beam_recordErlang compiler
beam_reorderErlang compiler
beam_splitErlang compiler
beam_trimErlang compiler
beam_typeErlang compiler
beam_utilsErlang compiler
beam_validatorErlang compiler
beam_zErlang compiler
binaryHandles binary data.
This is the Erlang string type

Elixir iex documentation : A

I recently found out how powerful the iex tool is for documentation.

This is the general form of getting detailed documentation:

This is the start of a long series about the modules that you get out of the box.
1> h Module.function_name/arity

This returns the documentation on this specific function in the specific module.

I also found that typing a capital letter and tab expands the known modules.

Here is an introduction to what you get out of the box with iex:

LetterModuleDescription
AAccessKey based access to structures.
AgentSimple abstraction around state
ApplicationWorking with applications and their callbacks
ArgumentErrorThis is a struct used for errors.
ArithmeticError This is a struct used for errors.
AtomConvenience functions for working with atoms.

Translating Elixir/Erlang terms:

An application is equivalent of a windows DLL, something that you compose a system out of. A typical application would be a logging system.

Atoms are a non-garbage collected resource. They make convenient aliases to global things.

Doing the same with :a gives the erlang modules

LetterModuleDescription
aapplicationGeneric OTP Application
application_controllerKernel supplied application controller
application_masterKernel supplied application master.
Responsible for knowing the topmost
supervisor
application_starterResponsible for starting applications in
a defined order
arrayArray structure, fixed or extensible
atomicsProvides atomic operations using
hardware instructions
authErlang network authentication server
(deprecated)

Working through “Property Based Testing in PropEr with Erlang and Elixir” in Elixir – Part 2

I have returned to this book now that the book is out of beta.

I also started using a different repo: https://github.com/chriseyre2000/property_testing

The only difference between my code and the samples is that I am using the pipe syntax where possible. My Execism.IO background in Elixir is also forcing me to make style changes to keep the code in Elixir format.

Part 1 chapter 4 does use the erlang queue library. It would be good to be able to get Elixir documentation on erlang packages.

The part 2 sample code is broken but I am working on fixing in my repo.

It turned out to be easier to download the code from the pragprog site.

Designing Elixir Systems With OTP – Part Two

I have continued working through this beta book and have updated the github repo: github.com/chriseyre2000/designing_elixir with the samples.

So far this book is easier to follow than say Programming Phoenix since it does not keep changing the code that you are working on. The testing code is normally close to where you have typed it, so finding mistakes does not require too much effort. I would recommend using mix test after each code sample has been typed in as fixing errors can be tricky.

Note the only difference between my code and the supplied sample is the use of the british name Maths rather than Math for the sample code. My degree is in Mathematics so I can’t let a typo like that stand!

The main points from the second half of the book are the very clean validation techniques used. I like that they called the process returned by start_link a session.

The book currently stops at the end of chapter 6, so I will have to resume work on this series once it has been updated.

Designing Elixir Systems With OTP – Part One

These are my notes as I am working my way through this book from PragProg. I am reading a beta version of the book so things may change. I’ll be posting this to my github repo: (https://github.com/chriseyre2000/designing_elixir)

The introduction starts by building a simple counter application as the domain and wraps it in a send/receive process with an API.

The code samples here are clean and clearly ordered. The only note that I have found so far is that I needed to delete the autogenerated Count.ex file to avoid a warning.

Takeaway from the intro: Supervisors are about lifecycles.

Takeaway from type introductions: Send the functions to the data!

Chapter 3 typo: response.ex is misspelt (reported).

excoveralls looks to be a better code coverage tool.

Essential Docker and Docker-Compose Commands

I spent some of today trying to work out why a Docker image that I was using did not work correctly.

This is the most useful command for debugging a docker image:

docker run -it ubuntu /bin/bash

This is based upon the ubuntu image but feel free to replace it with whatever you are trying to debug.

You can do the same with a docker-compose

docker-compose run --entrypoint /bin/bash

I am now looking at how to run a gradle build inside a docker container and then extract the log files. It will be especially useful to be able to view the log files. It may be possible to add a web server to view them.

It’s now also possible to stand up a Jenkins server on your dev machine to test Jenkinsfiles before deploying them to the build server.

docker run -p 8080:8080 -p 50000:50000 jenkins/jenkins:lts

Designing for Scalability with Erlang/OTP using Elixir – Part 2

Chapter 4

I am still working my way through this book.

The main difficulty that I am finding with chapter 4 is the scattered nature of the example code. It’s all small snippets, some of which references code in the previous chapter.

This is in addition to the Erlang => Elixir translations:

  • atom => :atom
  • VariableName => variable_name
  • moduleName => ModuleName

Some of the Elixir functions are reordered for consistency.

#Erlang
1> gen_server:start_link({local, timeout}, timeout, [], []).

#Elixir
1> c "Timeout.ex"
2> GenServer.start_link(Timeout, [], name: :timeout)
3> GenServer.call(:timeout, {:sleep, 100})
4> GenServer.call(:timeout, {:sleep, 5001})

The GenServer will by default timeout if a message takes more than 5 seconds to respond.

Samples for Chapter 4 have been pushed to

https://github.com:chriseyre2000/scalability_ex with hash: cc8dd44

Chapter 5

The Erlang sys module can be reference from iex using :sys

The the log commands require atoms.

:sys.log(Frequency, :print)

:sys.log(Frequency, :get)

Now at hash: 830c1ea

Chapter 6

This is an odd chapter for Elixir as the Finite State Machine was considered defective and was not adapted as a wrapper. I am going to skip this chapter

Chapter 7

GenEvent has also been Deprecated