Recently I have been working on a new service. It has a single rest endpoint that triggers some work.

The tests for this use Hammox to mock out the external services that it talks to. This ensures that the tests always match the declared typespecs for the services.

This week I added ex_coveralls to the project to see how complete the test coverage was. It revealled several parts of the code that had missing tests. Adding those revealled a few small bugs (one of which had been found in production while I was writing the tests!).

The test coverage was approaching 100% in the main processing area. Combined with a suite of tests that produced all of the possible output states I was confident about performing some refactoring.

The refactors were to replace some tuples with structs and them to remove some pointless wrapping error tuples. Coveralls helped to reveal dead code paths that once removed achieved the 100% coverage.

The tests also helped when working out what could be changed. By adding a log statement at a key points I could see all possible inputs to a function. This helped in finding what to change when adapting.

Now I have simpler code that with the tests I have confidence in.

This is a great example of using a python library within Elixir:

https://github.com/nelsonmestevao/pdf_extractor/blob/main/lib/pdf_extractor/pdf_plumber.ex

This means we can easily turn a python cli application into a managed Elixir webservice.
If we don’t have a native Elixir version we are free to use a Python one.
Keep these in small services as it may not play well with a supervision tree!

Elixir finds lots of ways to work well with other languages!

I recently wrote a useful application that does not have its own UI. Your users probably already have enough primary applications to work with so adding another will make their life more complex.

Instead the application writes the data it finds into the application they are already using. I have used this technique now with Contentful, Teams, and now Zendesk.

The trick is to combine webhooks from the application with some form of comment API to write what you need where the users are already working.

In this case I notice that a ticket has been submitted with some metadata and a well known pdf structure. The app reads the pdf, parses it and checks the details against what we have stored on our systems. It also writes both back to the ticket with a list of differences. This means that the user does not need to open the attachments (or our system) to start work.

This is not a language flamewar.

Each language has its focus area.

Rust wins on CPU bound operations while Elixir wins on IO bound ones.

I have been working on a small utility service in Elixir. It accepts a webhook to start. This spawns a process to do the work. The main work is a series of steps unified by a with block. The beauty is that it is all in small isolated files that you can see what it is doing. Having a repl in iex allows you to quickly experiment with services to see what is not working.

I am trying to read an equivalent Rust project to see how one of the system callx is used. Rust is high ceremony so there are many distinct declarations. The habbit of embedding the tests in the same files as the code makes searching for details far more painful than it needs to be.

Recently I reread Competitive engineering.

One of the interesting points that I remember this time is the figure given for unfound errors after a review. It argues that any review will only find 1/3 of the errors present.

This makes the case for many small reviews!

Development teams these days are broken down into 2 parts:

1. Engineering Manager
   • Leads the engineers
2. Product Manager
   • Leads the product people and designers

This is different to a few year back when we would have a distinct QA function.
Both sides get to input into the work to be done. The Engineering side keep the security and maintainability issues under control while product decide what new features should be added/changed.

This is a careful balancing act given a very restricted team size.

The M25 J5-J8 are having Speed Limit Tests applied. These seem to be highly irrational.
You start with a speed limit test, followed by another nonsense message such as congestion ahead or report of object in the road. These make no sense.

Apparently these tests need to be carried out when there are no roadworks. There are no roadworks this weekend because it is a bank holiday and they were removed to speed up traffic flow. Motorways work best when everyone is doing the same speed. Adding pointless 50mph sections encourages some drivers to ignore it leading to a 2 speed motorway which is clearly dangerous.

These tests have been going on for months. They are either poorly designed or ineffective.

DDD is intended to allow a large organisation to split itself into distinct domains. This allows parts to evolve and specialise apropriately.

Trandforming a company to this is not simple, especially if you are dealing with a multi-country company where each coutry has distinct needs.

There are several risks:

• Certain changes are slowed down as one team being too busy can block entire intiatives.
• Some issues can be seen by everyone to belong to someone else. This includes cross domain consustency issues.

Simon Wardley is writing a book Rewilding Software Engineering

The gist of this to use small specific tools to learn about behaviour or the structure if systems. This allows us to reduce the time to question and time to answer.

This led me to think where are the small tools that I have been using?

I lean heavily on diagrams. Mermaid, Plantuml and graphviz are my go tos. Other members of my team use Miro.

In the past spreadsheets would have covered this.

Another option that is useful in distributed systems are the observability tools. These can be used to find how frequently a known error condition has occoured.

We frequently build quick views to investigate production issues.

It is amazing how a few simple tools can allow you to build visualisations from data. I recently extracted the team and domain structure from our central security setup scripts. Its about 10 lines of sed/awk/grep/dot

In the past I have used neo4j to identify which project were currently using which version of our internal node libraries. This makes applying upgrades consistently across 80 git repos possible.

I have been adding a new feature to a graphql based application.

Normally there is a mix of frontend and serverside work required. The latest feature is an attempt to bring together disperate information in one place. For the first time I was able to add an entire feature purely in the front end.

This feels like a good solution.