Generative Programming: Wardley Map Analysis

The book “Generative Programming Methods Tools and Applications” was published in 2000. It made the hypothesis that software would follow the same pattern that car manufacturing does.

In the highly automated car production pipeline it is possible to order a highly customised car yet have it built on a stardard production line from standard parts. GP:MTA assumed that this would soon be possible in software.

Dollard 2005 [Code Generation in dotnet] provides some univetsal priniples for code generation that apply irrespective of the implementation language.

We now have the Wonder articles (phase I) but the process stalled there. Dollard 2005 gives the techniques needed. See github.com/chriseyre2000/perfectstorm for an old but distinct implementation of Dollards ideas.

These techniques did not get the required level of adoption so that reusable templates could be produced. This was left at the initial stages of evolution.

In order to be taken further it will require some big adoption that makes it to a confrence paper. This could bootstrap the reusable catalog idea.

Code generation like this is not a 4GL, merely a means of extracting the business details from the implementation technology. You can achieve reuse without using up key options (single inheretence languages can avoid using up this powerful tool).

Minimise noise in logs

Try to keep the logs quiet. If something is happening we need to know. Logs can tell you that things are happening or have gone wrong. A request for intervention should be made once, pteferably by opening an alert on OpsGenie

Try to keep error messages clean. If possible treat warnings as errors. Noisy compile messages will hide real errors.

The Royal Queue

This was an inspired and truely British thing to do to form a queue to pay tribute to the late Queen. Currently it is a 10 mile queue that can take upto 24 hours to complete.

Queues as entertainment are not new as Legoland has been selling this for years (one visit had 7 rides each preceded by a one hour queue). This is however the first time we have had a televised queue with its own TV feed and incinsistently run twitter feed. At one point the queue was closed on twitter, but failed on two counts: they did not tell the people at the start of the queue, and when they did is it resulted in the formation of a second unofficial queue.

Self organising queue management is something inherent in the British character. Americans are amazed that post office queues work without anyone policing them. This is a single queue with several servers handling it. These form in many stores and self manage.

CI Issue

In most CI setups a build process is triggered on push to a PR (or a given pattern for the name of the branch).

To avoid having to fetch everything anew it it common to cache succesful runs of the main branch and use that at the start of each CI run.

This works fine as long as the PR is ahead of main. However in active teams (or in a monorepo with many teams) it is easy to get behind. This can cause odd effects. Typically this presents as errors that exist on CI that don’t exist locally and vanish when the branch is updated.

One solution would to cache per branch but initialise the cache to the main.

Another posibility is to datestamp the main cache and only use the most recent one older than the current branch.

Both of these require careful cache management.

kino_wardley version 0.7.0

The new version was published to hex.pm last night. It includes markets.

Wardley Map showing markets and inertia

This was generated using the following:

KinoWardley.Output.new("""
title New Features
style wardley
market Market [0.5, 0.4] label [-80, 0] inertia
evolve Market 0.6 label [20, 0]
note This shows markets with evolution and Inertia [0.2, 0.25]
""")

Only a few features required to achieve parity with onlinewardleymaps.com

Missing items include:
– pipelines
– build/buy/outsource markers

Small Focused Unit/Integration Tests

Don’t try and test everything in a single test. Each test should have a single reason to fail.
If you are using infrastructure in your integration tests it is worth having some simple tests that check that it is present.
These are simple to write and quick to run and will make the rest of the test suit far more stable.

Assertions about infrastructure is about is that S3 bucket present. Have I created that database. Is the database at least this version. These are assertions that would be far too unstable to have in a distributed manner. In fact these should explain how to setup or configure the resource when they fail. Having a test framework with multiline strings can give you a way to tell future people (including you) how to fix something.

Royal Proclamation

Yesterday I attended the local Royal Proclamation. This is a public event that repeats the official statement made in Westminister the day before.

This is a tradition that predates modern communication methods, but does seem a little antiquated. Having a Town Crier introduce the local Mayor (flanked by his mace wielder – more of a golden hammer).

The pronouncement was also dated. It presumed that the entire crowd was Christian (which is now not true for the majority of the population). Having public collective worship is something I have not seen since school (outside of Weddings and Funerals).

They expected the crowd to sing the first verse of the national anthem (there was deliberate enouciation of King) shout God Save The King and recite the Lords Prayer. Most did.

Overall a very odd event.

Thoughts on CI/CD

Moving to CD should not be hard, but it is a change in mindset.

You will accidentally release something you shouldn’t. You need a fast rollback/forward process. Long lived caches can be an enemy here.

You need to think small changes. Deploy that then work on the next part.

Avoid excessive git merges as this can hide what you are deploying. Having a small unit test that fails if your code has been lost in a merge is a good safety net.

Feature Flags are your friend. Keep them positive, clearly named and not nested. Prune them aggressively. If you are using AB tests write your code so one side or other can be easily deleted after a test.