Time Travel with a Message Queue

I was discussing the usages of message queues with some colleagues and surprised them with temporal coupling.

Message Queues perform buffers between applications so that they can process data at different rates. It also protects from outages allowing each side to be restarted or upgraded independently.

However most modern message queues have additional options such as the ability to send a message to the future! The message is sent hidden for a time period. The trick to to send a command that will rollback the operation you are just starting. If your process stops normally then this message can be deleted before it is read. If the process dies then the message will appear and can be used to clean up. This is like having a finally block that will survive a process restart!

Another trick is to send messages that are deleted after a given time. I have used this to have a log available that starts logging 5 minutes ago. This means that when a problem occurs you can capture the last 5 mins of events leading up to it. This can prove invaluable.

Golang is weird!

I have just managed to get Golang to use local packages.

Go takes the approach that all packages are to be considered to be remote even if they are not.
This means that you need to define your projects package location in go.mod as if it were installed from the remote repository that you will deploy it to. You can then import the local module as if it were from that location but it will resolve to a local copy.

package github.com/company-name/project-name/project-root

You can then import github.com/company-name/project-name/project-root/mymodule

from a ./mymodule folder in your project. All go files in that folder will be imported.

The practical advantage is that you can use the same sample code if the library is local or remote without needing a distinct bundling tool.

The go test command is also odd. It by default only runs the tests in the current folder.
You need to use go test. ./... to get all the tests to run (which will be all files named *_test.go in the filesystem.

This is not very intuitive.

Integrating Typescript and Frontend Javascript

Recently I have been working on a AngularJS/Angular hybrid application.
Part of this involve having parallel routers. This requires coordinating the two systems as to which routes should be Angular and which should be AngularJS

Typescript does not allow build time metaprogramming that can generate the required javascript.

The typescript compiler cannot generate browser js.

Angular CLI does not allow the creation of modules that are not used for dynamic loading.

I considered using xslt to create both files that we needed.

In the end I used a json file that is checked by the typescript tests and put in the assets area so deployed by the Angular CLI.

Golang Magical Reference Date

Go solves the date parsing by using a magical reference date.

You start with this date

Mon Jan 2 15:04:05 MST 2006

Adjust it into whatever format that you use and use it for parsing dates.
This is neat solution. It would be even better if the go documentation for time.Parse were to mention it.

I found the solution here: https://programming.guide/go/format-parse-string-time-date-example.html

This has now been raised as https://github.com/golang/go/issues/48757

Thoughts on Golang

I am currently studying Golang as I may need to use it in an upcoming project.

So far it is C-- `as it is C with certain dangerous features removed.

There is no macro system. You can’t do pointer arithmetic.

The compiler is not quite as helpful as some languages that I have used recently (Elm and Elixir).

It’s not object-oriented or functional. You get old school procedural programming.

Given that it is compiled there is no repl to help. The documentation is a bit restricted. For example when looking at date parsing there are no examples on building custom strings.

It is fast and handles complex numbers natively.

Not yet reached using goroutines.

Thoughts on Security

It is almost impossible to secure a web app. You cannot tell if a browser is being used or a script simulating it. Anything that can be called from the browser should be treated as a public API (with the exception of insisting on backwards compatibility).

This means any endpoint reachable from the client must be treated as if the user is directly calling it. This means that you need to consider authentication (who is the user) and authorisation (what the user is allowed to do). Trust no-one!

This means that all endpoints need to be tested at the API level as well as via the UI.

All those bundled minified js bundles that you have carefully constructed form excellent documentation of your exposed APIs. All the endpoints exist as neat little strings among the minified code.

Bus Factor

The Bus Factor for a project is the number of people that if hit by a bus (or equivalent leave) would break a project.

In an ideal situation there are sufficient runbooks and documentation for a project to continue should all the team members leave. This is frequently not the case.

Pairing can help reduce the Bus Factor. Two people have context on most things. This can still go wrong with the minimum effective team size (3). METS allows for one to be on holiday, one to be sick and still have someone to continue some form of progress. The downside is that someone will not be pairing (or the team can only mob on a single item).

You can’t eliminate the Bus Factor but need to reduce it. If anyone has too much information in their heads then they need to start actively writing down more than they would normally do. This means carefully documenting how to do anything that only they need to know.

This can seem counterproductive if having special knowledge makes you feel special. However being unreplaceable also means being unpromotable as the essential knowledge that you hoard will become a burden.

How big is your projects Bus Factor?

JavaScript Options

JavaScript is currently the default for interaction on web pages. Just because the page loads JavaScript it does not limit you to using JavaScript to write it.

The simple option is just to write JavaScript. This works an is the common approach.

The next step is to use Babel to transpile various extensions into JavaScript. This allows the use of language features that the browsers don’t yet support. Hopefully the near future loss of support of Internet Explorer 11 will allow this process to become easier. This is the typical option for React. This allows non-javascript components to be translated.

The next option is to move to Typescript. This is a backwards compatible language that adds some structure to JavaScript. This makes refactoring easier and adds some type information. This only exists at compile time. This is the technique used by most Angular applications.

There is a further option to move to other languages that compile to JavaScript. Here I am going to consider Elm, but other options exist (PureScript, Gleam). Elm is specifically designed for frontend applications. It’s a strongly typed functional language that makes strong claims about eliminating runtime errors. It has its own model based UI tools. It won’t eliminate logic errors but will work hard to give you it’s famously useful error messages.

Elm uses expressions rather than statements. This means that you always need an else if when you use if. Expressions always return something. It is a compile error to omit this. The language also has strong opinions on tab Vs spaces, as a tab in code whitespace will give a compile error. Lists need to be of a uniform type. Tuples can only have 2 or 3 elements (this constraint will greatly simplify APIs).

Strategic Domain Driven Design

I have recently been helping with a series of discovery workshops. The aim was to make recommendations on the future architecture of the system.

Several of these involved using concepts from DDD. In particular the workshops identified the bounded contexts and built a context map between them. Occasionally we described the entities within them, but these were mostly used to clarify the boundaries between systems.

For strategic analysis the ideas in second half of Evans DDD was of more interest than the first tactical part. Here we didn’t have a greenfield environment, but were trying to find a way to improve a certain part of the process.

The concept of separating the problem space from the solution space helps here. To design a logical architecture requires that the system be described in terms of the problem rather than the (possibly) existing solution. This means there can be no reference to databases, proxies or internal system names.