I am working on integrating with a large system. This is one of those applications that my users spend half their time using,

There are two main approaches to integrating with systems like that.
One is to extract the data and present it inside the system you have. This means that the users have to use both the system they are using and yours.

The second approach is to push your data into that large system. There are several ways of doing this.
Embed a custom UI inside the system (if it allows for that). Another alternative is to use some form of commenting system as a UI. If you can write an HTML table then you can embed whatever data you need in the UI, This will be challenging but very rewarding for the users as they won’t have to switch away from their primary system.

I am currently working on a set of microservices. The initial design made sense.

A small microservice stood outside a monolith. It was triggered by webhooks from a 3rd party service. It had a service inside the monolith that provided it the required data.

This provided isolation so that the third party had no direct communication route with the main application.

A few months later and more features have been added. We now use a form inside the monolith to capture the data equivalent that the third party sent.

Moving the data around to perform the calculations in the small microservice is becoming increadingly complex. Once we capture all the data with the new form the small microservice now has no reason to exist.

This is a concept that is hard to track if you have a large amount of microservices. How do you know that a given microservice still needs to exist? This is frequently not documented well.

This concept repeats at other levels. Why does a given function exist? Without continous pruning you end up maintaining dead code.

The only answer is for each service to have a sponsor to keep it alive. If the sponsor is removed find another or remove the item.

This is a thought experiment about how the digital ID card would work in real world situations.

How do you avoid someone impersonating you? Identity theft is fairly common. This would require a system for flagging an account as having suspicious activity.

This is where it gets complicated: how do you authenticate the flagging of suspicious activity? There would be oppertunities for denial of service by bad actors.

It took me 3 months of chasing to get a failed fraud attempt off of my credit score. How would this work for a digital id?

How do you prevent “fake” id apps showing up? Are all requests going back to a central system – good luck keeping that up, secure and accessible from everywhere!

I am not sure that the details have been thought through (even if Mr Blair (Senior) has been pushing this for 20 years.

I am reading Advanced Functional Programming with Elixir.

The book shows how to use various techniques from Haskell in Elixir. Rather than build the library as you go the author has released the full version as https://hex.pm/packages/funx

Eq – Chapter 2 This is about Identity

Ord – Chapter 3 This is about sorting

Monoid – Chapter 4 This is about collecting things

Predicates – Chapter 5 – This is about boolean logic

It seems that all of the AI tools seem to have embraced clickops as a configuration method.

Clickops is the practice of using manual entry and configuration of a system. This contrasts with the infrastructure as code approach which is now the main case for traditional development.

Clickops makes having distinct production and staging environments difficult to maintain. How can you know what you had in place at a given time and restore it.

This is especially difficult for secret management. You eventually have to manually configure all the required secrets. This will prevent automatic expiry and renewal (or risk outages).

It might be worth trying to build desired state configuration tools with whatever APIs you do have.

Recently I have been working on a new service. It has a single rest endpoint that triggers some work.

The tests for this use Hammox to mock out the external services that it talks to. This ensures that the tests always match the declared typespecs for the services.

This week I added ex_coveralls to the project to see how complete the test coverage was. It revealled several parts of the code that had missing tests. Adding those revealled a few small bugs (one of which had been found in production while I was writing the tests!).

The test coverage was approaching 100% in the main processing area. Combined with a suite of tests that produced all of the possible output states I was confident about performing some refactoring.

The refactors were to replace some tuples with structs and them to remove some pointless wrapping error tuples. Coveralls helped to reveal dead code paths that once removed achieved the 100% coverage.

The tests also helped when working out what could be changed. By adding a log statement at a key points I could see all possible inputs to a function. This helped in finding what to change when adapting.

Now I have simpler code that with the tests I have confidence in.

This is a great example of using a python library within Elixir:

https://github.com/nelsonmestevao/pdf_extractor/blob/main/lib/pdf_extractor/pdf_plumber.ex

This means we can easily turn a python cli application into a managed Elixir webservice.
If we don’t have a native Elixir version we are free to use a Python one.
Keep these in small services as it may not play well with a supervision tree!

Elixir finds lots of ways to work well with other languages!

I recently wrote a useful application that does not have its own UI. Your users probably already have enough primary applications to work with so adding another will make their life more complex.

Instead the application writes the data it finds into the application they are already using. I have used this technique now with Contentful, Teams, and now Zendesk.

The trick is to combine webhooks from the application with some form of comment API to write what you need where the users are already working.

In this case I notice that a ticket has been submitted with some metadata and a well known pdf structure. The app reads the pdf, parses it and checks the details against what we have stored on our systems. It also writes both back to the ticket with a list of differences. This means that the user does not need to open the attachments (or our system) to start work.

This is not a language flamewar.

Each language has its focus area.

Rust wins on CPU bound operations while Elixir wins on IO bound ones.

I have been working on a small utility service in Elixir. It accepts a webhook to start. This spawns a process to do the work. The main work is a series of steps unified by a with block. The beauty is that it is all in small isolated files that you can see what it is doing. Having a repl in iex allows you to quickly experiment with services to see what is not working.

I am trying to read an equivalent Rust project to see how one of the system callx is used. Rust is high ceremony so there are many distinct declarations. The habbit of embedding the tests in the same files as the code makes searching for details far more painful than it needs to be.

Recently I reread Competitive engineering.

One of the interesting points that I remember this time is the figure given for unfound errors after a review. It argues that any review will only find 1/3 of the errors present.

This makes the case for many small reviews!