The project that I am working on finally got the requirements that allowed us to pick a database. Over the last few years I have been using MongoDb as the lazy choice for storage. The project makes more sense to use a relational database so I have returned to Postgres.
This has encouraged me to reread this book. I am planning on adding some details in these notes to add modern practices to the book.
To start with the book predates containers so all of the chapters require you to install the database on your machine. I am not encouraging you to use a database in a container for production. It does however allow you to test drive database code.
Over the last decade all of the big databases I have used have been cloud hosted. For a JVM based application we used hsqldb as an embedded replacement for the database in tests.
The book also fails to mention database migrations. This is a technique that allows you to version control the schema of a database.
Coming back to Postgres after years of Mongo makes me realize that you need to work a little harder to get data to and from the store. On the other hand SQL is far easier to make aggregate queries with compared to Postgres. It is easier to enforce constraints in SQL. Types, unique indexes, non-null fields and foreign keys all help.
The book also only works at the database level. I want to add examples of how to connect to the database.
Recently I have been working with Terraform to stand up the infrastructure for the project that I am working on. The project involves a database plus a number of lambda functions. We have a CD build pipeline and use Terraform for the infrastructure, run from cdflow.
Terraform is great for defining the infrastructure. The problem comes when you rename something (module names in particular). Terraform will attempt to tear down the service and recreate it. Sometimes AWS services are eventually consistent. This can mean that a deleted resource hangs around for a while after being deleted. A rename of a module will delete and recreate the item, which will frequently fail on the first pass.
You also need to be very careful that you only build to a given environment from a single branch of the build pipeline. Not doing so allows databases to be torn down. I have seen an incident where a developer comments out infra that is not needed for the current build, only for that change to delete the production database and all the backups. There are things that can be done to prevent this (marking backups as requiring an extra switch to be removed). These changes then leave Terraform unable to completely clean up.
I am not arguing against configuration as code, merely noting that you will get a lot of failed builds. Some of these are resolved by rerunning the job/pipeline. Others require the resource to be manually deleted.
Terraform operates at a level of abstraction that can both help and hinder. Some of the abstractions are a little weak (these will improve in time) especially when defining IAM permissions as you end up with inline strings that explicitly define the policy.
I have been collecting a small library of Elixir books. This is an introduction to the breadth of the topics covered by Elixir.
A lot of these are tutorial style books that need to be worked through to get the benefits.
My first introduction to Elixir was:
This is Seven More Languages In 7 Weeks which covers a range of languages. This gave a quick overview of a lot of the language.
Next up was the general introduction book:
Introducing Elixir
This is Introducing Elixir (there is a second version) a fairly straight conversion of Introducing Elixir. This is a gentle introduction to the language.
Next was an earlier version of:
Programming Elixir
This is Programming Elixir (I read one of the earlier editions). This is an in-depth exploration of the language.
Next was
Designing for scalability with Erlang/OTP
This is Designing for Scalability with Erlang/OTP. This goes into more depth on the OTP and how to design for scale. It took me several attempts to work through this.
Next was:
Programming Phoenix
This is Programming Phoenix. I have so far made three attempts to work through this one, the first was in the beta, the second when it was finished (I got distracted) and again recently.
This is one that I am still trying to find the time to read:
Craft GraphQL APIs in Elixir with Absinthe
Each time this one makes it to the top of the list I keep finding another book to read ahead of it.
Metaprogramming Elixir
This is Metaprogramming Elixir which gives a deeper understanding of when to use macros (and when not to).
I won a copy of this on a twitter competition:
Phoenix for Rails Developers
This is Phoenix for Rails Developers another more gentle introduction to Phoenix. The contrast with Rails is illustrating, pointing out pain points that Phoenix solves.
The next book is less about the code and more about how to get a project to use Elixir:
Adopting Elixir
This is Adopting Elixir. This covers some case studies of Elixir being used in production environments.
This is another book that explains how to design with Elixir
Functional Web Development, with Elixir, OTP and Phoenix
This is Functional Web Development with Elixir, OTP and Phoenix.
The next one would make a great second book for Elixir:
Designing Elixir Systems with OTP
This is Designing Elixir Systems with OTP. The approach of building Fun Things, with Big, Loud Worker Bees is a great project structuring approach. It explains the layers that should be used to design a great application.
This is another that I have not yet finished reading, but do get a lot out of:
Learn You Some Erlang for Great Good!
This is Learn You Some Erlang for Great Good! It’s a huge book and covers a lot of details about working with Erlang. I have been meaning to create a repo converting the examples in this book into Elixir.
By the same author is:
Property-Based Testing with PropEr, Erlang, and Elixir
Another one of the books that I have not yet finished (I did buy it in beta):
Real-Time Phoenix
This is Real-Time Phoenix. It covers the soft real-time features of Phoenix.
Another one that I recently finished reading:
Genetic Algorithms in Elixir
This is Genetic Algorithms in Elixir. This covers a topic that you would not naturally associate with Elixir. It makes a good case of why Elixir is very good at it (parallel execution can speed these up).
This is another one that I started working through in beta, and have not yet returned to:
Testing Elixir
This is Testing Elixir. It goes into depth about how to get the most out of ExUnit.
This covers one of the tools that is used heavily by Phoenix.
Programming Ecto
This is Programming Ecto. It covers the database interaction code in more detail than the other books. I like that Ecto provides both Migrations and the data access abstractions.
This is Modern CSS with Tailwind. Technically it is not an Elixir book, but does form part of the PETAL stack (Phoenix, Elixir, Tailwind, Alpine, Liveview). I do plan to create an unofficial repo with the examples for this in Phoenix.
"each change is suspect when there’s a problem. 1 change, 1 problem? 1 suspect. 2 changes, 1 problems? 2 suspects. 17 changes, 3 problems? 51 suspects — yikes!" this may be the only CI/CD post featuring a hungry sock, bc @jessitron is a fucking genius.https://t.co/etjgdtMjnn
This weekend I have been working through the published version of this book.
The ideas are great, but there are too many errors in the examples to make it easy to work through.
The book builds up a framework for using Genetic Algorithms, including some sophisticated logging, visualisation and performance tuning tricks. Some of the example code used clearly belongs with earlier drafts. I am sure if you download the sample code you could fix the mistakes. For example some of the logic in the tiger example is wrong in the first instance, but correct when used later on. Also I don’t think that you can pass an anonymous function to apply.
Recently I have been the victim of Identity Theft.
In October and November last year there were a total of 57 attempt to take out financial services in my name. So far two of these have succeeded in obtaining money.
Both of them were payday loan companies. I have reported the identity theft to Action Fraud. Both companies would openly talk to me about the loans and have recorded them as fraudulent.
I have signed up to a credit file checking service to allow me to see what the current state is. I will need to keep this up for a couple of months.
However neither of them have made any attempt to check that I am who I say I am. I have been sent the pdf bank statement used as part of the identification by one of the companies. Other than giving them publicly available information about myself (name, date of birth and address) no attempt has been made to validate who I am.
It can be hard to get a company to talk to you about fraud. Firms should really have an email address that can be used to report this. For example one of the finance checks went to Sky Mobile. I cannot find a means of talking to them without becoming a customer.
GDPR requests seem to be the only way to get a company to talk to them.
This is the first chapter where I have found some typos info_sys/application.ex needs the following to work:
alias InfoSys.Counter
Without that the app won’t start.
You also need to be careful with the examples. The name of the files are sometimes incorrect (although the listed path is right). Sometimes it asks you to edit a file in Rumbl when it actually means InfoSys.
There are also a lot of mistakes in the supervisor demos. Sometimes aliases are missed (as above) and sometimes it refers to the wrong project.
Currently I am at the end of Chapter 10. This has introduced using websockets in Phoenix to interact with the page. The application allows annotations to be made to a video that playback in real time.
I am having some trouble with the actual Youtube API since it now complains about cookies not being on the main site. I intend to finish working through the book and return to fix this issue.
Chapter 11 is about refactoring the application to use an Umbrella Application. This will split the Rumbl and RumblWeb applications apart. Having a real demo of refactoring is useful.
