Random outpourings of a software developer
I have recently noticed an odd pattern in code that I write.
First I add a helper method to a unit test to speed up setup.
This then become so useful to manual testing that it gets added to the test tools in the UI.
Subsequently the customer service team start to ask about this feature.
The general rule is if it is going to speed up development consider adding the test tool to the customer service ui. Edge cases in code will become customer service wrinkles.
This is a great way of seeing what is going on in Erlang/Elixir.
Fire up iex:
Type the following
:erlang.registered()
This lists the registered global names.
:observer.start
This launches the process observer which allows you to browse the current state of the system (can be attached to any erlang system that you can access).
Given that a little over two years ago I was a C# developer I have had to learn the JVM way of doing things fairly rapidly. Java does has many ways of solving things but these are those that I am familiar with:
Here is a map between the Java world and the Elixir world
JVM => BEAM
Gradle => Mix
https://search.maven.org/ => https://hex.pm
JUnit => ExUnit
JavaDoc => ExDoc
Dropwizard (or web framework of choice) => Phoenix
Jooq => Ecto
This is simple for plain projects:
./gradlew deps
Here is a useful series of articles on getting started integrating Elm into a Phoenix application:
http://codeloveandboards.com/blog/2017/02/02/phoenix-and-elm-a-real-use-case-pt-1/
The article uses Elm, integrates using Brunch and is deployed as part of the Phoenix application.
I am now working through the Elm samples and converting them into a elm-reactor application.
The source is here on github:
https://github.com/chriseyre2000/elm-samples.git
Reactor looks really useful. Elm applications hold their state in it’s model.
The default reactor app allows you to preview the commands sent to the model. This will make creating test cases really easy. This is the big advantage of functional programming – there are no side effects.
The type specifications reminds me of Delphi’s interface/implementation split or of C header files.
Interesting note: elm-reactor is case sensitive:
Ask for filename.elm and it compiles and runs the page.
Ask for filename.Elm and it shows the source.
So far I have needed to add:
elm-package install elm-lang/http
elm-package install elm-lang/svg
elm-package install evancz/elm-markdown
elm-package install elm-lang/html
elm-package install elm-lang/websocket
elm-package intstall elm-lang/mouse
I have to admit to being a bit of a language geek.
A few years ago I started working my way through Seven Languages in Seven Weeks (Ruby [x] , Io [x], Prolog [x], Scala [x], Erlang [x], Clojure [], Haskell []).
More recently I started working my way through Seven More Languages In Seven Weeks (Lua [], Factor [], Elm [], Elixir [x], Julia [x], miniKanren [], Idris [])
I have been posting various things about Elixir which seems to be a very promising language.
The Elm chapter of that book was based upon 0.13 but that language is now at 0.18 which has changed significantly.
I have now found a pluralsight course (https://www.pluralsight.com/courses/elm-getting-started) that gives the latest view on 0.18
Key points
Install the language tools with:
npm install -g elm
Add the cli completion with
npm install elm-oracle -g
Here is the language documentation:
Here are the useful examples:
Here is the package search engine:
There are a few basic tools supplied with the elm environment:
elm-make Compile an Elm file or project into JS or HTML
elm-package Manage packages from <http://package.elm-lang.org>
elm-reactor Develop with compile-on-refresh and time-travel debugging
elm-repl A REPL for running individual expressions
So far I have been experimenting with moving the samples from the above examples into a simple reactor app. The major problem that I have found so far is determining the required packages. The online tool clearly can find them yet it’s hard to find the right version locally.
For example I was trying to get the Markdown sample (http://elm-lang.org/examples/markdown) into a simple reactor project (copy code into a file called Mardown.elm). I was provided with the useful error message that it could not find the Markdown module. Searching on http://package.elm-lang.org/packages revealed that the best package was evancz/elm-markdown/3.0.2. Adding “evancz/elm-markdown”: “3.0.2 <= v < 4.0.0” to the elm-package.json seemed to work. I did need to delete the elm-stuff directory to get reactor to update.
Equifax managed to get itself into the headlines for all the wrong reasons by leaving known critical vulnerabilities unpatched.
It’s now one more step you need to add to the build pipeline:
Build, Test, Check for vulnerabilities, Deploy, smoke Test
Gradle managed to add a simple task to do this: https://github.com/jeremylong/DependencyCheck
This is very well designed. It creates a detailed report on your dependencies (including transitive ones) and matches these to listed CVE reports. The report can be used to generate the ignore statements needed to mark false positives (typically if you have a private library thats name is close to a know CVE entry).
Here is a useful site for the basics of Elixir:
In particular at the moment I am interested in Plugs:
https://elixirschool.com/en/lessons/specifics/plug/
There is a gotcha that caught be out: you need to visit http://127.0.0.1:8080 rather than http://localhost:8080
The next trick will be to deploy a trivial web application to heroku.
More on plugs: https://github.com/chriseyre2000/plug_demo
Once I am happy with Cowboy then I can move on to Phoenix.
Here is the plug sample: https://github.com/chriseyre2000/plug_demo
This is a trivial website application. It allows an http interface to be put onto an Elixir/OTP application. That is where things get powerful.
The development and deployment of server software has changed radically over the last few years.
It used to be the case that you had to buy servers, find space for the server, capacity plan the numbers. You added DR backups and test systems (always smaller than production). It would take weeks for kit to arrive.
Now given services like Heroku and Azure it is possible to set up an entire setup in under an hour billed monthly to your credit card. A test server can be the same size as production when needed then switched off when no longer required. The avoids the old queues to access test systems.
You do need to be careful that you have switched off what you no longer need or can face a large unexpected bill.
Given that in Heroku you can have an application server running for a month from $7, workers can be scheduled to run for seconds each month (batch processing using the Heroku Scheduler) costing $0.12. Now the costs of discussing options can exceed the annual costs of deployment.
For example we found one of our batch reporting tools running out of memory. We could have spent 2 days optimizing it or just dial it up to the next tier up at a cost of an extra $0.04 per month.
My current employer does not even need the developers to ask for any new services less than $100 per month for experiments. The meetings to discuss this would cost more than the work.
With careful use of a CDN we can host a very high volume site using minimal number of servers at a reasonable cost.
Heroku makes experiments very easy – push code to a git repo with a simple Procfile and you can have a server up and running in seconds. You can add a host of backend services at a reasonable rate – especially as it is billed per second that is enabled. When not needed just switch off.
I recently attended Jeffconf in London. The name came when someone pointed out that “Serverless development” actually required servers so the name was arbitrary hence Jeff.
I can see how serverless frameworks could be used to handle spiky loads but you normally will require some form of storage or batch processing which will exceed the serverless costs.
Serverless refers to services such as Amazon Lambda or IBM Open Whisk. You deploy a minimal script that is instantiated (possibly) per call and you are only charged for usage.
This looks like a useful component in a hybrid system. Use Serverless to store to a Google bucket with a polling service that copies this into BigQuery.
The development process is now a lot more flexible!
Developer Rants 