Category: elixir

Working with OpenTelemetry in Elixir

~ chriseyre2000 ~ Leave a comment

I have created two github projects to demonstrate using OpenTelemetry (admittedly badly).

The first project https://github.com/chriseyre2000/something_to_measure is a simple GenServer that has a method that can be used to generate OpenTelemetry Spans.

The second project https://github.com/chriseyre2000/span_eater is a simple GenServer that consumes spans on the default port that OpenTelemetry sends them on.

In production you would typically have a sidecar application to capture and rebroadcast the messages.

In OpenTelemetry terms a Span is a time interval during which a given process ran. These can be nested. The result is that an Observability tool could capture the spans and construct a visualisation of what was happening. Spans are more useful than raw log data as it has a controlled meaning which would need to be inferred.

span_eater currently just logs that it has received the message. I am planning to make it more sophisticated, and then build a LiveBook to host it in. Currently it is useful to remove the log messages that otherwise get generated:
`

[info]  client error exporting spans {:failed_connect,
 [{:to_address, {'localhost', 4318}}, {:inet, [:inet], :econnrefused}]}

These messages typically flood the logs of a locally run application that is instrumented to publish OpenTelemetry data

I have just worked out how to decode the Protobuf data sent over the wire. We can now listen to OpenTelemetry messages sent by our local machine.

Thoughts on Configuration and Supervisors

~ chriseyre2000 ~ Leave a comment

Back on a previous project I worked with Heroku.
Heroku has a great setup for applications. You deploy to a git repo and have a parallel set of configuration via a UI/API.
If either changes the application redeploys the application.

On a walk this morning I realised that you could recreate some of this behaviour inside your Elixir (or other BEAM based) application.

Supervision Tree

If you had a supervisor with an all for one restart policy, if the Config Watcher notices a change it can simply terminate itself and restart the Worker service that depends upon the configuration.

The Config worker is both the cache for the data and periodically checks for changes.
This requires you to keep the configuration watcher specific to the service that uses it to reduce the blast radius of changes.

This seems like a different pattern of use to the typical supervisor.

iex trick: how to continue after a typo

~ chriseyre2000 ~ Leave a comment

iex is the interactive elixir tool. It’s a great repl.

Sometimes when editing you make a typo and get stuck in the edit loop.
On a new line you can use:
iex.break

This will cancel the current edit item and allow you to continue.
This means that you don’t have the shell restarting and don’t have to set up all the global state that you have been working on.

Concurrent Data Processing In Elixir (B4)

~ chriseyre2000 ~ Leave a comment

Now that the book has reached the sent to print stage I am again working through it.

Here is the repo containing the samples from the first chapter:

https://github.com/chriseyre2000/sender

The first chapter works through the simplest solution to concurrency: Tasks.
It starts with a simple slow example and step by step adds more sophistication until you get to have a Task.Supervisor watching the code.

This is one of the more elegant examples of demoing the power and simplicity of Elixir tasks that I have seen.

My Elixir Education in a Series of Books

~ chriseyre2000 ~ Leave a comment

I have been collecting a small library of Elixir books.
This is an introduction to the breadth of the topics covered by Elixir.

A lot of these are tutorial style books that need to be worked through to get the benefits.

My first introduction to Elixir was:

This is Seven More Languages In 7 Weeks which covers a range of languages.
This gave a quick overview of a lot of the language.

Next up was the general introduction book:

Introducing Elixir

This is Introducing Elixir (there is a second version) a fairly straight conversion of Introducing Elixir. This is a gentle introduction to the language.

Next was an earlier version of:

Programming Elixir

This is Programming Elixir (I read one of the earlier editions). This is an in-depth exploration of the language.

Next was

Designing for scalability with Erlang/OTP

This is Designing for Scalability with Erlang/OTP. This goes into more depth on the OTP and how to design for scale. It took me several attempts to work through this.

Next was:

Programming Phoenix

This is Programming Phoenix. I have so far made three attempts to work through this one, the first was in the beta, the second when it was finished (I got distracted) and again recently.

This is one that I am still trying to find the time to read:

Craft GraphQL APIs in Elixir with Absinthe

Each time this one makes it to the top of the list I keep finding another book to read ahead of it.

Metaprogramming Elixir

This is Metaprogramming Elixir which gives a deeper understanding of when to use macros (and when not to).

I won a copy of this on a twitter competition:

Phoenix for Rails Developers

This is Phoenix for Rails Developers another more gentle introduction to Phoenix. The contrast with Rails is illustrating, pointing out pain points that Phoenix solves.

The next book is less about the code and more about how to get a project to use Elixir:

Adopting Elixir

This is Adopting Elixir. This covers some case studies of Elixir being used in production environments.

This is another book that explains how to design with Elixir

Functional Web Development, with Elixir, OTP and Phoenix

This is Functional Web Development with Elixir, OTP and Phoenix.

The next one would make a great second book for Elixir:

Designing Elixir Systems with OTP

This is Designing Elixir Systems with OTP. The approach of building Fun Things, with Big, Loud Worker Bees is a great project structuring approach. It explains the layers that should be used to design a great application.

This is another that I have not yet finished reading, but do get a lot out of:

Learn You Some Erlang for Great Good!

This is Learn You Some Erlang for Great Good! It’s a huge book and covers a lot of details about working with Erlang. I have been meaning to create a repo converting the examples in this book into Elixir.

By the same author is:

Property-Based Testing with PropEr, Erlang, and Elixir

This is Property-Based Testing with PropEr, Erlang and Elixir. The book is more biased towards Erlang. However the ideas in it have changed how I unit test things.

Another one of the books that I have not yet finished (I did buy it in beta):

Real-Time Phoenix

This is Real-Time Phoenix. It covers the soft real-time features of Phoenix.

Another one that I recently finished reading:

Genetic Algorithms in Elixir

This is Genetic Algorithms in Elixir. This covers a topic that you would not naturally associate with Elixir. It makes a good case of why Elixir is very good at it (parallel execution can speed these up).

This is another one that I started working through in beta, and have not yet returned to:

Testing Elixir

This is Testing Elixir. It goes into depth about how to get the most out of ExUnit.

This covers one of the tools that is used heavily by Phoenix.

Programming Ecto

This is Programming Ecto. It covers the database interaction code in more detail than the other books. I like that Ecto provides both Migrations and the data access abstractions.

This is my most recent purchase:

Concurrent Data Processing in Elixir

This is Concurrent Data Processing in Elixir. It seems to cover the tools needed for large scale data processing. Not yet started on this one.

The last one in this list is also as yet unread.

Modern CSS with Tailwind

This is Modern CSS with Tailwind. Technically it is not an Elixir book, but does form part of the PETAL stack (Phoenix, Elixir, Tailwind, Alpine, Liveview). I do plan to create an unofficial repo with the examples for this in Phoenix.

Experimenting With Elixir in Docker

~ chriseyre2000 ~ 1 Comment

I am working on my book Elixir Function Guide. Having noticed that there are a lot of mix tasks that I did not know about (and some that my exercism.io students don’t) then I thought that I would add some details at the start of the book.

I started using mix help to list the available functions:

mix                   # Runs the default task (current: "mix run")
mix app.start         # Starts all registered apps
mix app.tree          # Prints the application tree
mix archive           # Lists installed archives
mix archive.build     # Archives this project into a .ez file
mix archive.install   # Installs an archive locally
mix archive.uninstall # Uninstalls archives
mix clean             # Deletes generated application files
mix cmd               # Executes the given command
mix compile           # Compiles source files
mix credo             # Run code analysis (use `--help` for options)
mix credo.gen.check   # Generate a new custom check for Credo
mix credo.gen.config  # Generate a new config for Credo
mix deps              # Lists dependencies and their status
mix deps.clean        # Deletes the given dependencies' files
mix deps.compile      # Compiles dependencies
mix deps.get          # Gets all out of date dependencies
mix deps.tree         # Prints the dependency tree
mix deps.unlock       # Unlocks the given dependencies
mix deps.update       # Updates the given dependencies
mix do                # Executes the tasks separated by comma
mix escript           # Lists installed escripts
mix escript.build     # Builds an escript for the project
mix escript.install   # Installs an escript locally
mix escript.uninstall # Uninstalls escripts
mix format            # Formats the given files/patterns
mix help              # Prints help information for tasks
mix hex               # Prints Hex help information
mix hex.audit         # Shows retired Hex deps for the current project
mix hex.build         # Builds a new package version locally
mix hex.config        # Reads, updates or deletes local Hex config
mix hex.docs          # Fetches or opens documentation of a package
mix hex.info          # Prints Hex information
mix hex.organization  # Manages Hex.pm organizations
mix hex.outdated      # Shows outdated Hex deps for the current project
mix hex.owner         # Manages Hex package ownership
mix hex.publish       # Publishes a new package version
mix hex.repo          # Manages Hex repositories
mix hex.retire        # Retires a package version
mix hex.search        # Searches for package names
mix hex.user          # Manages your Hex user account
mix loadconfig        # Loads and persists the given configuration
mix local             # Lists local tasks
mix local.hex         # Installs Hex locally
mix local.phx         # Updates the Phoenix project generator locally
mix local.public_keys # Manages public keys
mix local.rebar       # Installs Rebar locally
mix new               # Creates a new Elixir project
mix phx.new           # Creates a new Phoenix v1.4.0 application
mix phx.new.ecto      # Creates a new Ecto project within an umbrella project
mix phx.new.web       # Creates a new Phoenix web project within an umbrella project
mix profile.cprof     # Profiles the given file or expression with cprof
mix profile.eprof     # Profiles the given file or expression with eprof
mix profile.fprof     # Profiles the given file or expression with fprof
mix release           # Assembles a self-contained release
mix release.init      # Generates sample files for releases
mix run               # Starts and runs the current application
mix test              # Runs a project's tests
mix xref              # Prints cross reference information
iex -S mix            # Starts IEx and runs the default task

It was only then that I realised that I have a number of mix extensions installed.

I need to start with what comes out of the box.
At this point I have three choices:

  • Uninstall the extensions (these are usefull .)
  • Manually edit out the extensions (error prone)
  • Use docker

The docker option is appealing as it will allow me to test using different versions of Elixir.

Here is the simple option that allows the use of docker:

docker run -it elixir:1.10 /bin/bash

This downloads the image and runs up the container. Note that this starts with bash, rather than the default iex.

From this mix help gives the more useful list:

mix                   # Runs the default task (current: "mix run")
mix app.start         # Starts all registered apps
mix app.tree          # Prints the application tree
mix archive           # Lists installed archives
mix archive.build     # Archives this project into a .ez file
mix archive.install   # Installs an archive locally
mix archive.uninstall # Uninstalls archives
mix clean             # Deletes generated application files
mix cmd               # Executes the given command
mix compile           # Compiles source files
mix deps              # Lists dependencies and their status
mix deps.clean        # Deletes the given dependencies' files
mix deps.compile      # Compiles dependencies
mix deps.get          # Gets all out of date dependencies
mix deps.tree         # Prints the dependency tree
mix deps.unlock       # Unlocks the given dependencies
mix deps.update       # Updates the given dependencies
mix do                # Executes the tasks separated by comma
mix escript           # Lists installed escripts
mix escript.build     # Builds an escript for the project
mix escript.install   # Installs an escript locally
mix escript.uninstall # Uninstalls escripts
mix format            # Formats the given files/patterns
mix help              # Prints help information for tasks
mix loadconfig        # Loads and persists the given configuration
mix local             # Lists local tasks
mix local.hex         # Installs Hex locally
mix local.public_keys # Manages public keys
mix local.rebar       # Installs Rebar locally
mix new               # Creates a new Elixir project
mix profile.cprof     # Profiles the given file or expression with cprof
mix profile.eprof     # Profiles the given file or expression with eprof
mix profile.fprof     # Profiles the given file or expression with fprof
mix release           # Assembles a self-contained release
mix release.init      # Generates sample files for releases
mix run               # Starts and runs the current application
mix test              # Runs a project's tests
mix xref              # Prints cross reference information
iex -S mix            # Starts IEx and runs the default task

Funny Characters in Elixir: Sigils

~ chriseyre2000 ~ Leave a comment

There are a number of special character sequences in Elixir, for example the pipeline operator |> but now I want to discuss the ~something options.

These are sigils. The magic is simple if you call ~x(“something”) then the compiler will translate this into a call to sigil_x

~r/foo/i becomes sigil_r(<<"foo">>, 'i')

See https://elixir-lang.org/getting-started/sigils.html for more details.

This does mean that you can now write your own sigils.

Here are the built in sigils. If there is an uppercase/lowercase pair they work the same except the lowercase one escapes the content.

sigil_C/2          sigil_c/2 returns a charlist

sigil_D/2          Creates date types

sigil_N/2         Creates native datetime

sigil_R/2          sigil_r/2        Regular expressions  

sigil_S/2          sigil_s/2          Creates strings from literals

sigil_T/2          Creates time types

sigil_U/2          Creates a UTC Datetime

sigil_W/2          sigil_w/2          Creates a list of words by splitting on whitespace

         

                  

Announcing About

~ chriseyre2000 ~ Leave a comment

Last weekend I wrote and released the Elixir package called “about”.

This adds About topics to the iex environment.

It is installed in the usual way:

Add {:about, "~> 0.0.1"} to your deps list in mix.exs

Run mix deps.get

Run iex -S mix

You can now use help topics like:

iex(1)> h About.pipeline

I am looking for suggestions/contributions to the project.

It’s hosted on https://github.com/chriseyre2000/about and available on hex.pm and hexdocs.