Event Schema Evolution in CQRS Event Sourcing

This is a collection of thoughts on this big topic. If you are working on an event sourced system it will need to cope with changes to the structure of events over time.

Here is some of the literature that I have found on this topic:

• https://www.sciencedirect.com/science/article/pii/S0164121221000674
• https://www.researchgate.net/publication/315637858_The_dark_side_of_event_sourcing_Managing_data_conversion
• https://resources.sei.cmu.edu/asset_files/Presentation/2017_017_001_497589.pdf
  
  There are other possible sources of inspiration:
• Avro
• Protobuf

Avro and Protobuf are wire protocol schemas.

For avro, here I am quoting from: https://docs.oracle.com/cd/E26161_02/html/GettingStartedGuide/schemaevolution.html

These are the modifications you can safely perform to your schema without any concerns:

• A field with a default value is added.
• A field that was previously defined with a default value is removed.
• A field’s doc attribute is changed, added or removed.
• A field’s order attribute is changed, added or removed.
• A field’s default value is added, or changed.
• Field or type aliases are added, or removed.
• A non-union type may be changed to a union that contains only the original type, or vice-versa.

Beyond these kind of changes, there are unsafe changes that you can do which will either cause the schema to be rejected when you attempt to add it to the store, or which can be performed so long as you are careful about how you go about upgrading clients which use the schema. These type of issues are identified when you try to modify (evolve) schema that is currently enabled in the store. See Changing Schema for details.

Avro tends to be used with centralised registries. This is both a benefit for consistency and.a constraint for rapid change.

Protobuf is slightly more flexible. Given that the underlying storage is determined by ids it is the practice to change an id when the meaning of a field changes. It is entirely possible to have two protobuf definition files that are compatible yet don’t share any field names. This is great for translations across bounded contexts. Protobuf permits a slight read/write disparity. Just because a field is required on the write side it may be optional or omitted on the read side.