After my last post I have found that Amazon (AWS), Microsoft (Azure) and Google (Google Cloud) are all in the process of constructing UK based regions. However they don’t seem to have gone for the Northern Powerhouse option…

Opsgenie seems to dominate my life for a week every other month.

OK, it’s a support rota.

However it’s a great way of getting teams to do things. Raise it as an alert and it can’t be ignored. It’s a simple http post to raise or close so automation is trivial.

Intgration with statuspage.io as a webhook very is powerful.

This is an unusual kind of a database.

To start with it’s very cheap to use – you generally only pay for the columns of data read. You can get a lot of data processing for a few pence.

It does have some interesting other characteristics:

You can only read and write data, you can’t update it or delete specific rows.

It’s not always very fast (it’s for analytics not transactional  data) and does have periodic outages.

It can contain nested repeated rows.

Tables are meant to be extended – you can add an optional  or repeated column but can’t change the type of an existing one.

Copying tables is effectively free.

You can replace the entire contents of a table with a select result (or create a new table from a select result).

This means that you need a different strategy for loading data into these tables. The trick is to find a means of making inserts idempotent without reading too much data.

This could involve writing to a staging table and using a copy insert to move distinct data back into the master.

You don’t have to use the rest of the google infrastructure to load into BQ – we use heroku scheduler tasks. This means we pay pennies to load data into a storage system that costs pennies to run. This can completely change the economics of software development. The most significant cost is now developer and management time – it can now be cheaper to delegate authority for data creation to the developer (only ask if it will cost more than $100 per month) than to hold a meeting to request approval.

This cloud economics also gets fun when deciding when to optimise a slow process. If you could dial up the size of the box that you use to add 7 pennies to the months bill then how many months would that process need to run in order for it to be worth spending 2 hours of developer time to save that cost?

• A pricing model that scales with our success and allows for spikey traffic.
• A stable interface. If you want to change something make it backwards compatable or give us notice of change.
• Have a status page similar to status.io that is accurately updated.
• Respond to support requests in hours not days.
• Use webhooks rather than emails to announce outages, maintenance windows and upgrades.
• We need to use an api to deploy or configure your service.
• We need an api to reconcile any billing transactions with payments.
• We will have multiple environments (qa, perf, staging, live) and only expect full price and support on live and perf.

Recently I have been working on a data warehouse project that is trying to extract billing and payment information from a range of cloud services. In all cases it seems that the data needed by the accountants has not been given the care that it needs.

The industry practice seems to be log into a portal and download a spreadsheet of data that may be accurate or timely. Any api’s exposing this data seem to only work on a per order basis – which means that an attempt to keep a database upto date could require geometrically increasing processing times (you need to check each order for changes to a transaction).  These api’s tend to be heavily restricted on the volume of data per call (hundreds of transactions per call) combine this with a frequently undocumented rate limit make catching up historical data difficult at best.

If you are providing services that accountants need think about providing an api that makes reconciliation easy. Reconciliation is important especially if you want to get your tax calculations right.

I am getting increasingly frustrated with the quality of the packages in NPM.

Is seems that it’s ok to randomly change code and leave things in a broken state (or at least have a stack of errors for simple usage.

The team that I am working on has been described by our outgoing contractors as a DevOps team. We develop, deploy and support the site. We add monitoring to everything we can (sometimes after it goes wrong once). We are an agile team loosly working in a scrum setup. The trick we have is to always dedicate one pair (or half a pair if the team is odd numbered that week) to the operations channel. The team member that is on the support rota gets first option of working on the operations tickets this week.

Dev-Ops is normally about having a dedicated team that perform automation and monitoring of systems. We do have that happening but from within the team. There is no passing it to an “other” team. In that sense we have achieved No-Ops – there literally is no ops team that we can pass off problems too (OK we have various vendors to manage but that is a different matter).