AI and large codebases

~ chriseyre2000 ~ Leave a comment

Given that I am working on a relatively large codebase with sigificant size domain model and tests it is surprisingly easy to ask an AI for answers to questions and get useful answers. Sometimes it can multiple passes to get the details right.

For example I was asking about an error message that was found in the logs. My first set of questions where about how to replace this with a useful message for the user. Having done this work the follow up questions really helped.

I asked how is it possible for this error to happen. It found an option that required the user to have the application open in two tabs at once. This allowed me to suggest adding a couple of simple rules to the system to make the error not possible to happen. This was much simpler than the first solution (which was causing problems to be tested). It makes no sense to edit anything other than the latest version and two rules will enforce that.

AI assistance here is helpful, provided that you review the output carefully so it does not attempt to cheat (bypass a failing test rather than fix the cause).

It is also very good at finding and preventing those flaky tests. You know the one that everyone just reruns. If you can stabalise the tests everything gets much faster.

Another benefit has been correcting bottlenecks in CI. By looking at the slowest parts then splitting them up we have dropped our CI build time from 15 mins to 5-7 mins. We uncovered a pointless step that had been costing 5 mins on every build.

Now AI is going to get more expensive over the next few years so take advantage of the current window to simplify the codebase and clean up that tech debt.

Copilot is surprisingly effective on a large codebase with tests

~ chriseyre2000 ~ Leave a comment

I had been skeptical about using AI for development. My first experiences were with the simpler auto-complete options.

These were erratic – it would offer to complete what you were typing but would frequently start using properties and functions that would have been useful, frequently did not exist. The AI had no context and only guessed.

More recently I have been using the more sophisticsted chat option. Here you can give the AI more context about what you want it to do. This seems to work better if you give the AI context and use the plan mode to agree what you are going to get it to help with. Between each of its steps run tests and ensure that you understand all the changes.

This works well when an application has reasonable test coverage. Good practices like TDD really help. Ensure that the test coverage remains high. Refactor things to keep them tidy. The AI will only build the minimal test (one assert). Ensure that you expand the coverage.

AI is great at finding race condition issues or that failure in the middle of a long chain. You may need to interact with alternative solutions if it gets stuck. Treat it as an enthusiastic novice.

Using AI for development: the good and the bad

~ chriseyre2000 ~ Leave a comment

I have finally started to use Copilot for development work.

Its a power tool. You can do more faster with it but need to be able to understand what it produces.

It will hallicinate (in one case provide a randomly generated value rather than correct hash).

It is great at finding and suggesting fixes to race conditions in a large codebase.

Now rather than just rerunning a flaky test I ask copilot to find why it is flaky.

If you can still understand the codebase after a transformation then it can be a positive thing. Just generating code and you not understaning it is a problem.

This is where TDD practices really help. The tests can give the AI context about the problem being solved. They also act as guardrails for changes.

What to do if SCRUM is not working for you

~ chriseyre2000 ~ Leave a comment

Many teams adopted Scrum as part of becoming “agile”.
This will start by bringing in consultants and hiring Scrum masters.

The starting point will look like this:
You have 2 week sprints.
These will include the following meetings:

  • Daily standups (Each workday)
  • Sprint Planning (every 2 weeks
  • Refinement Meetings (every 1 -2 weeks)
  • Retrospective (every 2 weeks)
  • Sprint Demo (every 2 weeks)

Teams attempt to complete every task that they agreed to within the 2 week window.

Does this seem familiar?

Is it working for you?

How many times do you find things roll over to the next sprint?

How many times do you need to change what you are doing within a sprint?
How frequently do you run out of time in the Refinement Meetings?
How much time is tied up in these agile cermonies?

The SCRUM technique came out of the Extreme Programming practices. It order to be able to be taught in 2 days it dropped the technical practices (pair programming, continuous integration, TDD, simple design …)

The basic cycle is sensible. This is the basic Observe, Orient, Decide, Act loop.
It needs to be applied at several levels.

The basic SCRUM settings are the default. To be effective you need to adapt them. You probably have never been told that you can change the process.

If you are having to change what is in the sprint frequently start by reducing the sprint length to 1 week.
Retros will have to happen every other week. The other meetings become shorter as you are now handling a smaller list of things to handle. You will be deploying functional units more frequently and getting better feedback. Disruptive business requests normally be satisfied with “can it wait until next week?” If not then you have a true emergency that you would have to stop for.

Once you have a 1 week Sprint it is not a big step to move to Kanban. It this you focus on the next important thing and work on getting that finished. This means that refinements become even simpler as you have one thing to work on. You can even have a task that is the analysis that will generate the backlog. Kanban is all about eliminating waste. Do what you need to fix the top task then move to the next.

Kanban needs to be performed at a sustainable pace.
There are technical challenges – how do I break everything down into small valuable increments.
The benefit of the fast feedback is useful – you can get to the point of multiple small releases per day.


Mermaid Wardley Map Example Tea Shop

~ chriseyre2000 ~ Leave a comment

I am starting to experiment with the Mermaid Wardley Map

<html>
  <body>
    <pre class="mermaid">
      wardley-beta
      title Tea Shop
      size [1100, 800]
      
      anchor Business [0.95, 0.63]
      anchor Public [0.95, 0.78]
      component Cup of Tea [0.79, 0.61] label [19, -4]
      component Cup [0.73, 0.78]
      component Tea [0.63, 0.81]
      component Hot Water [0.52, 0.80]
      component Water [0.38, 0.82]
      component Kettle [0.43, 0.35] label [-57, 4]
      component Power [0.1, 0.7] label [-27, 20]
      
      Business -> Cup of Tea
      Public -> Cup of Tea
      Cup of Tea -> Cup
      Cup of Tea -> Tea
      Cup of Tea -> Hot Water
      Hot Water -> Water
      Hot Water -> Kettle
      Kettle -> Power
      
      evolve Kettle 0.62
      evolve Power 0.89
      
      note "Standardising power allows Kettles to evolve faster" [0.30, 0.49]
      note "Hot water is obvious and well known" [0.48, 0.80]
      note "A generic note appeared" [0.23, 0.33]
      
     </pre> 
    <script type="module">
      import mermaid from 'https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.esm.min.mjs';
      mermaid.initialize({ startOnLoad: true });
    </script>
  </body>
</html>

Not a bad start.

Minimal Wardley Map in Mermaid js

~ chriseyre2000 ~ Leave a comment

I have been following the wardley map addition to mermaid js

<html>
  <body>
    <pre class="mermaid">
      wardley-beta
      title Software Platform Strategy
      size [1100, 800]
      
      evolution Genesis@0.25 -> Custom@0.5 -> Product@0.75 -> Commodity@1.0
      
      anchor Customer [0.90, 0.95]
    
      component Something [0.5, 0.5]
      Customer -> Something
     </pre> 
    <script type="module">
      import mermaid from 'https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.esm.min.mjs';
      mermaid.initialize({ startOnLoad: true });
    </script>
  </body>
</html>

This is a terrible example of a wardley map, but does give a starting point for exploring.

I now need to wait for the github and notion components to be updated!

Currently github is on 11.13.0

Notion is on 11.10.0

The miro plugin is back on 10.x so getting rather old!

It was the year that they immanentized the Eschaton

~ chriseyre2000 ~ Leave a comment

This is a line from the start of the Illuminatus Trilogy. This was a series of satirical novels written in the 1970s. I read them in the late 80s/early 90s.

The phrase literally means actively bringing about the end of the world.

This is the first time that I have known of a war when key agents on both sides are trying to bring about the end of the world.

Pete Hegseth (Secretary of War) openly pushes end-times prophecy.

Shia theology also has strong end of days themes.

This combo makes the mutually assured destruction thinking from the cold war break down.