AI-assisted development needs automated tests

Small but profound statement. You hear a lot of variety in feedback on how well LLMs work in contributing to code. What we don't hear enough about are what the conditions of that code are. Do they have good tests? Is it a spaghetti mess? In the early days of LLM's doing code I shared with many friends that coding is in some ways a much easier task than writing generic text. Why? Well in a way you are playing a game with the compiler or interpreter. There is another piece of software that will tell you if you are right (compiles) or wrong (wont compile). So the LLM can play that game at machine-speed to get to something that compiles. However, every developer will tell you that is interesting but means nothing. Just because it compiles doesn't mean it can do the thing you want. What does that? Automated tests and focusing on test-first development. It follows from the first example that tests would then give the LLM another "boss battle" to work through to get to a valid outcome.

All of this is to simply say that as Agentic coding becomes more common we will not just change a lot of tooling for how code is created but we will also likely change the process by which we come to understand what it is that we are building. In fact, if we don't assess that process as a whole we are almost certainly getting a suboptimal solution.

PS: Doing better at this would also help all developers. Test first development, similar to pair programming, is generally a known "good thing" but it is hard and many organizations find it slow in the beginning and they bale on it.