# AI-assisted development needs automated tests - Host: [simonwillison.net](https://links.thingelstad.xyz/hosts/simonwillison.net/) - Saved: May 29, 2025 (2025-05-30T02:45:04Z) - Record: https://links.thingelstad.xyz/links/93b2d71dfa81/ - JSON: https://links.thingelstad.xyz/links/93b2d71dfa81.json ## What Jamie wrote Small but profound statement. You hear a lot of variety in feedback on how well LLMs work in contributing to code. What we don't hear enough about are what the conditions of that code are. Do they have good tests? Is it a spaghetti mess? In the early days of LLM's doing code I shared with many friends that coding is in some ways a much easier task than writing generic text. Why? Well in a way you are playing a game with the compiler or interpreter. There is another piece of software that will tell you if you are right (compiles) or wrong (wont compile). So the LLM can play that game at machine-speed to get to something that compiles. However, every developer will tell you that is interesting but means nothing. Just because it compiles doesn't mean it can do the thing you want. What does that? Automated tests and focusing on test-first development. It follows from the first example that tests would then give the LLM another "boss battle" to work through to get to a valid outcome. All of this is to simply say that as Agentic coding becomes more common we will not just change a lot of tooling for how code is created but we will also likely change the process by which we come to understand what it is that we are building. In fact, if we don't assess that process as a whole we are almost certainly getting a suboptimal solution. PS: Doing better at this would also help all developers. Test first development, similar to pair programming, is generally a known "good thing" but it is hard and many organizations find it slow in the beginning and they bale on it. ## Facts - Kind: [Page](https://links.thingelstad.xyz/kinds/page/) ## Appears in - [WT321: Weekly Thing 321 / Saluting, Tetris, Sky](https://weekly.thingelstad.com/archive/321/), Notable ## Same host - [llm-prices.com](https://links.thingelstad.xyz/links/6f77e199569f/) (simonwillison.net, May 8, 2025) - [Highlights from the Claude 4 system prompt](https://links.thingelstad.xyz/links/8ff0eb636351/) (simonwillison.net, May 29, 2025) - [Large Language Models can run tools in your terminal with LLM 0.26](https://links.thingelstad.xyz/links/17c8a3bb1985/) (simonwillison.net, May 29, 2025) - [Design Patterns for Securing LLM Agents against Prompt Injections](https://links.thingelstad.xyz/links/161cfcb03e8b/) (simonwillison.net, June 16, 2025) - [Coding agents require skilled operators](https://links.thingelstad.xyz/links/8b405227b581/) (simonwillison.net, June 19, 2025) ## Saved the same week - [prompt-eng-interactive-tutorial: Anthropic's Interactive Prompt Engineering Tutorial](https://links.thingelstad.xyz/links/a9e496dc4731/) (github.com, May 29, 2025) - [Sky Extends AI Integration and Automation to Your Entire Mac - MacStories](https://links.thingelstad.xyz/links/cbb115d94ff5/) (macstories.net, May 29, 2025) - [Say Your Writing](https://links.thingelstad.xyz/links/789fa17abf00/) (martinfowler.com, May 29, 2025) - [Daring Fireball: Gurman: Apple Is Going to Re-Version OSes by Year, Starting With iOS 26, MacOS 26, tvOS 26, Etc.](https://links.thingelstad.xyz/links/1aafad425e12/) (daringfireball.net, May 29, 2025) - [The Five Layers of Sharing Thoughts and Ideas | Matt Mullenweg](https://links.thingelstad.xyz/links/d1055e4a00dd/) (ma.tt, May 29, 2025) ## Elsewhere - Wayback Machine: https://web.archive.org/web/*/https://simonwillison.net/2025/May/28/automated-tests/