# Running Python micro-benchmarks using the ChatGPT Code Interpreter alpha

<https://simonwillison.net/2023/Apr/12/code-interpreter/>

- Host: [simonwillison.net](https://links.thingelstad.xyz/hosts/simonwillison.net/)
- Saved: April 13, 2023 (2023-04-14T02:00:32Z)
- Record: https://links.thingelstad.xyz/links/0ef43037de4f/
- JSON: https://links.thingelstad.xyz/links/0ef43037de4f.json

## What Jamie wrote

This is an incredible way to use ChatGPT. It requires some alpha plug-in access most people don't have.

> Here's the initial prompt I sent it:
> 
> > In Python create an in-memory SQLite database with 100 tables each with 10 columns.
> > 
> > Time how long it takes to execute `PRAGMA schema_version` against that database 100 times.
> > 
> > Then add another 100 tables and time PRAGMA schema_version 100 times again.
> > 
> > Now run the whole benchmark a second time, but instead of `PRAGMA schema_version` time how long it takes to run hashlib.md5(db.execute("select group_concat(sql) from sqlite_master").fetchall()[0]).hexdigest() instead

Okay, that is a wild set of requirements that would take a significant amount of time to write a testing harness for.

> From start to finish, this entire benchmarking exercise took me less than five minutes—because ChatGPT did almost all of the work for me.

Willison goes on to tweak and modify this benchmark to make it more useful.

This whole process is remarkable. 🤯

## Facts

- Kind: [Page](https://links.thingelstad.xyz/kinds/page/)

## Appears in

- [WT250: Weekly Thing #250 / LLM, Shanghai, Wing](https://weekly.thingelstad.com/archive/250/), Notable

## Same host

- [Bing: “I will not harm you unless you harm me first”](https://links.thingelstad.xyz/links/fa1b1efc4200/) (simonwillison.net, February 16, 2023)
- [Think of language models like ChatGPT as a “calculator for words”](https://links.thingelstad.xyz/links/fd6fb5be4b85/) (simonwillison.net, April 5, 2023)
- [Prompt injection: What’s the worst that can happen?](https://links.thingelstad.xyz/links/8144e594e273/) (simonwillison.net, April 17, 2023)
- [Leaked Google document: “We Have No Moat, And Neither Does OpenAI”](https://links.thingelstad.xyz/links/228d2b2da890/) (simonwillison.net, May 9, 2023)
- [Mojo may be the biggest programming advance in decades](https://links.thingelstad.xyz/links/4c7d45a62ac2/) (simonwillison.net, May 9, 2023)

## Saved the same week

- [How to plan as an engineering executive. | Irrational Exuberance](https://links.thingelstad.xyz/links/e74ce0f4fc52/) (lethain.com, April 13, 2023)
- [The AI singularity is here | InfoWorld](https://links.thingelstad.xyz/links/8e92096170e2/) (infoworld.com, April 13, 2023)
- [Importing tweets to Micro.blog - Manton Reece](https://links.thingelstad.xyz/links/3850b41f9d96/) (manton.org, April 13, 2023)
- [The one good HomeKit-capable wall switch you should get – A Whole Lotta Nothing](https://links.thingelstad.xyz/links/3ec9ab279605/) (a.wholelottanothing.org, April 13, 2023)
- [Replacing my best friends with an LLM trained on 500,000 group chat messages](https://links.thingelstad.xyz/links/d97cf5a7e0ab/) (izzy.co, April 13, 2023)

## Elsewhere

- Wayback Machine: https://web.archive.org/web/*/https://simonwillison.net/2023/Apr/12/code-interpreter/