{
  "id": "0ef43037de4f",
  "url": "https://simonwillison.net/2023/Apr/12/code-interpreter/",
  "title": "Running Python micro-benchmarks using the ChatGPT Code Interpreter alpha",
  "note": "This is an incredible way to use ChatGPT. It requires some alpha plug-in access most people don't have.\n\n> Here's the initial prompt I sent it:\n> \n> > In Python create an in-memory SQLite database with 100 tables each with 10 columns.\n> > \n> > Time how long it takes to execute `PRAGMA schema_version` against that database 100 times.\n> > \n> > Then add another 100 tables and time PRAGMA schema_version 100 times again.\n> > \n> > Now run the whole benchmark a second time, but instead of `PRAGMA schema_version` time how long it takes to run hashlib.md5(db.execute(\"select group_concat(sql) from sqlite_master\").fetchall()[0]).hexdigest() instead\n\nOkay, that is a wild set of requirements that would take a significant amount of time to write a testing harness for.\n\n> From start to finish, this entire benchmarking exercise took me less than five minutes—because ChatGPT did almost all of the work for me.\n\nWillison goes on to tweak and modify this benchmark to make it more useful.\n\nThis whole process is remarkable. 🤯",
  "voice": "mixed",
  "saved": "2023-04-14T02:00:32Z",
  "host": "simonwillison.net",
  "kind": "page",
  "tags": [],
  "record": "https://links.thingelstad.xyz/links/0ef43037de4f/",
  "markdown": "https://links.thingelstad.xyz/links/0ef43037de4f.md",
  "issues": [
    {
      "number": 250,
      "slug": "weekly-thing-250",
      "title": "Weekly Thing #250 / LLM, Shanghai, Wing",
      "date": "2023-04-16T12:00:00Z",
      "section": "Notable",
      "position": 5,
      "url": "https://weekly.thingelstad.com/archive/250/",
      "page": "https://links.thingelstad.xyz/issues/250/"
    }
  ],
  "connections": {
    "same_host": {
      "count": 61,
      "page": "https://links.thingelstad.xyz/hosts/simonwillison.net/",
      "nearest": [
        {
          "id": "fa1b1efc4200",
          "title": "Bing: “I will not harm you unless you harm me first”",
          "url": "https://simonwillison.net/2023/Feb/15/bing/",
          "record": "https://links.thingelstad.xyz/links/fa1b1efc4200/"
        },
        {
          "id": "fd6fb5be4b85",
          "title": "Think of language models like ChatGPT as a “calculator for words”",
          "url": "https://simonwillison.net/2023/Apr/2/calculator-for-words/",
          "record": "https://links.thingelstad.xyz/links/fd6fb5be4b85/"
        },
        {
          "id": "8144e594e273",
          "title": "Prompt injection: What’s the worst that can happen?",
          "url": "https://simonwillison.net/2023/Apr/14/worst-that-can-happen/",
          "record": "https://links.thingelstad.xyz/links/8144e594e273/"
        },
        {
          "id": "228d2b2da890",
          "title": "Leaked Google document: “We Have No Moat, And Neither Does OpenAI”",
          "url": "https://simonwillison.net/2023/May/4/no-moat/",
          "record": "https://links.thingelstad.xyz/links/228d2b2da890/"
        },
        {
          "id": "4c7d45a62ac2",
          "title": "Mojo may be the biggest programming advance in decades",
          "url": "https://simonwillison.net/2023/May/4/mojo/",
          "record": "https://links.thingelstad.xyz/links/4c7d45a62ac2/"
        }
      ]
    },
    "same_week": {
      "count": 26,
      "nearest": [
        {
          "id": "e74ce0f4fc52",
          "title": "How to plan as an engineering executive. | Irrational Exuberance",
          "url": "https://lethain.com/planning/",
          "record": "https://links.thingelstad.xyz/links/e74ce0f4fc52/"
        },
        {
          "id": "8e92096170e2",
          "title": "The AI singularity is here | InfoWorld",
          "url": "https://www.infoworld.com/article/3693089/the-ai-singularity-is-here.html",
          "record": "https://links.thingelstad.xyz/links/8e92096170e2/"
        },
        {
          "id": "3850b41f9d96",
          "title": "Importing tweets to Micro.blog - Manton Reece",
          "url": "https://www.manton.org/2023/04/11/importing-tweets-to.html",
          "record": "https://links.thingelstad.xyz/links/3850b41f9d96/"
        },
        {
          "id": "3ec9ab279605",
          "title": "The one good HomeKit-capable wall switch you should get – A Whole Lotta Nothing",
          "url": "https://a.wholelottanothing.org/2023/04/11/the-one-good-homekit-capable-wall-switch-you-should-get/",
          "record": "https://links.thingelstad.xyz/links/3ec9ab279605/"
        },
        {
          "id": "d97cf5a7e0ab",
          "title": "Replacing my best friends with an LLM trained on 500,000 group chat messages",
          "url": "https://www.izzy.co/blogs/robo-boys.html",
          "record": "https://links.thingelstad.xyz/links/d97cf5a7e0ab/"
        }
      ]
    },
    "mentions": [],
    "mentioned_by": [],
    "saved_twice": []
  },
  "elsewhere": {
    "wayback": "https://web.archive.org/web/*/https://simonwillison.net/2023/Apr/12/code-interpreter/"
  }
}