{
  "id": "072f04967148",
  "url": "https://simonwillison.net/2025/May/2/what-we-missed-with-sycophancy/",
  "title": "Expanding on what we missed with sycophancy",
  "note": "Willison's commentary on OpenAI recent issue with \"sycophancy\" in the model. A recent model from OpenAI was overly praising — and for those that have used AI a lot you may be thinking they already do a pretty decent job of making you feel smarter and better than you probably are. This excerpt from Willison caught my eye.\n\n> Here's yet more evidence that the entire AI industry runs on \"vibes\":\n> \n> > In addition to formal evaluations, internal experts spend significant time interacting with each new model before launch. We informally call these \"vibe checks\"--a kind of human sanity check to catch issues that automated evals or A/B tests might miss.\n\nThis tickles a bit of unknown that bothers me in these LLM's and also bothered me with Neural Nets in general — they are black boxes to everyone. Nobody really knows what happens inside. So how do you train them — with \"vibe checks\"? Sounds a lot like how we dealt with way smaller models. \n\nOn one hand I can get comfortable with this. Neuroscientists don't specifically understand how our brains work after all, and we don't discount all input from our brain because of that. Mystery at the center of it all may be just fine.\n\nHowever it does put in stark contrast a need for determinism. In computing we have many things that fundamentally need to be deterministic. We need to be mindful of the problem we are solving, and insure that an open approach, where we fundamentally don't know the \"why\" of any of it, is okay.\n",
  "voice": "mixed",
  "saved": "2025-05-06T02:19:26Z",
  "host": "simonwillison.net",
  "kind": "page",
  "tags": [],
  "record": "https://links.thingelstad.xyz/links/072f04967148/",
  "markdown": "https://links.thingelstad.xyz/links/072f04967148.md",
  "issues": [
    {
      "number": 318,
      "slug": "318",
      "title": "Weekly Thing 318 / Sycophancy, Rollerblades, Yoga",
      "date": "2025-05-10T11:00:00Z",
      "section": "Notable",
      "position": 0,
      "url": "https://weekly.thingelstad.com/archive/318/",
      "page": "https://links.thingelstad.xyz/issues/318/"
    }
  ],
  "connections": {
    "same_host": {
      "count": 61,
      "page": "https://links.thingelstad.xyz/hosts/simonwillison.net/",
      "nearest": [
        {
          "id": "f44b8768a2c3",
          "title": "CaMeL offers a promising new direction for mitigating prompt injection attacks",
          "url": "https://simonwillison.net/2025/Apr/11/camel/",
          "record": "https://links.thingelstad.xyz/links/f44b8768a2c3/"
        },
        {
          "id": "d0074f537816",
          "title": "Backfill your blog",
          "url": "https://simonwillison.net/2025/Apr/25/backfill-your-blog/",
          "record": "https://links.thingelstad.xyz/links/d0074f537816/"
        },
        {
          "id": "6f77e199569f",
          "title": "llm-prices.com",
          "url": "https://simonwillison.net/2025/May/7/llm-prices/",
          "record": "https://links.thingelstad.xyz/links/6f77e199569f/"
        },
        {
          "id": "8ff0eb636351",
          "title": "Highlights from the Claude 4 system prompt",
          "url": "https://simonwillison.net/2025/May/25/claude-4-system-prompt/",
          "record": "https://links.thingelstad.xyz/links/8ff0eb636351/"
        },
        {
          "id": "17c8a3bb1985",
          "title": "Large Language Models can run tools in your terminal with LLM 0.26",
          "url": "https://simonwillison.net/2025/May/27/llm-tools/",
          "record": "https://links.thingelstad.xyz/links/17c8a3bb1985/"
        }
      ]
    },
    "same_week": {
      "count": 16,
      "nearest": [
        {
          "id": "aaa27f8a2155",
          "title": "Wherein I Go to Yoga (for the first time) | ruk.ca",
          "url": "https://ruk.ca/content/wherein-i-go-yoga-first-time",
          "record": "https://links.thingelstad.xyz/links/aaa27f8a2155/"
        },
        {
          "id": "73c1029940e6",
          "title": "Redis is open source again - <antirez>",
          "url": "https://antirez.com/news/151",
          "record": "https://links.thingelstad.xyz/links/73c1029940e6/"
        },
        {
          "id": "35d6a3ab88dd",
          "title": "Attachments helped email go global",
          "url": "https://buttondown.com/blog/email-attachments-history",
          "record": "https://links.thingelstad.xyz/links/35d6a3ab88dd/"
        },
        {
          "id": "005197d763c8",
          "title": "How I Created Perfect Wiki and Reached $250K in Annual Revenue Without Investors / Habr",
          "url": "https://habr.com/en/articles/905812/",
          "record": "https://links.thingelstad.xyz/links/005197d763c8/"
        },
        {
          "id": "9ab22ecec562",
          "title": "Post-Chat UI - Allen Pike",
          "url": "https://allenpike.com/2025/post-chat-llm-ui",
          "record": "https://links.thingelstad.xyz/links/9ab22ecec562/"
        }
      ]
    },
    "mentions": [],
    "mentioned_by": [],
    "saved_twice": []
  },
  "elsewhere": {
    "wayback": "https://web.archive.org/web/*/https://simonwillison.net/2025/May/2/what-we-missed-with-sycophancy/"
  }
}