# You Should Probably Pay Attention to Tokenizers - Cybernetist

<https://cybernetist.com/2024/10/21/you-should-probably-pay-attention-to-tokenizers/>

- Host: [cybernetist.com](https://links.thingelstad.xyz/hosts/cybernetist.com/)
- Saved: October 24, 2024 (2024-10-24T22:13:06Z)
- Record: https://links.thingelstad.xyz/links/abed5251cd7b/
- JSON: https://links.thingelstad.xyz/links/abed5251cd7b.json

## What Jamie wrote

Good background on the important role of tokenizers in developing AI solutions using RAG.

## Facts

- Kind: [Page](https://links.thingelstad.xyz/kinds/page/)

## Appears in

- [WT298: Weekly Thing 298 / Tool, Solar, Circles](https://weekly.thingelstad.com/archive/298/), Briefly

## Saved the same week

- [Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku Anthropic](https://links.thingelstad.xyz/links/25ba94e5654e/) (anthropic.com, October 24, 2024)
- [How I Use Claude](https://links.thingelstad.xyz/links/04edf185778e/) (avitalbalwit.com, October 24, 2024)
- [Hands-On With Apple's Genmoji AI Emoji Generator in Beta](https://links.thingelstad.xyz/links/06356ea2f966/) (blog.emojipedia.org, October 24, 2024)
- [The Global Surveillance Free-for-All in Mobile Ad Data – Krebs on Security](https://links.thingelstad.xyz/links/8fa75eda4085/) (krebsonsecurity.com, October 24, 2024)
- [Mersenne Prime Discovery - 2^136279841-1 is Prime!](https://links.thingelstad.xyz/links/b0286c3a0cc7/) (mersenne.org, October 24, 2024)

## Elsewhere

- Wayback Machine: https://web.archive.org/web/*/https://cybernetist.com/2024/10/21/you-should-probably-pay-attention-to-tokenizers/