# Computational Power and AI - AI Now Institute

<https://ainowinstitute.org/publication/policy/compute-and-ai>

- Host: [ainowinstitute.org](https://links.thingelstad.xyz/hosts/ainowinstitute.org/)
- Saved: October 5, 2023 (2023-10-06T00:05:40Z)
- Record: https://links.thingelstad.xyz/links/352a7e075c3b/
- JSON: https://links.thingelstad.xyz/links/352a7e075c3b.json

## What Jamie wrote

This is a very broad and reasonably deep description of how compute power is used to build and train models, and why this is hard and very expensive.

> Compute costs are predictably large: the final training run of GPT-3 is estimated to have cost somewhere between $500,000 to $4.6 million. Training GPT-4 may have cost in the vicinity of $50 million, but the overall training cost is probably more than $100 million because compute is required for trial and error before the final training run.

Those costs are huge, and it is a ton of power too. It is interesting that the big innovations over the last few years — crypto and AI — are both very high compute demand workloads. Crypto has an answer with proof-of-stake. It isn't clear what will make AI costs go down for model building.

## Facts

- Kind: [Page](https://links.thingelstad.xyz/kinds/page/)

## Appears in

- [WT264: Weekly Thing 264 / Dazed, Zany, Llama](https://weekly.thingelstad.com/archive/264/), Notable

## Saved the same week

- [We were never supposed to see our own faces this much | Dazed](https://links.thingelstad.xyz/links/e13217efcdc8/) (dazeddigital.com, October 5, 2023)
- [bruno](https://links.thingelstad.xyz/links/baefe77cf2ae/) (usebruno.com, October 5, 2023)
- [Amazon launches its Bedrock generative AI service in general availability | TechCrunch](https://links.thingelstad.xyz/links/28dc0eff33bc/) (techcrunch.com, October 5, 2023)
- [How To Turn Off Google’s “Privacy Sandbox” Ad Tracking—and Why You Should | Electronic Frontier Foundation](https://links.thingelstad.xyz/links/67193714aa0a/) (eff.org, October 5, 2023)
- [Introducing: Raspberry Pi 5! - Raspberry Pi](https://links.thingelstad.xyz/links/c529f926e647/) (raspberrypi.com, October 5, 2023)

## Elsewhere

- Wayback Machine: https://web.archive.org/web/*/https://ainowinstitute.org/publication/policy/compute-and-ai