# Scaling Kafka to Support PayPal’s Data Growth | PayPal Technology Blog

<https://medium.com/paypal-tech/scaling-kafka-to-support-paypals-data-growth-a0b4da420fab>

- Host: [medium.com](https://links.thingelstad.xyz/hosts/medium.com/)
- Saved: September 28, 2023 (2023-09-28T12:52:05Z)
- Record: https://links.thingelstad.xyz/links/1795068dac15/
- JSON: https://links.thingelstad.xyz/links/1795068dac15.json

## What Jamie wrote

Scaling tech to extreme scales is usually an interesting story. The levels that PayPal has scaled [Kafka](https://kafka.apache.org) are impressive!

> Today, our Kafka fleet consists of over 1,500 brokers that host over 20,000 topics and close to 2,000 Mirror Maker nodes which are used to mirror the data among the clusters, offering 99.99% availability for our Kafka clusters. During the 2022 Retail Friday, Kafka traffic volume peaked at about 1.3 trillion messages per day! At present, we have 85+ Kafka clusters, and every holiday season we flex up our Kafka infrastructure to handle the traffic surge. The Kafka platform continues to seamlessly scale to support this traffic growth without any impact to our business.

Detailed article describing how they have achieved this level of scale and resiliency, including their own custom libraries they built to interface with Kafka.

## Facts

- Kind: [Page](https://links.thingelstad.xyz/kinds/page/)

## Appears in

- [WT263: Weekly Thing 263 / Copilot, Del.icio.us, Pancake](https://weekly.thingelstad.com/archive/263/), Notable

## Same host

- [Stability without Pegs | Reflexer](https://links.thingelstad.xyz/links/a076e598383e/) (medium.com, December 7, 2022)
- [Bitcoin is the Detector of Imbeciles | by Nassim Nicholas Taleb | INCERTO | Jan, 2023 | Medium](https://links.thingelstad.xyz/links/6139d6f695ea/) (medium.com, January 12, 2023)
- [Introducing Ethical Monitor: A Privacy-Focused Service for Dappnode Alerts | DAppNode](https://links.thingelstad.xyz/links/498abb278ecb/) (medium.com, January 18, 2023)
- [RAI, The Free-Floating Stablecoin That Actually Works | IntoTheBlock](https://links.thingelstad.xyz/links/3a672c4fe962/) (medium.com, March 16, 2023)
- [Kernighan and Pike were right: Do one thing, and do it well | by Keaton Brandt | Source and Buggy | Jul, 2023 | Medium](https://links.thingelstad.xyz/links/1f4a7e1a33d5/) (medium.com, August 24, 2023)

## Saved the same week

- [Announcing Microsoft Copilot, your everyday AI companion - The Official Microsoft Blog](https://links.thingelstad.xyz/links/c540bb1e0103/) (blogs.microsoft.com, September 28, 2023)
- [The invisible problem – Scott Jenson](https://links.thingelstad.xyz/links/96f256d6035d/) (jenson.org, September 28, 2023)
- [AWS: Cannot Escape IPv4](https://links.thingelstad.xyz/links/db5a3b9ae1aa/) (tty.neveragain.de, September 28, 2023)
- [Odin Programming Language](https://links.thingelstad.xyz/links/18ea2704dfb0/) (odin-lang.org, September 28, 2023)
- [OpenAI Cookbook](https://links.thingelstad.xyz/links/f295cd1ee91b/) (cookbook.openai.com, September 28, 2023)

## Elsewhere

- Wayback Machine: https://web.archive.org/web/*/https://medium.com/paypal-tech/scaling-kafka-to-support-paypals-data-growth-a0b4da420fab