manuelmoreale.com · Saved September 28, 2023

Bots, Spiders, and Crawlers: The Results – Manu

manuelmoreale.com/bots-spiders-and-crawlers-the-results

What Jamie wrote

I've long assume robots.txt is ignored by most robots. Moreale's experiment is good proof of that. If you really want to blog robots, you'll need to do it on the HTTP server and it is going to be a big pain.

Same host · 8 links from manuelmoreale.com

Places on the web – ManuSep 4, 2023
On Ad Blockers – ManuNov 21, 2023 WT271
On POSSE – ManuMar 10, 2024 WT280
The web is not dying – ManuApr 28, 2024 WT286
On hobbies, side projects, and money – ManuAug 18, 2024

All 8 links from manuelmoreale.com

Saved the same week · 17 links that week

The Screens are the Symptom | danah boydzephoria.org · Sep 28, 2023 WT263
macOS Sonoma: The MacStories Review - MacStoriesmacstories.net · Sep 28, 2023 WT263
How Mastercard sells its ‘gold mine’ of transaction datapirg.org · Sep 28, 2023 WT263
The World’s Best Pancake Recipekottke.org · Sep 28, 2023 WT263
Using ChatGPT as an editor – recursion by Luke Franclrecursion.org · Sep 28, 2023 WT263

Everything saved in September 2023