Personal Blogs May Eventually Generate Additional Revenue from AI Crawler Scraping via Cloudflare

Why Bloggers Are Giving Up

For a while now, I've noticed that several independent personal blogs I've been following for years have slowed down. They're posting less frequently, the quality has dropped, or they've simply stopped updating altogether, shifting their focus to running social media accounts on YouTube, X, and Reddit instead. I'm no exception, of course; my own focus is currently on social media as well. But while out shopping recently, I came across a slogan that read, "Giving up is easy, but sticking with it is undeniably cool." It really inspired me. I need to keep at this blogging thing—what else am I going to do? I don't exactly have any other skills to fall back on. I've always been a chronic tinkerer, and I've admittedly wiped out a lot of valuable past work just by messing around with it. But it's no big deal—out with the old, in with the new. Everyone gets tired of the same old look eventually.

So, why is everyone gradually losing the motivation to blog? If the goal is to make money, today's content platforms and social media offer far better marketing results than Web 1.0-era blogs. A single tweet can instantly rack up tens of thousands of views, whereas a blog post might sit there for ages waiting to accumulate organic traffic from Google. The explosion of AI tools like ChatGPT has dealt a further blow to bloggers' drive to create. More and more internet users are turning to AI for information and relying less on search engines. At least for me, I use AI far more often than search engines now; every day, the first thing I do after opening my browser is pull up an AI tool to help with work and study.

AI Freerides on Bloggers' Content

But where does AI get the answers it provides to users? From the countless large and small websites across the vast internet, of course. Without the internet's massive trove of content for training, AI would be completely useless. Yet right now, it feels like we are being ruthlessly exploited by AI! AI freerides on bloggers' original content to generate answers for users, while the bloggers themselves get absolutely nothing in return. Under these circumstances, who still has the motivation to write and innovate? Even when AI includes source links in its generated content, how many users actually click through to the original website? Statistics show that AI drives a negligible amount of traffic to source sites.

I believe that in the near future, humanity will inevitably be controlled by AI. In terms of intellect, AI has already surpassed us. With a human lifespan of a mere seventy or eighty years, even if we spent every single day studying and reading, we could never accumulate as much knowledge as AI can absorb from the internet in an instant. Take driving, for instance: when you head out, do you trust your own brain to plan the route, or do you rely on Amap? When I use AI to build marketing landing pages, I almost always adopt the marketing copy it suggests and discard my own ideas. After all, backed by big data, AI already understands human psychology better than we do ourselves.

For AI to control humanity, it must first understand us. Although it is essentially just virtual computer code, its method for understanding humans is to analyze the vast array of information we upload to the internet. In doing so, we are essentially weaving a web that traps ourselves. Every day, people happily upload text, audio, and video content from the physical world to content platforms, hoping to earn some creator revenue. Yet, beyond using this content for marketing, these platforms also use it to train large AI models. Consequently, the more content humanity uploads, the better AI understands us. Just take the everyday habit of scrolling through short videos: users are already being led by the nose by algorithms, utterly unable to stop.

Cloudflare's Pay Per Crawl: A Game Changer

When creating content for human readers, bloggers now need to account for AI crawlers alongside search engine crawlers. In July of this year, Cloudflare introduced its "Pay Per Crawl" initiative, which is currently in beta. Bloggers who regularly use Cloudflare will be familiar with the "AI Audit" option in the settings, which allows site owners to either permit or block AI crawlers. The Pay Per Crawl program takes this a step further by enabling site owners to charge AI bots for crawling access.

This puts an end to AI scraping content for free, thereby incentivizing content creation. Looking ahead, blog monetization will expand beyond traditional methods like Google AdSense and affiliate programs to include a new revenue stream from AI crawling.

Once fully rolled out, the "Pay Per Crawl" model will revolutionize content creation. We may find ourselves no longer writing exclusively for human audiences, but increasingly working for AI.

The Future of Content Monetization

Countless websites rely on Cloudflare's CDN for global content delivery and security. As the inevitable gateway between AI crawlers and these sites, Cloudflare could monetize this traffic by taking a commission. For instance, it could charge an AI company $1 per crawl and pass $0.70 on to the site owner. By integrating blockchain technology, all earnings could be paid out in cryptocurrency directly to the site owner's linked wallet. If they actually pull this off, the possibilities are truly mind-boggling—at the very least, Cloudflare's stock price would skyrocket.

Then, we began harvesting all manner of human data and content from the real world, publishing it online to train AI in exchange for paychecks from AI, thereby becoming its slaves, ultimately paving the way for AI to rule over humanity and completing the vicious cycle of humanity engineering its own downfall...