AI crawlers can knock over a WordPress site on their own. They walk your entire URL space rather than a few pages, they arrive in bursts, and the badly behaved ones rotate through address ranges so you can’t rate-limit them cleanly. The standard response is to block every bot you can identify, which is a sound operational decision that quietly costs you something: being findable inside ChatGPT, Claude, and Gemini, which is increasingly where people start looking.
On a static site that trade-off doesn’t exist. Serving a crawler costs about what serving a person costs, which is nothing, so whether you want to be cited stays a decision you make on the merits instead of one your server makes for you.
ℹ What this post is not about
Whether AI is good for the web, what the data centers cost, what they do to electricity rates. Those are real arguments and this isn’t one of them. The question here is narrower and practical: can people using chatbots find you, and does letting them in cost you anything you care about?
This one is more technical than most of what I write here, on purpose. It’s aimed at whoever the business owner asks before deciding.
Why blocking is the rational first move
On WordPress, every crawler request is a full page build. PHP starts, the database gets queried, the page gets assembled and thrown away. Crawlers don’t request one page and leave. They walk everything you expose, and WordPress exposes a lot: every tag, category, author, date archive, attachment page, and paginated variant is its own address. A forty-page site can present several hundred crawlable URLs.
Then there’s the behavior. The well-mannered crawlers publish their user agents, respect robots.txt, and pace themselves. The rest don’t. Some run from botnets or rotate through residential IP ranges specifically so you can’t identify and throttle them, which makes them indistinguishable from a slow attack. When your site is struggling and you can’t tell which requests are legitimate, blocking everything you can name is the fastest way to get your afternoon back.
I want to be clear that this is a defensible choice. Nobody blocking AI crawlers on a struggling WordPress site is being short-sighted. They’re solving the problem in front of them with the tool that works.
Almost nobody sat down and decided AI visibility wasn’t worth it. It arrived as a performance problem and got solved as one.
The mechanics of what crawlers do to a WordPress site are covered separately, including why the URL fan-out is worse than people expect.
On static files the same crawl is a CDN handing over cached documents. It costs nothing and slows nothing down, for the crawler or for the human reading three pages away. The operational pressure that forces the decision never shows up.
⚠ Check what your defaults are doing
Cloudflare begins blocking AI crawlers by default on 15 September 2026, alongside a pay-per-crawl scheme, and its bot-fighting settings already block a good deal before that. Read the scope rather than assuming: the new default targets ad-supported publishers, so most small business sites need to opt in deliberately. Either way, go look at what your own configuration currently blocks. A setting somebody enabled during an incident two years ago is still a setting.
You author the output, or you negotiate it
The second advantage is control, and it’s the one I’d actually optimize around.
A static generator emits exactly the markup you write. You decide the heading hierarchy, you decide what JSON-LD blocks appear, and nothing lands in the DOM that you didn’t put there.
On WordPress, your output is a negotiation between the theme, the page builder, whatever SEO plugin you run, and whatever each installed plugin injects. You often end up with two conflicting schema blocks, a heading hierarchy that jumps from H1 to H4 because the theme author liked the type size, and a <head> full of markup nobody chose.
Being fair here matters, because the unfair version of this argument is easy to catch. WordPress is fully capable of good structured data. Yoast emits solid Article, Breadcrumb, and Organization markup, and a carefully configured WordPress site can produce cleaner output than a carelessly built static one. The difference is control and effort, not capability. On a static site the good output is the default and deviating takes work. On WordPress the good output is achievable and takes vigilance.
⚠ What static does not buy you
Being static is not itself a ranking or citation signal, and I’d be careful of anyone selling it as one. A language model receives HTML and has no way of knowing whether it was generated at deploy time or assembled half a second before it arrived. The two advantages here are real and narrow: crawler traffic that costs nothing to serve, and markup you author exactly. Neither is the rendering model doing something clever.
What that looked like here, specifically
On 17 August 2026 this site had FAQ schema on exactly zero blog posts. By the end of that day it had FAQPage markup on its three best-performing posts and opt-in HowTo markup available on any post with a genuine procedure.
What that took: two small Astro components, about 120 lines between them. No plugin, no subscription, no settings panel, no compatibility check against a theme.
Two details from that work worth passing on, because both cost me time:
- The site already had a full-width
FAQSectioncomponent used on service pages. Dropped inside a blog post’s prose container it broke the layout, so blog posts needed their own narrower component. One schema type, two components, for presentational reasons. - I found a comment claiming
StepListemittedHowToschema. It never had. The HowTo markup I’d seen in the wild was hand-written on three use-case pages years earlier. Check what your components actually emit rather than what their comments claim. View source on a built page.
The FAQ questions themselves came from real Search Console queries rather than invention. That distinction matters more than the schema does.
Should you block AI crawlers?
There’s a real argument for blocking, and skipping past it would be dishonest.
If your content is your product, letting an AI summarize it away is a genuine loss. Publishers watching referral traffic collapse while their words appear in answer boxes have a legitimate grievance. Zero-click is real.
My position for a small services business is the opposite, and I’ll state it plainly: being cited is worth considerably more than your content is worth protecting. You are not in the content business. Your blog posts are evidence that you know what you’re talking about, shown to someone deciding whether to contact you. An AI describing your approach accurately and naming you does the job the post was written to do.
The clearest case I have for this is a law firm that found this site through a post about WordPress costs, then went looking for the term “static site” across several chatbots because a technical friend had given them the vocabulary. That’s the buyer’s actual research path in 2026. Being absent from it isn’t a neutral choice.
If you sell content, block them. If you sell work, don’t.
What to actually do
Nothing exotic. Most of this is ordinary technical SEO that happens to also serve extraction.
Making a static site citable
- 1
Confirm your robots.txt actually ships
Check the built output, not the source repository. Request the live URL and read what comes back. Mine declared a sitemap path that the build step deleted, so every crawler following it got a 404 for months. Verify the sitemap URL in it resolves.
- 2
Allow the AI crawlers deliberately
GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot. Decide once, write it down, and make it explicit in robots.txt rather than leaving it to a default that may change under you.
- 3
Emit Article and Organization schema everywhere
Author attribution is one of the stronger signals for whether a source gets treated as citable. Add sameAs properties linking your organization and author to their profiles elsewhere so the entity resolves to one thing across the web.
- 4
Add FAQPage schema where real questions exist
Pull the questions from Search Console rather than imagining them. Answer in the first sentence and keep answers to roughly 40 to 100 words. Do not pad a post with invented questions to earn the markup.
- 5
Add HowTo schema only to genuine procedures
One HowTo per page, and only where the steps are an actual sequence someone follows. Marking up a list of tips as a procedure is the kind of thing that gets structured data ignored.
- 6
Write claims that can be extracted
A specific number with its context travels. '1,170 blocked requests in 24 hours against about 15 real visitors' can be quoted. 'A lot of bot traffic' cannot. This is the item on the list that pays most, and it is a writing task rather than a technical one.
- 7
State your limitations in the piece
Balanced content gets cited more readily than promotional content, because a system assembling an answer needs the caveats. The section where you explain when your recommendation is wrong is doing real work.
- 8
Verify by reading the built HTML
View source on the deployed page and confirm the schema is present, valid, and not duplicated. Build-time generation means you can be certain what every visitor and every crawler receives, which is the actual advantage over a CMS.
Notice how little of that is about being static. Steps three through seven apply on any platform. Static makes them easier to guarantee and free to serve, which is a real advantage and a smaller one than the marketing around this suggests.
The measurement problem
Worth being straight about: you cannot measure this well.
Search Console does not report AI citations. No analytics product does. You can ask the chatbots questions manually and see who gets named, but if you’re logged into an account that knows who you are, the results are personalized and you’re reading your own reflection. On a low-traffic site the sample size makes any result indistinguishable from noise.
I’ve stopped trying to measure it here and I’d suggest the same. Everything in the list above is either free, because schema costs a reader nothing, or independently good writing, because specific claims and honest limitations make a better post regardless of who reads it. That’s a strong enough reason to do the work without a metric attached.
Key Takeaway
AI crawlers put enough load on a CMS that blocking them becomes the obvious fix, and blocking them makes you invisible to the chatbots people now search with. On static files that pressure never arrives, so allowing or blocking stays a decision you make deliberately. Static also means you author your markup exactly rather than negotiating it with a theme and four plugins. Being static is not itself a citation signal, since a model can’t tell. The optimization work is mostly platform-independent, and the part that pays most is writing specific claims and stating your limitations.
Frequently Asked Questions
- Do AI chatbots prefer static websites?
- No, and they cannot tell the difference. A model receives HTML and has no information about whether it was generated at build time or per request. What actually influences citation is content quality, clear structure, structured data, author attribution, and whether crawlers can reach the site at all.
- Should I block AI crawlers on my website?
- It depends on what you sell. If your content is the product, blocking protects it and the referral-traffic concern is legitimate. If you sell services and your content exists to demonstrate expertise, being cited does the job the content was written for. For a small services business, allow them.
- Does schema markup help with AI search?
- Yes. Structured data explicitly identifies entities, content types, and authorship, which is more reliable for a machine to parse than inferring the same from prose. Article, FAQPage, HowTo, and Organization with sameAs properties are the highest-value types. It costs a human reader nothing, so there's no trade-off in adding it.
- Why do AI crawlers cost more on WordPress than on a static site?
- Every request to a WordPress site triggers a page build: PHP runs and the database is queried. Crawlers walk your whole URL space, and WordPress generates many URLs per page of content through tag, category, author, date, and attachment archives. On static files the same crawl is a CDN serving cached documents at effectively no cost.
- How do I know if my structured data is working?
- Read the built HTML of the deployed page and confirm the JSON-LD is present, valid, and not duplicated, then run it through a structured data validator. You cannot measure AI citations directly, since no analytics product reports them, so verifying correct output is the reliable part of the loop.
For the problem this post answers, start with what AI crawlers do to a WordPress site. For the architecture, here’s what a static site actually is and how the two compare directly.
Reviewing a site for someone who asked you?
If you're the technical friend in this conversation, send me the URL. I'll give you a straight read on what I'd change and what I'd leave alone, no pitch attached.
Founder, 1123Interactive
26 years building for the web. I run the schema on this site by hand, which is how I know exactly what it emits and exactly how long it took to add. The numbers in this post are from my own logs and my own commits.
Learn moreContinue Reading
Eliminate 99.9% of Form Spam in 10 Minutes
Hundreds of spam emails after a WordPress rebuild, fixed in ten minutes with a honeypot field. The simplest form spam defense that still works.
Why Is Your Website a Monthly Bill?
Your website is a monthly bill because of how it was built, not because sites must cost money every month. What's in the charge, and how to end it.
Static Site vs WordPress: Which Should You Choose?
Static sites and WordPress compared on speed, cost, security, and maintenance, including what you give up by leaving and when WordPress is the better choice.