# Do AI crawlers run JavaScript? Mostly, no

Google has rendered JavaScript for years, so many teams assume every crawler does. AI crawlers
mostly do not. If your page text appears only after JavaScript runs, ChatGPT, Claude and
Perplexity are likely to see an empty shell where your content should be.

## What the crawlers actually do

In December 2024, Vercel and MERJ published an analysis of AI crawler traffic across Vercel's
network. Their conclusion: "none of the major AI crawlers currently render JavaScript." The
crawlers they checked include:

- OpenAI: `GPTBot`, `OAI-SearchBot` and `ChatGPT-User`
- Anthropic: `ClaudeBot`
- Perplexity: `PerplexityBot`
- Meta: `Meta-ExternalAgent`
- ByteDance: `Bytespider`
- Common Crawl: `CCBot`

These crawlers do download JavaScript files: 11.5% of ChatGPT's requests and 23.8% of Claude's
were for JavaScript. They just do not execute them.

Two crawlers render pages like a browser:

- **Googlebot.** Gemini uses Googlebot's infrastructure, so Google's AI features see rendered
  pages.
- **Applebot.** Apple says it may render pages in a browser, and the Vercel study observed it
  doing so.

The study was a snapshot, and crawlers change. The safe assumption is still the same: content
that is not in the HTML response may not exist for an AI system.

## What disappears without JavaScript

Anything your server does not send in the HTML response:

- **Single-page app shells.** A page that arrives as `<div id="root"></div>` plus a bundle has
  no content for a crawler that does not run the bundle.
- **Content fetched after load.** Text, prices or reviews that a component loads from an API
  after it mounts.
- **Metadata added by scripts.** A title, description, canonical or JSON-LD block that a
  client-side library writes into the head.
- **Content behind interaction.** "Load more" buttons, infinite scroll and tabs whose content is
  requested only when clicked. Tabs and accordions are fine if their content is already in the
  HTML.
- **Content behind a consent or login wall** that is lifted by a script.

Vercel notes one partial exception: data included in the initial HTML, such as JSON or React
Server Components payloads, "may still be indexed." That is data, not readable text, so do not
rely on it for content you want cited.

## Test a page in a minute

Pick a sentence that only appears in your page's main content, then fetch the raw HTML:

```sh
curl -sS https://example.com/pricing | grep -c "Billed annually"
```

`0` means the sentence is not in the HTML your server sends. You can do the same with a
crawler's user agent, which also shows whether a firewall treats bots differently:

```sh
curl -sS -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" \
  https://example.com/pricing | grep -c "Billed annually"
```

In a browser, open DevTools, run **Disable JavaScript** from the command menu and reload. What
you still see is roughly what an AI crawler gets. **View page source** shows the raw HTML, while
the Elements panel shows the page after scripts have run.

A [CrawlWise scan](/) does this for every audited page: it compares the text in the raw HTML
with the text after rendering in Chromium, and reports how much of the content needs
JavaScript.

## How to fix it

The goal is simple: the HTML response contains the page's main content. Interactive parts can
still use JavaScript.

### Next.js

In the App Router, Server Components are the default, and they render on the server. Content
goes missing when it is fetched in a Client Component inside `useEffect`. Fetch it in a Server
Component instead, or generate the page at build time with static generation. Keep `"use client"`
for the interactive parts, not for whole pages.

### Nuxt, SvelteKit, Remix and Astro

These frameworks render on the server by default. Check for pages or routes where server
rendering has been switched off, such as `ssr: false` in Nuxt, and for content loaded only in
client-side hooks.

### React, Vue or Angular single-page apps

A client-rendered app built with Vite or a similar tool has no server-rendered content. Move
the public pages to a framework that renders on the server, use Angular's server-side rendering,
or prerender the public routes to static HTML at build time. Marketing pages, documentation and
product pages matter most; a logged-in dashboard does not need to be crawlable.

### Rails, Django, Laravel and other server-rendered apps

Server views already send the content. Watch for parts added later with JavaScript, for example
a Turbo Frame with a `src` that loads after the page, or a widget that fetches its content from
an API in the browser. If that part matters, render it in the initial response.

### WordPress and site builders

Most themes render on the server. Third-party widgets are the usual gap: reviews, pricing tables
or FAQs injected by an external script. Prefer blocks or plugins that output HTML.

## Do not serve bots a different page

Serving prerendered HTML only to bots, known as dynamic rendering, is a workaround that Google
does not recommend as a long-term solution. It doubles what you have to keep correct, and if the
bot version differs from what people see, it looks like cloaking. Render the same HTML for
everyone.

## Sources

- Vercel and MERJ: [The rise of the AI crawler](https://vercel.com/blog/the-rise-of-the-ai-crawler)
- Apple: [About Applebot](https://support.apple.com/en-us/119829)
- Google: [Dynamic rendering as a workaround](https://developers.google.com/search/docs/crawling-indexing/javascript/dynamic-rendering)
- Google: [Understand the JavaScript SEO basics](https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics)

---

Last updated 3 November 2026.

HTML version: <https://crawlwise.dev/guides/ai-crawlers-javascript>

Every CrawlWise page in Markdown: <https://crawlwise.dev/llms.txt>