Skip to content
createwithlogic

Original data · October 14, 2026

Can AI assistants read
your real estate website?

We took 197 real estate agent and team websites from Google Maps in 10 US metros and loaded each one without running JavaScript, the way AI crawlers first fetch a page. Robots.txt is rarely the problem: one live site blocks ChatGPT’s search crawler. The problem sits in front of the text. For 42 of the 189 sites we could check (22%), a request without JavaScript did not get the site’s text: it was turned away, got no answer, or found a home page with almost no words. To see what an assistant reads before it names an agent, we add the sources Google’s AI cited when we asked for the “best real estate agent” in 20 large US cities.

Free live session on October 22: how to get recommended by ChatGPT

For US real estate agents and accounting firms. Leave your email for a seat, and before the session I run this check and an AI answer check on your business.

One email with the date and link, one reminder. No newsletter.

Tim Muenzel

Tim Muenzel

builds and runs the systems himself

How we checked

On October 14, 2026 we pulled businesses with a website from Google Maps for “real estate agent” in Austin, Dallas, Houston, Miami, Atlanta, Charlotte, Phoenix, Denver, Nashville and Seattle, the same 10 metros as our check of CPA firm websites a day earlier, and kept agents, teams and agencies: 197 websites. We loaded the address from each Google listing with a common browser user agent and without running JavaScript, counted the visible words, looked for structured data that names the agent or firm, and read robots.txt for ten crawlers with a parser that follows the robots.txt standard (RFC 9309). Every site that showed no page the first time we asked again about 20 minutes later, one site at a time. 4 sites on one website platform had first answered “too many requests” and then showed their pages, so we count them as answered. 8 sites on another platform answered both times with a notice page instead of the site, while two other sites on that platform showed their pages, so we count those eight as not checked. That leaves 189. We loaded the pages with almost no text again in a real browser.

Robots.txt is rarely the problem

Of the 163 sites that answered with a page, 3 block OAI-SearchBot, the crawler OpenAI documents for surfacing websites in ChatGPT search. Two of the three are not live websites, a “coming soon” page and an error page, and block every crawler. One live site blocks ChatGPT’s crawlers by name. 4 sites (2.5%) block PerplexityBot, and 10 (6.1%) block GPTBot, which OpenAI uses for training rather than search. Among CPA firm websites, 13% blocked PerplexityBot, all of them on one website platform. Agent sites show no such cluster.

One in eight sites turned our request away

23 of the 189 sites (12%) did not show us the site: 16 answered with a Cloudflare page that asks the browser to prove itself by running JavaScript, 3 with the same kind of check from Vercel, and 4 refused outright. Another 3 did not answer. Among CPA firm websites, 13 of 200 (6.5%) turned the same request away. AI crawlers do not run JavaScript either, so they get past such a check only if the site’s firewall is set to let them through, and nobody outside can see that setting. One brokerage that turned us away was itself a source in Google’s AI answer for its city, so Google’s crawler gets through. Whether OAI-SearchBot, PerplexityBot and Claude-SearchBot get through is a question for whoever runs the site. Cloudflare, behind 16 of these checks, announced on July 1, 2025 that it blocks AI crawlers on new domains by default and that owners can allow them at any time.

16 home pages are almost empty without JavaScript

16 of the 163 pages (9.8%) had fewer than 50 visible words without JavaScript. In a real browser, 6 of them filled in with 334 to 1,282 words: their text only appears through JavaScript, and 4 of those 6 run on the same website platform. Vercel and MERJ analyzed AI crawler traffic and reported on December 17, 2024 that none of the major AI crawlers rendered JavaScript, so those crawlers got a page without text. In our own server logs on October 14, 2026, OpenAI’s search crawler ran the JavaScript on 4 of the 7 pages it fetched, and ChatGPT-User, PerplexityBot and GPTBot showed no sign of it. 4 more showed a “Checking your browser” page from Google’s reCAPTCHA instead of the site, all four with the same robots.txt template. 4 were not a working website: a parked domain, a “coming soon” page, an error page and a brokerage login page, each linked from the agent’s Google listing. One loads another site in a frame, and one is a splash page with 20 words. Among CPA firm websites, 3 of 184 (1.6%) showed their text only with JavaScript.

Nearly half the sites do not say in markup who the agent is

76 of the 163 sites (47%) had no structured data that names the agent or firm as a person, organization or local business, against 35% for CPA firms. 46 (28%) used RealEstateAgent, the schema.org type for agents and brokerages. Corrected on October 15, 2026: we first reported 80 (49%) and 34 (21%), because our script missed types written as a list, such as "@type": ["RealEstateAgent", "Person"], and checked only the first ten types on pages with many. Reading the affected sites again, 4 of the 80 name the agent or firm and 12 more use RealEstateAgent. Sites we could not read again stay as first counted, and one site that had changed its markup since is not counted. 8 of the 197 Google listings link to a page the agent does not control: a brokerage profile or office page (6, one of them a login page) or a link-in-bio page (2). 3 of those 8 turned our request away.

What Google’s AI cites for “best real estate agent in [city]”

On October 13, 2026 we asked Google, from the US, “best real estate agent in [city]” for 20 large US cities. 17 AI Overviews appeared, 15 with sources. Google cited its own business listings in 13 (87%), RealTrends rankings of agents by sales volume in 7 (47%) and the US News real estate agent directory in 3 (20%). Zillow and Redfin came up in 2 each. In 14 of the 15, at least one agent’s, team’s or brokerage’s own website was among the sources. The answers read agent websites, so it matters whether a crawler can read yours.

The named agents have far more reviews

Google’s AI named 5 agents with a rating and a review count: a median of 1,136 reviews at 4.9 stars. Across the 197 agents in our Google Maps sample, the median was 81 reviews at 5.0 stars, and 8 had 1,000 or more. The named agents had about 14 times as many reviews as the median agent, at the same or a slightly lower rating. Five agents is a small sample, so read it as a direction.

What to do this week

Open your website with JavaScript switched off in the browser settings. If little or no text is left, ask your website provider whether the pages can carry their text in the HTML. Ask whoever runs your firewall or hosting whether OAI-SearchBot, ChatGPT-User, PerplexityBot, Perplexity-User, Claude-SearchBot and Claude-User get past its bot check. Check that your Google listing links to your own, live website. Then work on the sources the assistants cite: your Google Business Profile and its reviews, rankings such as RealTrends, which ranks agents by verified sales volume, directories such as US News, and your Zillow and Redfin profiles. Measure again with the same questions after 30 days.

The numbers at a glance

Measured on October 14, 2026. Websites: 197 US real estate agents from Google Maps in 10 metros, 189 checked, 163 answered with a page. AI answers: Google AI Overviews for “best real estate agent” in 20 US cities on October 13, 2026.

Sites that turned a request without JavaScript away 23 of 189 (12.2%), CPA firms 6.5%
Sites that block OAI-SearchBot (ChatGPT search) 3 of 163 (1.8%), 2 of them not live
Sites that block PerplexityBot (Perplexity search) 4 of 163 (2.5%), CPA firms 13%
Sites that block GPTBot (OpenAI training) 10 of 163 (6.1%)
Home page with fewer than 50 words without JavaScript 16 of 163 (9.8%)
Text only appears with JavaScript 6 of 163 (3.7%), CPA firms 1.6%
No structured data naming the agent or firm 76 of 163 (47%), CPA firms 35%, corrected October 15
Agent AI Overviews citing an agent, team or brokerage website 14 of 15 with sources
Reviews of agents named by Google’s AI Median 1,136 (5 agents), against 81 for all 197

The website checks follow fixed rules, and anyone can repeat them with the script on GitHub. The AI answer sample is small and comes from one day and one engine, so read those shares as a direction.

Common questions, answered straight

Does my real estate website block ChatGPT?

Probably not through robots.txt: one live site out of 163 in our sample blocked ChatGPT’s search crawler there. Check two other things. Does your home page show text with JavaScript switched off? And does your firewall’s bot check let AI crawlers through? In our sample, 6 sites showed their text only with JavaScript, and 23 of 189 turned a request without JavaScript away.

Can my website platform be the problem?

Yes. The problems clustered: 4 of the 6 sites whose text only appears with JavaScript run on one platform, and the 4 sites that showed a reCAPTCHA check share one robots.txt template. Ask your provider two questions: do the pages carry their text in the HTML without JavaScript, and do OAI-SearchBot, PerplexityBot and Claude-SearchBot get past the bot protection?

Which crawlers should a real estate agent allow?

For AI search: OAI-SearchBot and ChatGPT-User for ChatGPT, PerplexityBot and Perplexity-User for Perplexity, Claude-SearchBot and Claude-User for Claude, Googlebot for Google’s AI Overviews and AI Mode, and bingbot for Bing and Copilot. Training crawlers such as GPTBot and ClaudeBot are a separate decision that does not affect search. Google-Extended does not affect Google Search, but Google says it also decides whether Gemini Apps may use your pages to ground their answers.

Does it matter if my Google listing links to my brokerage profile?

The page an assistant finds through your listing then belongs to your brokerage, so you cannot change what it says or how it is served. In our sample, 8 of 197 listings linked to a brokerage page or a link-in-bio page, and 3 of those turned our request away. In 14 of 15 Google AI answers about agents, an agent’s, team’s or brokerage’s own website was a source.

Can you run this check on my website?

Yes. Every agent who registers for the free session on October 22 gets this check and a check of what ChatGPT, Gemini, Perplexity and Google AI Mode say, before the session. You can also run the script on GitHub yourself: it sends two requests and prints the same checks for your site.

Sources

Related reading

Want to know what the assistants say about you?

Join the free live session on October 22. Every registered agent gets this check and an AI answer check before it starts.

Save a seat in the free session Book a free 20-minute call