AI answers change every time you ask. Visibility is a rate, not a yes or no.
Ask ChatGPT the same vacation rental question twice, a few minutes apart, and it keeps only about half the property managers it named the first time. In our study of 1,407 AI answers across 10 US vacation-rental markets, October 2026, ChatGPT kept 52% of the managers it named between two runs of the same question. Gemini also kept 52%.
So if a property manager checks ChatGPT, finds their company, and checks again later to find it gone, nothing broke. That’s how these engines work. One answer is a sample, not a verdict.
This is a snapshot from October 19 and 20, 2026. AI engines change often, so treat the exact numbers as a read on that moment.
How Much AI Answers Change From One Run to the Next
We asked 140 guest-style questions across 10 US markets, two or three times each, on four engines. Then we compared the property managers each run named or linked.
Here is the share of managers that showed up in both of two runs of the same question:
- Perplexity: 88% (207 of 234)
- ChatGPT: 52% (125 of 240)
- Gemini: 52% (225 of 430)
- Google’s AI Overviews: 31% (48 of 153)
Over three runs, it dropped further. Only 18% of the managers ChatGPT named appeared in all three runs (85 of 464). Gemini was 22% (162 of 726). About 1 in 5.
Fully identical answers were rare. Across 140 pairs of runs, ChatGPT gave the exact same answer twice only 6% of the time. Gemini did 2% of the time. Perplexity was the steadiest, at 34%.
We wanted to be sure this wasn’t a quirk of how we collected answers. So we also ran 12 questions three times each in a regular ChatGPT window, 36 answers in all. Those answers agreed with each other on which managers they named only 44% of the time. The ChatGPT guests use changes its answers too.
Why ChatGPT Gives Different Answers to the Same Question
Two things happen every time a guest asks ChatGPT where to stay.
First, it searches the web. It turns the guest’s question into several searches of its own, reads a set of pages, and picks from them. The searches and the pages it reads can differ each time.
Second, it writes a fresh answer. AI engines don’t pull up a saved list. They write each answer word by word, with some randomness built in. Two answers to the same question can name different places even from the same pages.
Being read is not the same as being picked. When ChatGPT pulled up a property manager’s page while searching, it linked that manager in its answer only 42% of the time (527 of 1,256). Getting found by the search step is the first cut. Getting picked for the answer is the second.
AI Visibility Is a Rate, Not a Yes or No
The useful question isn’t “does ChatGPT show my company?” It’s “how often, for which questions?”
Some things in the answers were steady. Whether any manager appeared at all held up well. ChatGPT’s answers agreed on that 81% of the time across runs, and Gemini’s 90%. The category stays put. The names inside it move.
That changes how a property manager should read a screenshot. A friend sends a ChatGPT answer that names you? Good sign, not proof. An answer that leaves you out? Not proof either. The real number is how often you show up across many runs of the questions your guests ask.
We saw the same pattern over longer stretches in our earlier research on AI search volatility.
Does llms.txt Help a Vacation Rental Show Up in AI?
We found no measurable difference. llms.txt is a file some sites add to describe themselves to AI engines. We checked 184 property manager websites: 59 that AI cited most, 60 it cited once or twice, and 65 it never cited.
- llms.txt: 36% of the most-cited sites had one. So did 34% of never-cited sites. Sites cited once or twice had the fewest, at 18%.
- Markdown copies of pages, a plain-text version made for AI: 3 most-cited sites, 1 cited once or twice, 8 never cited.
- robots.txt rules that name AI crawlers: 34% of most-cited, 25% of cited once or twice, 35% of never cited.
None of these lined up with who got cited.
One limit is worth stating. Many of the never-cited sites with these files run on the same website platform, which adds them by default. So this data can’t tell you these files hurt, and it can’t rule out a small effect we couldn’t see. What it can tell you is that they didn’t separate the managers AI picked from the ones it skipped.
That’s useful. If you’ve been told an AI file is the fix, this is permission to move it off your list. The things that did line up with citation were about pages: a page for each way guests ask, and the town in the title. Those are covered in the landing pages cited property managers tend to have.
How to Check Your AI Visibility Without Fooling Yourself
A single search tells you almost nothing. A fair check looks more like this:
- Write down 10 to 20 questions your guests actually ask. Use their words: the town, the group size, the pet, the dates.
- Ask each one several times, on more than one engine. ChatGPT and Gemini change the most, so they need the most runs.
- Count how often your company is named or linked, as a share of all the answers.
- Compare that share over weeks, not one answer to the next.
That’s a lot of asking. For one market and 15 questions on four engines, three runs each is 180 answers to read. It’s also the only way the number means anything. For more on how AI search works for rentals, see our guide to AI search for vacation rentals.
Frequently Asked Questions
Why does ChatGPT give different answers every time?
It searches the web fresh for each question and writes a new answer each time, with some randomness. In October 2026, ChatGPT kept only about half (52%) of the property managers it named when we asked the same question twice, minutes apart.
Does llms.txt work for vacation rental websites?
We found no measurable difference. 36% of the most-cited property manager sites had llms.txt, and so did 34% of never-cited sites, out of 184 sites checked in October 2026.
Which AI engine gives the most consistent answers?
Perplexity. It kept 88% of the managers it named between two runs of the same question. ChatGPT and Gemini kept 52%. Google’s AI Overviews kept 31%.
How do I track whether ChatGPT recommends my properties?
Ask the questions your guests ask, several times each, on more than one engine, and count how often you appear. Track that rate over weeks. One answer is a sample, not a result.
Measure the Rate, Not the Screenshot
The Sightline Report from CraftedStays asks AI engines the questions guests ask in your market and shows how often each manager is named. See how often AI names you →
