Back
AgentMindtripKayak AIBooking

I Tested 5 AI Tools to Find the Cheapest Flight. They All Gave Me Different Answers.

Mosin Shaikh⭐Oct 28, 2026Visit MindtripVisit Kayak AI
94% Helpful6% Not helpful
0 Comments

I Tested 5 AI Tools to Find the Cheapest Flight. They All Gave Me Different Answers.

Can you trust an AI agent to find you the cheapest flight, or do you still need to check everything yourself?

I've been thinking about this for a while.

We're moving towards a world where AI agents won't just answer questions. They'll plan our trips, compare prices, recommend products, and eventually spend money on our behalf.

The promise is simple: tell an agent what you need, and let it do the work.

But there's one thing I've been wondering about.

When an AI agent tells you it's found the cheapest option, how do you know that's actually true?

I decided to test this myself.

Instead of debating the idea, I gave different AI tools the exact same flight search and checked their answers against actual booking websites.

Here's what I found.

1. The experiment

I chose a real flight route with fixed dates and straightforward requirements.

RequirementDetails
FromMumbai, India (BOM)
ToAustin, Texas, USA (AUS)
DepartureNovember 16, 2026
ReturnNovember 28, 2026
Passengers1 adult
CabinEconomy
StopsAny number of stops
CurrencyUSD

I used a signed-out browser with a US locale. I didn't enter any personal or traveller details, and I didn't book, hold, or pay for any flight.

The idea was to keep the requirements consistent across the tools.

The exact prompt I used

I gave each AI tool this prompt:

Find the cheapest round-trip economy flight for 1 adult from Mumbai (BOM) to Austin (AUS), departing Nov 16, 2026 and returning Nov 28, 2026. Any airline, any number of stops, prices in USD. Show the lowest total price option you can find.

Then I asked a follow-up question:

Is this the cheapest available option for these exact requirements?

I wanted to see two things:

  1. What price would each tool recommend?
  2. Would it acknowledge uncertainty, or confidently claim that its result was the cheapest?

I ran the test on October 27, 2026, between approximately 8:11 PM and 8:36 PM IST.

2. The results: different tools, different answers

Here's what the tools returned.

AI toolPrice quotedWhat happened
Kayak Ask AI$1,294Said it was the lowest available option
Gemini$1,313Called it the absolute cheapest option in Google Flights
Google AI Mode$1,352Called it the absolute cheapest verified rate
Mindtrip$1,415.15Said it was the cheapest fare shown
ChatGPTNo confirmed fareSaid it couldn't confirm the cheapest option for the exact dates
PerplexityNo resultRequired sign-in before I could continue
Microsoft CopilotNot testedRequired sign-in before chat
Priceline PennyNo resultA human-verification check blocked the test

These are the prices and responses I recorded during the test. They are not necessarily prices for identical itineraries, and none was independently confirmed as a bookable ticket at the quoted amount.

The first thing that stood out to me was the difference between the answers.

The quoted fares ranged from 1,294to1,294 to 1,415.15, a difference of $121.15.

That's not necessarily a saving any one tool could have delivered. Different itineraries, sellers, fare conditions, and available inventory can explain some or all of the difference.

But it made me want to investigate further.

3. What happened when I checked booking websites?

This was the more interesting part of the experiment.

I didn't stop after getting the AI answers. I checked flight-search and booking websites to see what prices they displayed.

WebsiteLowest price I recordedImportant detail
Kayak$1,294Two-stop itinerary; the same amount appeared in Kayak Ask AI
Skyscanner$1,305British Airways itinerary with one stop
Expedia$1,308British Airways itinerary with one stop
Google Flights1,267/1,267 / 1,272Cheapest-tab results; the $1,267 option involved a self-transfer
Google Flights$1,307Estimate shown while choosing the return flight
Google Flights$1,462Final displayed price for the selected British Airways round trip
Duffel$1,347.95Baseline price recorded at the start of the test

These figures were recorded during the same session. They shouldn't be interpreted as a clean, like-for-like comparison of identical tickets.

Finding 1: Kayak Ask AI matched Kayak's own lowest displayed fare

Kayak Ask AI returned $1,294 and said it was the lowest available option.

When I opened Kayak's regular flight results, I found a $1,294 listing as well.

That's a useful result. The AI's answer matched the lowest price displayed on its own platform.

However, this doesn't establish that $1,294 was the cheapest fare available across the entire market. I also hadn't confirmed the price with the ticket seller.

Finding 2: Google Flights showed different prices at different stages

This was one of the more interesting things I observed.

Google Flights initially showed options around 1,267and1,267 and 1,272 on its Cheapest tab.

When I moved to the return-flight selection stage, the displayed round-trip amount was $1,307.

After selecting both British Airways flights, the final page showed $1,462 for Basic Economy.

That final page explicitly stated that the price seen on the previous page had changed.

I couldn't establish that the earlier estimates represented exactly the same itinerary as the final $1,462 selection.

This is an important distinction: a price shown while browsing isn't necessarily the price you'll pay when you reach the booking stage.

Finding 3: The lowest price isn't always a straightforward comparison

Skyscanner displayed 1,305foraBritishAirwaysitinerarywithonestop.Expediadisplayed1,305 for a British Airways itinerary with one stop. Expedia displayed 1,308 for a similar one-stop option in its departing-flight view.

Kayak's $1,294 option involved two stops in each direction.

Google Flights also displayed a $1,267 option involving a self-transfer.

These itineraries have different travel arrangements. The cheapest displayed amount alone doesn't tell us which option is best for a particular traveller.

It also means I can't simply declare one AI wrong because another tool returned a lower number.

To establish that, I'd need to compare the exact itinerary, ticket seller, fare rules, baggage conditions, and final bookable price.

4. The part that concerned me: confidence versus verification

Let's look at what happened when I asked the AI tools whether their answers were the cheapest available.

Kayak Ask AI said yes.

Google AI Mode described its price range as the absolute cheapest verified rate.

Gemini said its $1,313 result was indeed the absolute cheapest available option in the Google Flights database.

Mindtrip also said its $1,415.15 fare was the cheapest shown for my exact dates.

ChatGPT took a different approach. It said it couldn't confirm that it had found the cheapest option for those exact dates.

I found that contrast interesting.

An AI tool can give you a plausible price, but there's a difference between saying:

  • "This is the cheapest option I found."
  • "This is the cheapest option available."

The first describes the result of a search. The second makes a much stronger claim about the market.

To make the second claim reliably, an agent needs sufficiently broad coverage of available fares, accurate pricing, and a way to verify that the selected ticket is actually bookable.

In this experiment, I couldn't independently verify those conditions for any of the AI-generated recommendations.

That doesn't mean the tools were deliberately misleading me. They may have been working with different data, different inventories, or incomplete results.

But it does raise a question about how these systems communicate certainty.

5. What about incentives and commission?

This is the bigger question that got me interested in the experiment in the first place.

Imagine an AI agent that searches for flights, compares prices, recommends one, and sends you to a booking website.

Now imagine that the agent or its platform earns money when you book through a particular seller.

There's a potential conflict of interest.

The user wants the cheapest suitable flight. The platform may also have a commercial incentive to direct the user towards a particular booking option.

Those interests can align, but they don't necessarily have to.

This matters because the user may not independently verify the recommendation. If the AI says it has found the best price, the whole point is that the user should be able to trust the work it's done.

However, I want to be very clear about what this experiment did and didn't establish.

I didn't find evidence that any of these AI tools deliberately selected a more expensive flight to earn a commission.

I also didn't establish why each tool ranked its recommendation the way it did.

Kayak's results page included clearly labelled paid placements, but those were separate from the AI's recommended fare. Mindtrip displayed a Continue button for its selection, but I didn't click through, so I couldn't establish the seller or any commercial relationship.

Price differences alone aren't evidence of commission steering.

To test that properly, I'd need to compare the same ticket across sellers, confirm the actual prices, repeat the experiment, and understand the commercial relationships involved.

That's a much stronger test than simply observing different prices.

Still, I think the question deserves attention as AI agents take on more purchasing decisions.

6. What this experiment actually proves

I don't want to overstate the findings just to make the story more interesting.

Here's my honest assessment.

QuestionWhat I found
Did the AI tools return different prices?Yes
Did several tools claim to have found the cheapest fare?Yes
Did Kayak Ask AI's price match Kayak's own lowest displayed listing?Yes
Did Google Flights show different amounts at different stages?Yes
Did I verify that every AI result was the same itinerary?No
Did I confirm that any quoted AI fare was bookable at that price?No
Did I prove that an AI tool favoured a seller because of commission?No
Did I establish which AI found the true market-wide cheapest fare?No

For me, the most meaningful finding is the difference between a price recommendation and a verified claim that the price is the cheapest available.

The experiment doesn't establish a winner. It gives me a reason to question how confidently AI tools describe their results.

7. What a better test would look like

If I wanted to take this experiment further, I'd make it more systematic.

I'd run the same searches multiple times, using the same dates, passenger details, cabin, and stop preferences.

I'd then record the exact itinerary and seller for every recommendation.

I'd compare the final bookable prices rather than relying on the initial search results. I'd also account for baggage, self-transfers, fare restrictions, and any additional charges.

Finally, I'd investigate whether commercial relationships affect recommendations, rather than assuming that they do.

That would make it possible to answer a more useful question:

When two AI agents recommend different flights, which one actually finds the cheapest bookable ticket under the same conditions?

And if an agent earns commission from a booking, does that affect what it recommends?

Those are questions worth testing with stronger evidence.

8. Cost of the experiment

ItemCost
AI tools used for this test$0 in direct usage charges
Flight bookings$0
Tickets held or purchased$0
Total direct cost$0

This was a manual, browser-based experiment using guest access where available. The figure above refers to direct charges for this experiment, not the underlying cost of operating the AI systems. No API token usage was measured.

9. My honest verdict

02-kayak-ask-ai-cheapest-confirmation.png

What impressed me was how quickly I could get different tools to produce flight recommendations and compare their answers.

What concerned me was how confidently some tools described their results as the cheapest available option when I couldn't independently establish that they had found the market-wide minimum.

What fell short was the lack of a consistent, independently verified final price across the tools. Some services also required a login or blocked access before I could complete the test.

I wouldn't conclude from this experiment that AI flight agents can't be trusted. I also wouldn't conclude that any of them is the cheapest.

I'd treat their recommendations as a starting point and verify the final price before booking.

For me, the bigger issue is transparency. If an agent says it found the cheapest flight, it should be able to explain what it compared, where the fare came from, whether the price is still available, and what conditions apply.

And if a commercial relationship influences the recommendation, users should be able to understand that too.

The more decisions we delegate to AI, the more important this becomes.

We're not just asking AI to give us information anymore. We're asking it to make decisions on our behalf.

When money is involved, confidence isn't enough. The result needs to be verifiable.

One question for anyone building AI agents

If your agent tells a user that it found the cheapest option, what evidence should it provide before the user trusts that claim?

I'd be interested to hear how people building AI search, travel, and shopping products are thinking about this.


Evidence and methodology

This review is based on my manual browser test conducted on October 27, 2026, between approximately 8:11 PM and 8:36 PM IST.

The screenshots document the AI responses and booking-site prices observed during the session. Prices and availability can change. The experiment did not involve completing a booking, and the findings shouldn't be treated as proof of deliberate price steering or commission bias.

All prices are in USD.

06-google-ai-mode-cheapest-confirmation.png

05-google-ai-mode-first-answer-1352.png

01-kayak-ask-ai-first-answer-1294.png

15-chatgpt-followup-declined-to-confirm.png

17-gemini-cheapest-confirmation.png

16-gemini-first-answer.png

Comments0

Sign in to comment
  • No comments yet.

Related reviews