Playing with AI - Florida special election edition

I've been having some fun lately playing around with the various AI models and seeing how they respond to various questions. Today it was Gemini's turn.

Question of the day to Gemini 2.5 Pro:

Which county in Florida swung democrat in the recent special election?

I would think this would be relatively easy to answer given that there were two districts holding special elections in FL within the last week: one to replace Matt Gaetz and one to replace Michael Waltz. The answer of course is that Escambia County, in FL 1 (Matt Gaetz's old district) was won by Democrat Gay Valimont even though she lost the election.

Gemini responded:

To provide the most accurate answer, I need a bit more specific information about which "recent special election" you are referring to. There can be multiple special elections held at different times.
However, a notable recent special election in Florida was for House District 35 on January 16, 2024.

Couple of things here that already stand out to me. The first is that the most recent special election was less than a week ago and Gemini used a special election from over a year ago instead. The second is that if you type "recent special elections in Florida" into regular Google search it comes up with the right elections. Did Gemini not consult the biggest search engine on the planet first? Per it's thinking process it did and used a similar query I did so not sure how it ended up being wrong:

It seems to have chosen the 3rd query for some reason despite the fact that the first query (type it into Google yourself) comes back with information from 3 days ago. At that point it's off the rails and wrong.

So I tell it:

The election to replace Matt Gaetz.

to be more specific since it was way off. Gemini responded:

Okay, thanks for clarifying. Matt Gaetz resigned from his seat in Florida's 1st Congressional District in November 2024.  
A special election was held to fill his seat on Tuesday, April 1, 2025.

* Republican Jimmy Patronis (Florida's CFO) won the election, defeating Democrat Gay Valimont.
* This means the seat remained under Republican control and did not swing Democrat.

Florida's 1st Congressional District (covering counties like Escambia, Santa Rosa, Okaloosa, and Walton) is a reliably Republican area. While Democrats invested in the race, the results confirmed the district's Republican leaning, and there were no reports of any county within the district swinging Democrat in this special election.

OK so now we're closer but it's dead wrong when it says:

and there were no reports of any county within the district swinging Democrat in this special election.

Escambia County did and that shows up in the many search results it would have had access to.

I corrected it:

My understanding is that Escambia county did swing towards the Democrats.

Gemini responded:

You are correct to point that out, and my apologies for the initial oversight.

It then went on to point out that in fact Valimont did win Escambia County which is correct but also interesting because in the thinking process it said:

Identify the core claim: Escambia County shifted significantly towards the Democratic candidate (Gay Valimont) compared to a previous baseline, even if the Democrat didn't win the county outright. "Swung towards" is different from "swung Democrat" (which implies flipping from R to D).

I don't think those two phrases are different. I can see where it might come up with that but I think if two people were talking that phrase wouldn't be questionable.

So it took what should have been a pretty easy query to get right and not only got it wrong on the first try but required two more statements, one a correction, to get to the right answer.

Just for fun I decided to ask the same question to Perplexity.ai. I gave the exact same question to Perplexity Pro in "Best" mode where it chooses the AI model to use. Unfortunately there is no way to see which one it chose but it was correct right out of the gate:

In the recent special election in Florida, Escambia County, located in the Panhandle and home to Pensacola, swung Democrat. Gay Valimont, the Democratic candidate for Florida's 1st Congressional District, won Escambia County by 3 percentage points, marking the first time a Democrat had carried the county in a federal election since 1992.

What to make of all of this? I'm not sure other than general searches that I'd probably make in Google Search Perplexity is pretty much always dead on whereas I get inconsistent results with Gemini (I have yet to test Claude in these types of questions).

At this point it seems that each model has its strengths and weaknesses regardless of how they do on the various "benchmarks". They are fun to play around with but I certainly won't be relying on them without questioning the answers.