Claude AI's Novel Web Search Evaluation in Relation to Gemini and ChatGPT
Let's Chat About AI Web Searches, Shall We?
Artificial intelligence is stepping up its game, helping you navigate web searches as well as managing your tasks. The latest AI assistant on the block is none other than Claude, a chatty AI that can browse the web for you, whether you're after the latest news headlines or discounted gadgets.
"With web search, Claude gets its information from the latest events and news, making it more accurate on tasks benefiting from the most up-to-date details." So says its developer, Anthropic. The feature is available now for all paid subscribers, and will soon be coming to those on the free plan.
Web search is a whole new ball game for AI models, testing their abilities to sift through and evaluate web content, not just information embedded in their training data. Here's how Claude stacks up against Google's Gemini and fellow AI, ChatGPT.
Claude's Web Adventures
To give Claude web access, simply toggle the Web search option on when using the AI. It'll then scour the web for answers as needed, but if you want to make sure it uses online info in its response, include "search the web" in your prompt. When a web search is initiated, Claude will let you know and usually takes a little longer to come back with an answer.
You can run just about any query you'd type into Google, covering everything from weather forecasts and sports scores, to deep dives into music history or troubleshooting computer problems. As usual, you can follow up with more questions about the search results Claude provides.
It's safe to say AI could drastically shake up the way we surf the web, by offering a more natural, conversational experience than the traditional list of links on Google. But don't forget the issues: Can we really trust these AI bots, and will real people have no incentive to publish on the web when the bots can do the work for them?
Breaking the News
I tasked Claude to bring me the latest tech news headlines, and while it did a decent job, the citation links often took me to front pages of news sites rather than individual articles. Gemini was on par with Claude, managing to link to specific articles, not just news hubs.
ChatGPT wasn't as impressive when it came to discovering news I cared about from the most reputable tech news sources. While it still did well, I prefer the results I got from Claude and Gemini. In fact, when asked to fetch the latest news from Lifehacker, neither Claude nor ChatGPT could deliver, while Gemini actually provided the latest stories complete with links.
Checking the Facts
I tested Claude with a movie question it should have known the answer to: How many Oscars were won by One Flew Over the Cuckoo's Nest? It got the right answer and the right year, and gave the context that it's only the third film in history to win the big five Academy Awards (Best director, Best actor, Best actress, Best picture, and Best adapted screenplay).
Gemini also got the answers and the context of the big five win, with its sources covering a wider variety of sites and even YouTube. It gave a shorter, more concise response than Claude, and didn't include background information on box office takings.
As for ChatGPT, it provided an accurate answer with the useful context about the big five success and the other movies that have also achieved this feat. But unlike Claude and Gemini, it included a YouTube video of the best picture Oscar presentation.
These kinds of web searches aren't particularly challenging. More complex questions may pose more problems, especially if the answers aren't readily available and the AI tries to make them up. I did try to trick these AI bots into thinking Daniel Day-Lewis won two Best Actor Oscars in consecutive years, but all three were quick to identify that this never happened.
Going Shopping
Online shopping could be transformed by AI. While real humans will always outshine bots when it comes to picking out the perfect presents, AI can quickly scan and summarize the opinions of real shoppers, providing a user-friendly shopping experience while taking a cut of the sales.
I asked Claude to recommend a fun gift for me, based around sports or movies, and it dutifully looked up pages listing fun sports and movie gifts. While it did a decent job, I believe this is one query I could have easily handled myself on Google without involving AI.
Gemini offered a more personalized, chattier response and provided no web links, perhaps because Google wants users to stick to its main search engine for these types of queries. ChatGPT proved the most useful in this area, thanks to recent shopping upgrades, listing top picks with pricing and links to buy them on the web.
Web search is still very much a work in progress for all these AI tools. Sometimes it works better than a traditional Google search, but not always, and trusting these confident, polished answers without checking their original sources can be tricky.
Disclosure: Lifehacker's parent company, Ziff Davis, filed a lawsuit against OpenAI in April, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.
[1]
"Claude, equipped with web search, utilizes the latest technology to sift through up-to-date web content, offering a more conversational web search experience."
"In tech news headline searches, Claude displayed mixed results, providing front page links instead of individual articles, while Gemini and ChatGPT showed a more refined approach."
