After months of fine-tuning, Snopes has launched FactBot, its first-ever artificial intelligence (AI) tool to fact-check your burning questions about online rumors.
Type a phrase such as " Is Bigfoot real? " or " Is the house in Home Alone a real house in Illinois? " and the bot will search Snopes archives, give you an answer and let you know which articles it found the information in.
As many of you know from the recent popularity of ChatGPT , AI models today often hallucinate wrong information. The wrong answers often sound believable, so it can be difficult to determine what is real. Another wrinkle: AI models are trained in the past, so even if an answer was correct at the time, it may be outdated now. Neither is good for fact-checking.
What Is FactBot?
Enter FactBot. Working with programmers from California Polytechnic State University (Cal Poly) and Amazon Web Services (AWS)— a partnership through the Cal Poly Digital Transformation Hub — we set out to find a solution.
We were able to leverage AI but also search through the 30 years of Snopes articles in real time to get up-to-date information while also citing where the answer came from. As a rule, when working with AI, you should always check your sources before relying on an answer.
The AI tool (dubbed Snopes FactBot ) is available to use as of this writing, and you can find it here . It costs nothing and it is currently open to anyone. We do, however, restrict you to a few queries per day unless you create a free account.
Those free accounts support our journalism, by allowing us to serve more relevant and valuable advertisements to users who provide us with an email. You can become a member for free here .
The search page on the Snopes website receives tens of thousands of queries daily, and AI models take a lot of processing power. If FactBot becomes super popular, we may add a limit for free accounts, too, until the cost comes down (but we'll cross that bridge if we get there too quickly).
How Does FactBot Technically Work (Nerd Alert)?
The teams tried numerous different methods before arriving at a final product. In the end, the AI model that met all of the goals was Anthropic's Sonnet 3.5 model (released June 25, 2024 ). Before that release, we started to lose hope this could work, as all other open models fell short — most were painfully slow and yielded more inaccuracies than correct answers.
We used a technique called RAG in order to leverage the Snopes archive for the AI model. We ran every Snopes article through a titan-embed-text-v2 model to generate vectors and store them in OpenSearch. When a question is entered, we run it through the same titan model and run a similarity search to quickly find relevant articles. The responses are then run through a z-score reduction before being sent to the AI model.
This method was agreed upon by Snopes and the Cal Poly team as it is far more energy- and cost-sustainable than constantly retraining a Large Language Model each time we publish – five times per day or more for Snopes. It also allows us the flexibility of updating the Large Language Model to a newer version of Anthropic's Sonnet, say, when that becomes available.
Everything is run on AWS services. Those services run on EC2 servers behind nginx — it's a web socket endpoint that is written in Golang in order to deliver the text without waiting for it to be full typed out. The AI model is queried through AWS Bedrock, and as previously mentioned, the articles are then converted to vector format and stored in OpenSearch.
FactBot Feeds Our Newsroom
Snopes' dedicated team of fact-checkers are always scouring the internet for signs of what needs their attention pronto. Monitoring internet and social media trends will continue, but the chatbot represents an improvement to Snopes' current contact flow. Instead of only monitoring those sources and an inbox of emails from users with story ideas, links and questions, staff will also hear back from the chatbot on what the most frequent topics of conversation are, offering a new story-idea pipeline.
Most Popular
Comments / 0