One of the biggest drawbacks of AI-based chatbots is the so-called «hallucinations», when AI makes up invalid information, i.e., actually lies. Some experts say that this is one of the interesting features of AI, and it can be useful for generative models that create images and videos — but not for language models that answer questions from users who expect accurate data.
Google’s DeepMind lab and Stanford University seem to have found a workaround to the problem. The researchers have developed a verification system for large language models of artificial intelligence: The Search-Augmented Factuality Evaluator, or SAFE, checks long answers created by AI chatbots. Their research is available as a preprint on arXiv, along with all the experimental code and datasets.
The system analyzes, processes, and evaluates answers in four steps to check their accuracy and relevance. First, SAFE breaks down the answer into individual facts, reviews them, and compares them to Google search results. The system also checks the relevance of the individual facts to the query.
To evaluate SAFE’s performance, the researchers created LongFact, a dataset of approximately 16,000 facts. Then they tested the system on 13 large language models from four different families (Claude, Gemini, GPT, PaLM-2). In 72% of the cases, SAFE produced the same results as the human review. In cases of disagreement with AI results, SAFE was right in 76% of cases.
The researchers claim that using SAFE is 20 times cheaper than human verification. Thus, the solution proved to be economically viable and scalable. Existing approaches to assessing the relevance of model-generated content typically rely on direct human review. Despite its value, this process is limited by the subjectivity and variability of human judgment and the scalability issues of applying human labor to large datasets.
Source: Marktechpost