American researchers from the University of Michigan have developed a new text recognition test, generated by AI and the one created by man.
Recognizing AI-generated content from human-generated content is not an easy task. There are no so many tools, that can effectively distinguish between the two, generated by AI from the human-made and avoid false accusations.
The new test by American researchers may be especially useful for scientists and students, who are increasingly faced with the fact that the works they create are perceived as generated by artificial intelligence. The developers have named their tool «Liketropy», as the theoretical basis of the method includes the statistical ideas of likelihood and entropy.
American researchers have created statistical tests without the need for additional training examples for AI that can detect whether a text was written by a human or generated by artificial intelligence. This tool is focused on Big Language Models and uses statistical properties of the text itself, such as the degree of surprise or predictability of the words used.
The developers claim that their test has demonstrated high performance on large datasets, including those whose models have been hidden from the public. The effectiveness of this test in text detection, generated by the LLM, reached more than 96%, and the error was only 1%.
«We deliberately did not create a detector that simply points. AI detectors can be overconfident, and that’s risky, especially in education and politics. Our goal was to be cautious of false accusations, while labeling content created by artificial intelligence with a statistical significance of», — explains the study’s co-author, Tara Radvand is a doctoral student at the Ross School of Business at the University of Michigan.
At the same time, the researchers unexpectedly learned that they needed very little information about a particular LLM to determine the text it generated. The researchers wanted to be fair, especially to international students whose first language is not english. Recent studies indicate that the work of students whose english is a second language after their mother tongue is often unfairly labeled as AI-generated, due to similar sentence structure.
The researchers plan to expand the trial version of their test and adapt it to different fields. They found that industries such as legal and scientific, as well as areas such as college admissions, have different thresholds in the ratio «precautionary effectiveness».
Another important area for identifying AI-generated content is to limit the spread of disinformation on social media. Some tools deliberately encourage LLMs to adopt extreme and radical beliefs and spread disinformation on social media to manipulate public opinion.
Since such AI systems are capable of generating large amounts of fake content, the researchers say it is crucial to develop tools that detect and flag it. Early detection helps platforms limit the spread of malicious material.
The results of the study are published on the preprint server arXiv
Source: TechXplore
Spelling error report
The following text will be sent to our editors: