Meet CriticGPT — «teacher» ChatGPT, which will search for errors in chatbot answers

«I’m still putting a deuce in pencil!» (c)

OpenAI developed a separate CriticGPT model that will search for errors in ChatGPT — beginner «teacher» answers will focus on code fragments and, as noted, will only be an auxiliary tool for human experts who will check chatbot texts manually.

CriticGPT, based on the GPT-4 family of language models, was additionally trained on a set of code samples with deliberately inserted errors and in the first tests proved to be better than humans in 63% of cases. It allegedly wrote better and more detailed criticism, reducing the frequency of so-called hallucinations in the chatbot more often.

During training, CriticGPT successfully found both errors inserted deliberately by humans and errors added by ChatGPT initially.

OpenAI researchers have also created a new technique called Force Sampling Beam Search (FSBS), which helps CriticGPT write more detailed code reviews and can be balanced depending on the training needs of the critic model.

Interestingly, at one stage of the experiment, CriticGPT was given answers that people had previously marked as perfect — and it found errors in 24% of cases (later confirmed by reviewers). OpenAI believes that this demonstrates the model’s potential for checking non-code-related tasks and emphasizes its ability to catch «the most subtle errors» that even careful human review might miss.

Despite its promising results, CriticGPT, like all AI models, has limitations. It was trained on relatively short ChatGPT answers, so it is not yet ready for longer and more complex tasks.