A new study shows that giving artificial intelligence systems «internal monologue» makes their results significantly better. Essentially, AI has been trained to think before it answers queries, just as humans think about what they have to say next before they speak. This is different from how popular AI language models such as ChatGPT behave. The latter do not «think» about what they write and do not anticipate different possibilities for the next steps in the conversation.
The new method, called Quiet-STaR, instructs the AI system to generate many internal arguments in parallel before answering a query. When the AI responds to the prompts, it generates many options and outputs the best answer. Ultimately, AI learns by discarding options that turn out to be incorrect. In essence, the learning method gives AI models the ability to predict future conversations and learn from current ones.
Researchers from Stanford University and Notbad AI applied the Quiet-STaR algorithm to Mistral 7B, a large open-source language model, and published the results on arXiv. The Quiet-STaR-trained version of Mistral 7B scored 47.2% on the argumentation test, compared to 36.3% before any training. The model still failed the school’s math test, scoring 10.9%. But this is almost twice as much as the 5.9% result of the original version.
Models such as ChatGPT and Gemini do not correlate data with common sense or context, so they don’t actually understand their own answers, simply generating words. Previous attempts to improve the “thinking” ability of language models have been very specialized and could not be applied to different AI models.
The self-learning algorithm STaR, which the researchers used as a basis for their work, is one example of such learning, but it is also constrained by these limitations. The scientists who developed Quiet-STaR named the method so because STaR runs in the background. It can work with different models, regardless of the training data. Now, they want to explore how similar methods can bridge the gap between neural network-based artificial intelligence systems and human reasoning capabilities.
Source: Live Sciense