Презентація GPT-o1 / OpenAI
ChatGPT with language model o1 spends “more time thinking” for a better response. Users have found that “thinking” sometimes occurs in Chinese, regardless of the query language.
It seems the algorithm “thinks” not only in English, but also in other languages, including Chinese. Most people only pay attention to the final response, but users have the opportunity to see how ChatGPT forms and justifies it. At this point, many noticed that the LLM started including Mandarin and Cantonese in the process. The model also uses other languages.
“Why did o1 pro randomly start thinking in Chinese? None of the conversation parts (5+ messages) were in Chinese… very interesting… influence of training data,” — asked Rishab Jain on X. Other users with similar questions tagged OpenAI and ChatGPT in their posts, but no answers were given.
The simplest thing that might come to mind is the use of Chinese data for training. Industry representatives, including Elon Musk, acknowledge the deficit of data for training models. But a more complex explanation lies in the principle of how language models work, which are rightly called so. Perhaps the essence is that other languages work differently, and may turn out to be more effective in forming a response. The model discovers that in some situations, Chinese or other constructions will work better, and the user is given the final answer in the language of the query.
The real reason is difficult to determine given the opacity of language models, and it’s not just about corporate restrictions. OpenAI once admitted that it actually does not know how the GPT model works. Indeed, developers are only aware of the technical side, and to a lesser extent how the model “thinks.” Currently, OpenAI has not provided answers to user questions and journalist inquiries. It is possible that the company simply does not know them.
Source: Gizmodo