Anthropic Company introduced a new artificial intelligence tool, Computer Use, along with updated versions of its language models — Claude 3.5 Sonnet and Claude 3.5 Haiku. Computer Use can control the user’s computer mouse and perform basic tasks on the computer.
The updated version of Claude 3.5 Sonnet demonstrates improvements in all areas, especially in programming, where the model was already an industry leader. The Claude 3.5 Haiku achieves the performance of the Claude 3 Opus — the company’s previous most powerful model in many respects, while maintaining the same price and speed.
With Computer Use, the system executes multistep instructions for working with a computer: viewing the screen, moving the cursor, pressing buttons, and entering text. At this stage, the tool is available only through the API and works with the Claude 3.5 Sonnet model.
To successfully control the cursor, Computer Use uses a special mechanism: the system takes consecutive screenshots and calculates the number of pixels to move the cursor accurately. It is the ability to count pixels correctly that has become the key to executing commands.
However, the current version of the tool has certain technical limitations. Because it works with individual screenshots rather than a video stream, the system may miss short-term notifications. Also, Computer Use has not yet mastered some common actions, such as dragging and dropping objects.
Before the public launch, Anthropic engaged several powerful partners in testing. Among the companies that have tested Computer Use are Amazon, Canva, Asana, and Notion. Their experience helped to improve the system before open beta testing.
There is fierce competition in the development of such tools. Anthropic’s main rival is OpenAI — is also working on a similar technology, but keeps it in the public domain for now. Analysts predict that such tools will generate significant profits in the coming years.
Given the potential risks, Anthropic has taken care of protection against abuse. The company has implemented systems to detect suspicious activity and set restrictions: Computer Use can’t interact with social networks, register domains, or work with government websites. Developers can join public beta testing on the official Anthropic website.
Microsoft is also not far behind Anthropic, which recently introduced a large-scale update of its Copilot artificial intelligence, adding voice communication and image recognition capabilities.
Source: Arstechnika, Anthropic