GitHub has announced a number of updates for Copilot, as well as a glimpse into the «agent-based future of its AI-powered programming assistant.
Among the notable updates is a feature called Vision for Copilot, which allows users to add a screenshot, photo, or diagram to a chat, and Copilot generates an interface, code, and alternative text to bring it to life.
So, for example, someone from the marketing team might take a screenshot of a web page and illustrate some changes they want to make to that page. Instead of requesting such changes via text prompts, it is now possible to upload an image and simply ask Copilot to implement the changes as specified in the file.
It’s worth noting that this has been available as an extension in VS Code since October, and Microsoft confirmed in the program description that it will eventually be removed in favor of its own feature on GitHub, Copilot Chat. Previously, it also required the user to have their own API key.
GitHub is also introducing a new feature called «next edit suggestions» as part of Copilot’s broader functionality for autocomplete code. Until now, GitHub Copilot in VS Code worked from the cursor position, but now it will look at other recent edits to predict what you might want to do next. Essentially, it uses richer contextual clues to automatically identify and suggest what the next edit should be. If the developer wants to accept the suggestions, they can press the Tab key, or Esc to reject.
Last November, GitHub debuted a new feature called Copilot Edits, which is a new way to make multi-file edits using natural language prompts. This feature is now moving to general availability and also includes a new «agent mode» that identifies all files relevant to the changes a developer is trying to make, instead of relying on them to manually select the files to which the changes should be applied.
«Copilot [now] does more work to figure out your intent with your initial request and then tries to resolve it,» said GitHub CEO Thomas Doomke.
In the long term, the goal is to have such «agent modes» applied to other aspects of Copilot, allowing for greater automation across a wider range of interconnected tasks. That’s why GitHub also announced a new initiative called Project Padawan. It’s essentially a SWE (software engineering) agent that can handle entire tasks on its own under the guidance of the developer who assigns the Copilot task.
Doomke did not give any indication as to when these updates might be ready for widespread use, only that work is underway with partners and the community to improve them.
«We are looking for partners who are also creating agents to integrate into this flow and customers to work with us and give us feedback, because we know that at this stage it will not be perfect,» said Doomke.
Source: TechCrunch