The study revealed a significant level of vulnerability in web coding products. With some programming languages, the result is much worse than average.
The company’s report Veracode covers more than 1600 code samples created by GPT-4, GPT-3.5, and Codex from OpenAI, which received 12 different tasks. Overall, 45% of the results contained significant vulnerabilities. These included authorization errors, input validation, SQL injection, and more.
Java was the champion in terms of vulnerabilities: 80% of the code had them. Instead, the results of JavaScript and Python were half as many «holes» — 30-40%. Interestingly, there is a simple way to significantly improve quality, namely, to explicitly point out to AI that it needs to take security into account.
The company explained the results and provided recommendations on how to improve the code. According to analysts, AI tends to create code that looks correct but does not always take security into account. In addition to direct instructions to take quality into account, coders are advised to check the result with vulnerability scanners and manually.
In particular, the company advises to run static code analysis immediately, use Veracode or other tools to fix bugs without relying on AI reliability, use open source library analyzers, and use «firewall for» packages, which blocks known dangers before installation.
Web coding is becoming wildly popular and allows to achieve outstanding results not only for programmers, but also for “shovel salesmen”who create tools for it. Some of the programmers literally already on the street. Therefore, it is worth considering not only its strengths but also its weaknesses. We recently wrote about how the web coding service deleted all working data user and hid it — thus, he hid all the bugs and vulnerabilities as well.
Source: DOU
Spelling error report
The following text will be sent to our editors: