Does LLM Write Performant Code? Study Says No
Saurabh Misra, founder and CEO of CodeFlash, highlighted the company’s study on AI’s ability to generate performant code at the Infobip Shift Conference in Miami last week. Increasingly, AI is used to write code, Misra said, noting that Microsoft uses AI to write 25% of its code and that 30% of Google’s code is AI-written. At 25% of startups, 95% of code is now written completely by AI, he claimed.
It’s one thing to write code, but another thing to write performant code. The Python code optimization company wondered if the code from large language models (LLMs) is performant.
“What we found was shocking,” Misra said. “What we found was that LLMs actually struggled to write performant code.”
The company optimized 100,000 open source functions and found that 62% of the time, the new code that AI generated included code that, if accepted, would result in bugs.
Image via CodeFlash’s blog.
“Where the code was correct, we found that 28% of the time the code was actually not performing — so it was slower or had the same performance as before — and only 10% of the time the code was actually faster,” he said at the conference. “What this means is [that] 90% of the time LLMs failed to optimize the performance of the code.”
Their research also found that 90% of the optimizations suggested by leading LLMs are incorrect, he reported in a blog post. The post added that industry data from Harness.io supports CodeFlash’s finding. According to Harness, 52% of engineering leaders and 54% of developers reported that increased AI usage directly leads to performance problems.
“Writing performance code is just a hard problem,” Misra said. “You have to really understand what’s happening at a very deep level, understand the options available, what algorithms [and] what libraries might be available, and it requires a lot of deep insight.”
An AI Model Accessibility Checker
Thursday was the 14th annual Global Accessibility Awareness Day (GAAD), and to honor the occasion, the GAAD Foundation and workflow automation company ServiceNow launched an AI Model Accessibility Checker.
The accessibility checker evaluates and compares how well coding-focused large language models can generate accessible code. It provides a benchmark for companies to test and demonstrate the accessibility of their models’ output, according to the GAAD press release.
“By encouraging accountability and improvement in AI-generated code, the project is helping the technology industry drive toward more inclusive standards that can lead to a more accessible world for the 1.3 billion people living with disabilities,” GAAD stated.
The checker is open source and will be available on May 22. For more on web and frontend accessibility, check out the following articles:
- 3 Lessons in Accessible Development From an Expert Tester
- A Workflow for Component-Based Accessibility Testing
- HTML Markup Tips for Developing Accessible Websites
New Angular Site Incorporates AI-Driven Documentation Search
Unsure if the version of Angular you’re using supports a particular feature? Trying to decide if the upgrade to the next version is worth the hassle?
Frontend software engineer Gérôme Grignon has developed a site that can help. Angular Can I Use checks feature compatibility across versions for you. It documents what’s experimental, what’s in developer preview, what’s stable, and everything deprecated or removed in Angular versions 13-20.
But what may be even more valuable is its AI-powered chat, which allows developers to query the Angular documentation.
The page is available via Angular Courses Lab, which the About page explains is an “educational project created to provide alternative ways to learn and retain Angular knowledge.” The plan is for data visualization, AI chat, quizzes and interactive tutorials to be incorporated into the site.
There’s also a blog that promises a weekly deep dive into complex Angular topics.
Microsoft Advocates for React Native
Microsoft recently shared how its Office team used React Native to improve app reliability and upgrade their development experience.
This isn’t the first time Microsoft has encouraged developers to use the Meta-created framework, according to DevClass.com. It also noted that this is despite Microsoft’s own investment in C# or C++.
“Windows developers have often observed though that the Office team within Microsoft does not embrace the desktop frameworks that the company creates for others,” the article noted. “Office never used .NET Windows Forms or WPF (Windows Presentation Foundation), for example.”
Microsoft does offer a cross-platform app UI framework called .NET MAUI but Office is mostly built with C++ and React Native, the article added.
“React Native can be embedded into existing Windows applications, which allows apps to choose which experiences to migrate onto the platform,” Microsoft software engineer Chiara Mooney wrote on the company’s React Native blog — yes they have a whole blog on the topic, which is telling, really. “This feature is critical for Office’s modernization efforts.”
She explained that React Native content islands can integrate into an app’s UI, giving a consistent look and feel across an application. The islands can interact with one another and pass app data between platforms, she added.
The Top-Ranked AI for Web Development
Google recently proclaimed its Gemini 2.5 Pro is the top AI for frontend web development. It bases this on its 1420 score on the WebDev Arena leaderboard. It bested Claude 3.7 Sonnet’s ranking of 1357.
The WebDev Arena leaderboard measures human preference for a model’s ability to build aesthetically pleasing and functional web apps, according to Google’s announcement written by Logan Kilpatrick, the senior product manager for Gemini API and Google AI Studio.
“Implementing new features means manually diving into design files and inspecting components to match style properties like colors, fonts, padding, margins, and borders then manually writing the CSS code needed to replicate those visual properties accurately,” Kilpatrick wrote. “Now imagine using Gemini 2.5 Pro in an IDE and having the model generate new features, like adding a video player in the style of the other apps in the Gemini 95 starter app.”
The New Stack wrote about Gemini Pro’s latest evolution in more detail earlier this month, focusing on what this means more broadly for developers.
