Google ‘Retrofits’ Spatial Memory Safety Onto C++
After analyzing nearly 10 years of CVEs, from July 15, 2014 to Dec. 14, 2023, Google researchers calculated that at least 40% of safety exploits in C++ were related to spatial memory exploits (like writing to an out-of-bounds memory location).
But Google is trying to do something about it — while setting an example for a world filled with unsafe legacy code.
Google researchers showed they were able to “retrofit” spatial safety onto their C++ codebases, and to do it with a surprisingly low impact on performance. A blog post about their results kickstarted a wider conversation with some interesting follow-ups online, some even wondering aloud whether this low impact means that C++ code could include spatial memory safety by default.
It’s also raised the possibility that long-standing assumptions can be challenged over time — and that when faced with a serious problem, there’s always the possibility that new technologies will help.
Retrofitting Secure-by-Design
It all started with a blog post describing how Google protected its server-side production systems by adding bounds checking to its C++ code — improving security on Gmail, YouTube, Google Maps and even the Google search engine. Google was already an early adopter of memory-safe languages and other safe coding practices, but a full transition would take years, meaning they’d also need an additional step: “retrofitting secure-by-design principles to our existing C++ codebase wherever possible.”
Their blog post was written by senior staff software engineers Alex Rebert and Kinuko Yasuda (working with Google’s Max Shavrick, also part of their Security Foundations team). And it starts by noting that LLVM’s implementation of the C++ standard library includes several hardening modes to spot undefined behaviors, making it possible to do bounds checking on C++ code.
“[We’ve] now made it default across our server-side production systems,” the blog post explains — while closely monitoring the rollout.
The results?
- The team found over 1,000 bugs. (Google estimates that on average this would scale into finding between 1,000 and 2,000 bugs every year.)
- Google also experienced “a 30% reduction in our baseline segmentation fault rate across production.” Their blog post attributes this to better code reliability and quality: “Beyond crashes, the checks also caught errors that would have otherwise manifested as unpredictable behavior or data corruption.”
- It ultimately also helped Google “identify and fix multiple bugs that had been lurking in our code for more than a decade.” Hardened C++ checks “transform many difficult-to-diagnose memory corruptions into immediate and easily debuggable errors.”
- It even disrupted an internal red team exercise, “demonstrating its effectiveness in thwarting exploits.”
‘I Was Wrong’
But one specific result drew the most attention. Google’s blog post reported that hardening libc++ “resulted in an average 0.30% performance impact across our services.” (They even wrote, in emphasis: “yes, only a third of a percent…”)
Bounds-checking in C++: so people ask if the .3% overhead is real. It’s not just a benchmark result, we got this through our Google-Wide profiling, that gives us the live insights from DCs. This surprised us too as it was much cheaper than we thoughthttps://t.co/zBUvoYzGi1 https://t.co/7NAWBuxdtP
— Kinuko Yasuda (@kinu) November 16, 2024
Their post got a reaction from Chandler Carruth, a Google distinguished engineer who is also founder and co-lead of the new Carbon programming language. Carruth wrote his own blog post where the very first section was headlined “Overhead of bounds checking: I was wrong.”
“A number of historical reports by others of the costs coupled with some fairly casual experimentation on my own instilled a pretty strong belief that bounds checks couldn’t realistically be made cheap enough to enable by default. However, so far they are looking very affordable.”
Unfortunately, this widespread belief had kept high-quality dynamic safety checks out of libc++ (and other C++ libraries), and initially out of LLVM.
The Road to Hardened C
But Carruth saw compiler-based checks going into Microsoft Visual C++, while “all the folks at Apple driving the LLVM RFC for safe buffers in C++ did an amazing job here of getting the LLVM ecosystem, both Clang and libc++, to have solid tools in this space at last.” And Carruth thinks another factor was memory-safe language developers contributing more to LLVM (as more non-C/C++ languages began using it), bringing “a steady and systematic series of improvements.”
Looking back Carruth thinks there were “many years” of improvements in LLVM, and “I think we didn’t really notice when a tipping point was reached. At that point, they combined to radically reduce the practical costs of these kinds of checks and make them affordable by default and pervasively.
“This level of availability changes the security game, as we no longer have to make painful tradeoffs of performance or security, we can have both.”
Carruth writes (in bold letters) that with effort, and with compilers continuing their support for memory-safety checks, “I think we have a real chance at making spatial memory safety achievable by default, even in C and C++, and even in the most performance constrained environments.”
And this raises the question of whether other safety checks like reference counting should also now at least be considered — even ones we’d assumed also would come with prohibitive performance impacts.
“I think there is some compelling evidence that for smaller systems (phones, laptops, things attached to batteries), the cost in cache traffic and potentially synchronization overhead is marginal at best. Swift I think has shown strong evidence that with some investment in optimization infrastructure and careful implementation, reference counting can be extremely efficient on these processors.”
And of course, Google’s post triggered more discussions on LinkedIn:
With or Without Optimizations?
Carruth also looked back at efforts toward “making LLVM better at optimized hardening.” Here he suggests performance-sensitive workloads benefited from profile-guided optimization (or PGO), and from “the folks systematically building out optimization infrastructure … isolating the hot paths of the code from the overhead of safety checks, and unlocking all the other optimization techniques LLVM was growing around them to minimize their cost.”
When it comes to Google, their blog post does acknowledge that and other performance-improving tricks — before adding that “even without those advanced techniques, the overhead of bounds checking remains minimal.” (Google’s blog post instead attributes the low impact to the fact that hardened libc++ is designed to be efficient — and the way that the compiler can eliminate redundant checks during optimization.)
Although when they found an unnecessary check being performed by LLVM — which was significantly impacting performance — they coded up a fix and contributed it to the LLVM project “to share the benefits with the broader C++ community.”
Reactions
Google’s post attracted some positive responses around the web. On Hacker News, it drew a reaction from Ben Titzer, the co-founder of WebAssembly, who remembered arguing 20 years ago about the need for bounds checking in C++. “Programs have bugs that bounds checking catches. And making it a language built-in exposes it to compiler optimizations specifically targeting bounds checks, eliminating many and bringing the dynamic cost down immensely.
“Just turning them on in libraries doesn’t necessarily expose all the compiler optimizations, but it’s a start. Safety checks should really be built into the language.”
Another response came from Walter Bright, who created the alternate Zortech C++ compiler. Bright is also the creator of the system programming language D, and he described what happened after D added array bounds checking 20 years ago: “It’s a huge win.” And in another comment, Bright remembered making bounds-checking the default option in D. “In order to get array bounds turned off, you had to throw a switch and it only happened for code marked @system.
“This turned out to be the right move.”
But maybe the most positive response call was posted on LinkedIn by Shane Miller (a distinguished advisor for the Rust Foundation). “It’s lovely that you didn’t stop at capturing your security wins with this initiative.
“The data y’all have on identifying bugs and improving reliability is a wonderful reference.”
