It’s now official: the US National Institute of Standards and Technology (NIST) will unveil an industry consortium to help it run the world’s most widely used software vulnerability repository.
NIST, an agency within the US Department of Commerce, launched the US National Vulnerability Database (NVD) in 2005 and has operated it ever since.
This situation was expected to change, with vetted organizations helping the agency from as soon as the beginning of April 2024.
The NVD program manager, Tanya Brewer, made the official announcement during VulnCon, a cybersecurity conference hosted by the Forum of Incident Response and Security Teams (FIRST) and held in Raleigh, North Carolina, from March 25 to 27, 2024.
The news came after weeks of speculation over a possible shutdown of the NVD.
NIST Halted CVE Enrichment in February 2024
In early March, many security researchers noticed a significant drop in vulnerability enrichment data uploads on the NVD website that had started in mid-February.
According to its own data, NIST has analyzed only 199 Common Vulnerabilities and Exposures (CVEs) out of the 2957 it has received so far in March.
In total, over 4000 CVEs have not been analyzed since mid-February.
Since the NVD is the most comprehensive vulnerability database in the world, many companies rely on it to deploy updates and patches.
If such issues are not resolved quickly, they could significantly impact the security researcher community and organizations worldwide.
Speaking to Infosecurity, Tom Pace, CEO of firmware security provider NetRise, explained: “It means that you’re asking the entire cybersecurity community, overnight, to somehow go figure out what vulnerability is in what operating system, software package, application, firmware, or device. It’s a totally impossible, untenable task!”
Dan Lorenc, co-founder and CEO of software security provider Chainguard, called the incident a “massive issue.”
“We are now relying on industry alerts and social media to ensure we triage CVEs as quickly as possible,” he told Infosecurity.
“Scanners, analyzers, and most vulnerability tools rely on the NVD to determine what software is affected by which vulnerabilities,” Lorenc added. “If organizations cannot triage vulnerabilities effectively, it opens them up to increased risk and leaves a significant gap in their vulnerability management posture.”
To stay operational amidst the NVD backlog, several security companies, such as VulnCheck, Anchore and RiskHorizon AI, started working on projects that could provide an alternative to some parts of vulnerability disclosure traditionally provided in the NVD.
This episode coincided with the release of the latest revision of the Federal Risk and Authorization Management Program (FedRAMP Rev. 5), a US federal law requiring any company that wants to do business with the federal government to use the NVD as a source of truth and remediate all known vulnerabilities inside it.
Challenges Within the NVD Led to a “Perfect Storm”
Before the NIST statement, speculation as to what was happening included:
- Budget issues within NIST, as lawmakers recently approved a $1.46bn budget for NIST for the current fiscal year, a nearly 12% decrease from the previous year
- An ending contract with a contractor, possibly Huntington Ingalls Industries – a shipbuilding contractor that publicly works with NIST on the NVD
- Internal discussions to replace some vulnerability standards used by the NVD, such as Common Product Enumerators (CPEs) that act as fingerprints for IT products, used to clearly identify software, hardware, and systems
- Internal discussions to start adopting Package URLs (PURLs), a new standard listing universal addresses for software packages
At VulnCon, Brewer did not delve much into the reason for the NVD issue, saying, “Although there is a story behind it, it is long, convoluted and very administrivia.”
She added that a few challenges led the NVD program to “this perfect storm.”
“In May 2023, I saw that we needed to do things differently and start working differently with industry. We’ve been working on that ever since. Unfortunately, we had our perfect storm and didn’t get it done as quickly as we wanted.”
In a written statement sent to Infosecurity on March 29, a NIST spokesperson said that the backlog was "based on a variety of factors, including an increase in software and therefore vulnerabilities, as well a change in interagency support."
"Currently, we are prioritizing analysis of the most significant vulnerabilities. In addition, we are working with our agency partners to bring on more support for analyzing vulnerabilities and have reassigned additional NIST staff to this task as well, the spokesperson added.
Brewer said at VulnCon that NIST has started reallocating personnel and increasing its collaboration with other government agencies over the NVD program.
She said enrichment data should start flowing again within a few weeks.
“We’re not going to shut down the NVD; we’re in the process of fixing the current problem. And then, we’re going to make the NVD robust again and we’ll make it grow,” she insisted.
NIST Provide Details on an Upcoming NVD Consortium
On February 15, the NVD website announced that NIST “is currently working to establish a consortium to address challenges in the NVD program and develop improved tools and methods.”
This was confirmed by Brewer at VulnCon.
“Although the official paperwork is not out yet, NIST has every intention of putting together the NVD Consortium to make the NVD more relevant in the future. It should be operational within two weeks,” she explained.
The NVD Consortium will help NIST with funding and feedback on future developments.
Also present at VulnCon, J’aime Maynard, consortia agreements officer at the Technology Partnership Office (TPO), gave information about who can join the NVD Consortium and how to do it.
In summary, candidates must be organizations, sign the same Cooperative Research and Development Agreement (CRADA) with NIST and accept the same conditions and risks. A membership fee is being considered.
Entities that are prohibited from signing a CRADA may be allowed to participate in the Consortium under an alternative appropriate agreement.
Each member will have one seat on the steering committee. The Consortium will be structured into different working groups.
NIST will issue a Federal Register Notice detailing the main objectives of the NVD Consortium, how to apply and the relevant points of contact at NIST.
Meanwhile, interested parties can contact the following email address: nvd_consortium@nist.gov.
NVD’s One-to-Five Year Plan
Once the NVD is up and running, Brewer said the program will consider new approaches to improving its processes within the next one to five years, especially around software identification.
Some of the ideas include:
- Involving more partners: Being able to have outside parties submit CPE data for the CPE Dictionary in ways that scale to fit the ever-growing number of IT products
- Software identification improvements: Dealing with software identification in the NVD in a way that scales with growing complexities (the adoption of PURLS is considered)
- New types of data: Developing capabilities to publish additional kinds of data to the NVD (e.g. from EPSS, NIST Bugs Framework)
- New use cases: Developing a way to make NVD data more consumable and more customizable to targeted use cases (e.g. getting email alerts from NVD when CVEs are published)
- CVE JSON 5.0: Expanding the NVD’s capabilities to utilize new data points available in CVE JSON 5.0
- Automation: Developing a way to automate at least some CVE analysis activities
“We want to get away from needing any human analysis for CVE enrichment. Recent developments in AI could help,” Brewer insisted.
Some “Long Overdue Clarity”
Before VulnCon, many vulnerability researchers criticized NIST’s decision to keep its first public statement for the conference.
This was illustrated by Lorenc’s comment during Resilient Cyber, a LinkedIn video podcast hosted by Chris Hughes, president of Aquia. “You announce a flashy new product at a conference, you don’t update the world on what’s going on with such an important thing as the NVD,” Lorenc said.
However, Brewer’s session did answer many questions that vulnerability researchers have been asking NIST for the past month.
Speaking to Infosecurity, Aquia’s Hughes commented: “The comments provided some long overdue clarity that the industry has been asking for. The forthcoming collaborative approach should bring new support and participation and also help longstanding issues such as NVD support for PURL which helps address challenges NVD has right now around the open source ecosystem and software supply chain security.
“The consensus is that this brief disruption is going to help drive broader industry collaboration via the consortium as well as modernizing longstanding challenges with the NVD and its operations and functionality.”
Patrick Garrity, security researcher at VulnCheck, agreed.
“The presence of NIST NVD at the conference has provided reassurance to the community that NIST NVD is actively working to address the current gap in processing CVEs. While there is no definitive timeline for resolution, it's evident that they are diligently working towards a solution, also emphasizing collaboration with the community through a new consortium,” he said.
However, some voices remain critical regarding Brewer’s VulnCon speech. In a post published on March 28 on the Digital Utility Group forum of EnergyCentral’s website, Tom Alrich, the co-leader of the OWASP SBOM Forum, said he regretted that Brewer did not address the nature of the issues the NVD program has been experiencing or the reason behind the recent backlog.
Dozens of cybersecurity experts signed an open letter written by Lorenc and addressed to Congress and Gina Raimondo, the US Secretary of Commerce. It requests them to fund and protect the NVD, calling it “critical infrastructure for a large variety of cybersecurity products.”
This article was updated on April 2 with comments from NIST.