Are CVE counts a good indicator of a software's security?

Can we say that a software is "more secure" than another, based on the number of CVEs these softwares have?

No. CVE entries are not a good source to rank products by their "overall security".

The main idea behind the CVE system is to create unique identifiers for software vulnerabilities. It's not designed to be a complete and verified database of all known vulnerabilities in any product. That is, a vendor or researcher could simply decide to not request a CVE number for a given flaw. Further, entries sometimes combine related bugs under a single ID or don't disclose the exact impact, making a simple "bug count" a rather meaningless security criterion. Also, for a ranking you'd have to find sensible metrics to compare different severities. (How many DoS bugs equal a remote code execution...?)

That said, CVEs do surely give you an idea about what kind of vulnerabilities have been found in a product and they're a good starting point for research. But the amount strongly depends on the age of the software and how much attention it receives through security auditing. You can't really reason if a lot of CVE assignments means that the software is poorly written or if it actually means that it's particularly secure because evidently a lot of vulnerabilities are getting fixed. I personally tend to find it suspicious if an older product has a very short record of patched vulnerabilities because that could indicate it hasn't been audited thoroughly.

So you should think of CVE as a dictionary rather than a database that simply assigns handles to vulnerabilities so that you can reference them easier — don't use it as a tool to compare security.

Here are some better indicators for a secure product:

  • The software is used and developed actively.
  • The vendor encourages people to search for vulnerabilities (and maybe even offers bounties).
  • New security bugs are processed and patched quickly.

As a voluntary system, CVE is as open to abuse as the software products it tracks and is often highly subjective. This is somewhat compounded by the scoring mechanism that's used to track severity which is typically CVSSv2.

In an ideal world, when a vulnerability is discovered in a product the developers will register a CVE, later publishing it along with whatever fix they produced for their product.

However, as others have pointed out, sometimes CVE's just aren't created or developers will take a bunch of vulnerabilities, cut one release that fixes them all and create on accompanying CVE.

If I'm interested in using a certain software product and had reservations around it's security, I'm more likely to take a look at how they handle security issues, receive reports etc than to look specifically at their CVE database.

One exception to the above is software products that use other components, in these cases, taking a look at how long it took a product to catch up with the CVE's that were issued against it's constituent products/services can be enlightening. Commercial software that re-packages open-source components can often lag many months behind on security fixes. This is probably the most useful thing I make use of the CVE database for.


CVE only represent the bugs on applications which people are actively trying to exploit. While open source may or may not be subject to a higher or lower volume of CVEs, it could be assumed reasonably that it does not imply any application is more or less prone to exploitation. Consider many exploits/vulnerabilities are often sold as 0day before even being considered as a CVE entry. Also consider that many people in the open source community tend more towards reasonable disclosure than towards profit. In any case I would measure the vulnerability of two applications based on a couple of metrics:

  1. How severe are the impacts exploits of the past?
  2. Volume of CVEs: As you implied, more CVEs probably implies that the application is audited more frequently by security research.

  3. Open source: I personally consider open source 'generally' more secure. View the source code for yourself, and look at the coding style. Maybe you can find a vulnerability yourself.

Tags:

Cve