Can Your Antivirus Handle a Zero-Day Malware Attack?

Testing signature-based antivirus protection is a snap. You gather hundreds or thousands of known malware samples, run a scan, and note how many your antivirus product detected. However, for a brand-new, zero-day virus (or other type of malware) there's necessarily no signature available. Testing protection against zero-day threats is tough, but the researchers at AV-Comparatives have worked out a technique that satisfies them. Note, though, that not all antivirus vendors approve of this particular test; quite a few opted out of the latest edition, the results of which have just been released. By definition, it's not possible to run a test using actual zero-day samples. By the time the researchers could capture and validate a sample, the antivirus vendors would already be on the way to preparing a signature. AV-Comparatives simulates zero-day detection by "freezing" a product's signature database and then using only samples that first appeared after the big freeze.
Some products will detect new malware using heuristic techniques, identifying them by similarity to known malware or by other characteristics. The researchers launched each sample not caught by heuristics, noting whether the product's behavior-based detection or other real-time protection prevented infestation. Products earned full credit for blocking the malware all by itself and half-credit in situations where blocking required a correct decision by the user.
Very Good Detection
Based solely on their detection rates, 11 of the 16 tested products would have earned an ADVANCED+ rating, the top rating. Bitdefender topped this group, with 97 percent detection; Kaspersky and Emsisoft both managed 94 percent. Panda and Avast would have earned ADVANCED. Microsoft would also have gotten an ADVANCED rating, but AV-Comparatives uses it only as a baseline. At the bottom, AnhLab and Vipre would have passed with a STANDARD rating.
Pesky False Positives
Heuristic and behavior-based detection systems have to be very carefully tuned to avoid flagging valid programs as dangerous—that's what we call a false positive. Quite a few of the tested products lost points for too many false positives. Since the detection test was performed using signatures frozen last February, the researchers were able to re-use the false positive results from a test performed in March.
Six of the tested products lost one rating level due to too many false positives. For Emsisoft, eScan, and G Data, that meant dropping from ADVANCED+ to ADVANCED, while Panda dropped from ADVANCED to STANDARD. As for AhnLab and Vipre, they were both already at the lowest passing level, so their final rating became merely TESTED; they did not pass.
Cloud Controversy
Vendors who submit their products for testing by AV-Comparatives must agree to participate in all of the required tests. The signature-based file detection test is one of the required set; Symantec doesn't approve of that test, which is why you won't find results for Norton in AV-Comparatives reports.
The proactive test, on the other hand, is optional. According to the report, "AVG, McAfee, Qihoo, Sophos, and Trend Micro decided not to take part, as their products rely heavily on the cloud." The zero-day test necessarily excludes cloud-based detection, as there's no way to "freeze" the cloud. These vendors felt their products would score poorly without access to a cloud connection.
While AV-Comparatives did allow these vendors to bow out, the report scolds them just a bit. "Even several weeks later, a number of the malware samples used were still not detected by some cloud-dependent products, even when their cloud-based features were available," it states. "We consider it a marketing excuse if retrospective tests... are criticized for not being allowed to use cloud resources." The report concludes, "If a file is completely new/unknown, the cloud will usually not be able to determine if it is good or malicious."
If your antivirus earned a top rating in this test, that's a good sign that it will defend against brand-new zero-day threats. But since the test doesn't literally use real-world never-before-seen samples, a poor score (or no participation) doesn't necessarily prove it won't do the job. For a full understanding, you'll want to look at a wide variety of tests, and at PCMag's in-depth hands-on antivirus reviews.