Some antivirus tests are really, really simple. For example, you could run a scan on a test system containing 100,000 static malware samples and record how many of those were detected. Testing how products handle malware that has already infected the system is quite a bit tougher, but can reveal more about an antivirus product's malware-fighting prowess. That's what the malware removal test by AV-Comparatives tries to do.
To get started, the researchers chose 11 widely prevalent samples known to be detected by every product under testing. I wish they'd used a larger sample set; 11 is pretty small. The report notes that they started with more, but eliminated some samples on finding that "their malware behavior/disinfection process was identical to samples already included."
With sample selection complete, they installed each on a test PC, carefully monitoring the changes it made to the file system and Registry. Then they installed an antivirus product and recorded how well it managed to clean up the problem.
As I've seen in my own testing, sometimes getting protection installed on an infested system can be difficult. If malware prevented initial installation, they tried installation in Safe Mode. If that failed, they resorted to a rescue CD (if available). Once they got the product installed, they ran a full scan and rebooted.
Clean and Convenient
Just how do you rate an antivirus product's success at cleaning up malware? AV-Comparatives choose to rate on thoroughness of removal, but also on convenience.
A product that cleaned up all malware traces, or all but the most negligible, earned an A for removal. If some executable files or other significant traces remained, that earned a B. Failing to correct dangerous malware-caused problems like a compromised HOSTS file or disabled Task Manager drops that grade to a C. Finally, failing to remove the malware or leaving the test system unusable rates a grade of D.
Getting the system clean is good; doing it without hassling the user is better. If the researchers managed to install the product and run a full scan without incident, that's worth an A for convenience. A cleanup that needed rebooting to Safe Mode or other manual actions got a B. If getting the system cleaned up required booting to a Rescue CD, that's a C for convenience. And of course a failed removal earns a D.
They went on to quantify these ratings for comparison purposes. A thorough and convenient cleanup would rate AA, valued at 100 points. Thorough cleanup with some manual labor would get AB, worth 90 points, and so on.
Clear Winners
Kaspersky Internet Security (2014) almost aced this test. It earned an A for removal in every case, and an A for convenience in all but one. With an average of 98 points it's at the top. Bitdefender Internet Security (2014) came very, very close—all AA ratings except for one AB and one AC, averaging out to 97 points. These two, along with five other products, earned an overall rating of ADVANCED+, the highest possible rating.
avast! Free Antivirus 2014 is a different kind of success story. In the previous removal test it rated STANDARD, the lowest passing grade. This time it joined the winners circle with an ADVANCED+ rating.
Advice for the Vendors
The full report concludes with advice for antivirus vendors, things they could do to improve the malware cleanup skills of their products. Most companies offer a bootable rescue CD for emergencies; those that don't, well, they should. The report also recommends creating an alternate installer that doesn't require access to the company website, in case access is blocked by malware, and checking for active malware at the start of the installation process.
If you've got a malware problem and need to clean it out, look for a product with a good score in this test. AV-Comparatives only runs the malware removal test once each year, which is a shame. I'd like to see them put it on a more frequent rotation, with more samples.
No comments:
Post a Comment