Are Malware Test Files Useful?

Several of the major anti-malware developers frequently participate in independent tests conducted by Virus Bulletin. Numerous companies have received VB 100 certification, including Sophos and Symantec. Since many products on the market today are effective, some have posed the question of how useful malware test files really are. We have discovered some information that will shed light on the subject and hopefully answer this question

False Positives

Earlier this year, each of the three anti-malware products submitted by Trend Micro into Virus Bulletin's independent test produced failed results because of false positives. A total of 20 products were submitted, with six generating false positives when scanning a batch of known clean files. Aside from Trend Micro, other products that failed to meet requirements for VB 100 certification were FortiClient, Ikarus Utilities and VirusBuster.

As one of the big four anti-malware developers, Trend Micro's products falsely identified a Microsoft development tool as a piece of spyware. The test files were the first conducted by Virus Bulletin on the 64-bit Windows Vista system. The malware test files included were known to be clean and chosen from the "most popular" lists on various free download sites.

The Detriment of False Positive Alerts

The fact that some of today's best software resulted in such an upsurge of false positives raises huge concerns. False detections can actually stir up as much chaos as genuine malware, causing end users to panic and needlessly delete legitimate files under the assumption they are being attacked. The results of such a situation can be very damaging. According to John Hawes, senior consultant at Virus Bulletin, most products that failed with the malware test files showed a significant reliance on heuristic detection techniques. He goes on to say that anti-malware developers have a long way to go if they expect to minimize this alarming number of false detections.

In February of 2007, Microsoft received criticism from all angles after their OneCare conumser AV software failed to meet the requirements for VB 100 certification when tested on the 32-bit Windows Vista platform. In the most recent test, ForeFront, their enterprise product, gave a strong performance and was awarded VB 100 certification on the 64-bit Vista platform.

Requirements for meeting VB 100 status calls for anti-virus products to scan test files of numerous viruses from the "in the wild" list, strains that are known to be circulating on systems around the world. In order to earn VB 100 certification, these products must be able to detect 100% of the malware categorized in the wild, all without generating any false positives when scanning a set of clean files.


Are malware test files useful? Companies like Trend Micro and Microsoft would respond with a resounding yes. An outbreak of false positives resulting from different programs would be an absolute nightmare for the industry. Unfortunately, three more anti-virus developers recently failed to meet VB 100 requirements for malware detection on Windows XP. This is clear evidence that the struggle continues.

Log in or sign up to comment.

Post a comment

Log in or sign up to comment.

Fraud causes hundreds of millions of dollars in damage each year and affects just as many people.

Credit card fraud is the most common type of fraud to occur each year and cost its victims up to $500 million dollars in damages each year. Despite the frequent occurrence of this type of fraud, millions of credit card users are still unaware of how to protect themselves against this type of thievery.

No one is completely safe from being defrauded. But, by learning how to protect against fraud, you will be better equipped to prevent yourself from falling into a scam that could cost you everything. Taking the time to protect yourself can help to keep you safe.