Virus Bulletin is widely regarded as "The Bible of The Antivirus Industry", and its VB100% Award is the award every antivirus vendor strives to win, but winning it isn't easy --- Virus Bulletin's test team are full-time professionals who vigorously test antivirus software against thousands of real viruses.

NOD32 has not missed a single "in the wild" virus in a Virus Bulletin test in the past four years, and it has several "clean sweeps" of every virus in every category to its credit --- and NOD32 holds more VB100% Awards than any other antivirus program in the world.

In his CNet / ZDNet review, Ken Feinstein, a part-time amateur antivirus tester with no credentials whatsoever in the antivirus industry, used a handful of Rosenthal Utilities simulated viruses to rate NOD32 much lower in virus detection than Virus Bulletin (or any other competent antivirus software reviewer) has ever rated it !!!

One would think CNet / ZDNet would have learned from their past mistakes, but it seems they still think their amateur reviewers know more about antivirus product testing than antivirus professionals.  

Joe Wells, Founder and Director of the WildList Organization, wrote a scathing commentary about Gregg Keizer's September 2000 CNet antivirus program review in an open letter to CNet, in which he said :

If a product does not report a simulated virus as being infected, it's right.  And if a program does report a simulated virus as being infected, it's wrong.  Thus, using simulated viruses in a product review inverts the test results.   It grossly misrepresents the truth of the matter because : 

• It rewards the product that incorrectly reports a non-virus as infected.
  
  
• It penalizes a product that correctly recognizes the non-virus as not infected.

Norton AntiVirus --- CNet's long-time "Editor's Choice" --- does detect Rosenthal's non-virus files as infected!

NOD32 --- rated as "Poor" in detection by CNet --- doesn't detect Rosenthal's non-virus files as viruses!

During the 18 months since Joe Wells informed CNet that their antivirus product testing methodology was fatally flawed, CNet has continued to use Rosenthal's simulated viruses in their tests and ha continued to reward Norton AntiVirus with "highest detection" ratings and "Editor's Choice" awards for detecting them as infected while penalizing other antivirus programs for ( correctly! ) not detecting them as infected !!!

 ( Joe Wells' open letter was co-signed by no less than 19 antivirus professionals!   Read the full text here )

In his open letter condemning CNEt's September 2000 review, Joe Wells said :

Most antivirus companies are under some form of self-imposed restrictions that prevent them from knowingly creating new viruses or virus variants. In addition, competent testing and certification bodies such as ICSA, Virus Bulletin, Secure Computing, and AV-Test.org, do not create new viruses or virus variants for testing.

Indeed, the consensus throughout the antivirus development and testing community is that creating a new virus or variant for product testing would be very bad — and totally unnecessary. To do so would undoubtedly raise questions about their ethics.

Whether or not CNET knew this fact is unknown, but they did in fact create two new virus variants for their testing. Please note this fact as described in the "How We Tested" section.

We scanned for the I Love You virus in three different ways. In the first test, we left the code as is. In the second test, we changed every reference to love in the code. In the third test, we changed the size of the file by inserting a comment that did not affect the code.

Changing an existing virus results in a new virus. If a testing body does this, they brand themselves with, as it were, a scarlet "V" (as has CNET at this point). They mark themselves as a virus creating organization in the eyes of antivirus experts worldwide.

In his September 2000 review, Gregg Keizer states, "We scanned for the I Love You virus in three different ways. In the first test, we left the code as is. In the second test, we changed every reference to love in the code. In the third test, we changed the size of the file by inserting a comment that did not affect the code."

In his April 2002 review, Ken Feinstein states, "To test the I Love You virus, we copied and pasted the virus's code into Notepad and tested it three different ways. In the first test, we left the code untouched. In the second test, we changed every reference to love in the code. In the third test, we changed the size of the file by inserting a comment that does not affect the code."

In his September 2000 review, Gregg Keizer states, "CNET Labs used Rosenthal Utilities, a program that simulates viruses, to test for virus detection in main memory, in the file sector of floppy disks in A: drive, on the hard drive, and in the boot sector of floppy disks in A: drive."

In his April 2002 review, Ken Feinstein states, "We used Rosenthal Utilities, a program that simulates viruses, to test for virus detection in three places: main memory and the file and boot sectors of floppy disks in A: drive."

In his September 2000 review, Gregg Keizer states, "We also tested email protection by sending scrap files to the test computer. The scrap file (.shs) we tested will, when executed, attempt to reformat any disk in A: drive. Finally, we tested the current Outlook-specific email virus called KakWorm."

In his April 2002 review, Ken Feinstein states, "We also tested e-mail protection by sending a Scrap Object (SHS) embedded in a DOC file to the test computer. This object, when double-clicked, attempts to format whatever disk is in A: drive. Finally, we tested an Outlook-specific e-mail virus called Kakworm."

( Read "How we tested" from the September 2000 CNet review by Gregg Keizer here )

( Read "How we tested" from the April 2002 CNet review by Ken Feinstein here )

One could be forgiven for wondering :

• Why does CNet continue to create new viruses for their detection tests ?
  
  
• Why does CNet continue to use pretend viruses in their detection tests ?
  
  
• Is "Ken Feinstein" also "Gregg Keizer" ?
  
  
• Did Ken Feinstein modify the "I Love You" virus in exactly the same way as Gregg Keizer modified it, despite the fact that Joe Wells clearly told CNet "They mark themselves as a virus creating organization in the eyes of antivirus experts worldwide" back in October 2000 ?
  
  
• Did Ken Feinstein actually test NOD32 (or any of the other antivirus programs in his review) at all --- or did he simply plagiarize Gregg Keizer's comments from his September 2000 review, modify them slightly, and include them in his own April 2002 review ?

This highlights Mr Feinstein's complete lack of understanding of On Access Scanners!

On Access Scanners are designed to detect virus infiltrations when an infected file is accessed.

No On Access Scanner in the world would have detected Mr Feinstein's "a few virus-created files in the Windows directory" !!!

If Ken Feinstein had deliberately infected TEN THOUSAND files with Gibe for his test, AMON would have detected the virus ONLY in the file he executed, in memory, and the virus-created Registry entries.

By detecting the Gibe infection when it was triggered, AMON did exactly what it should have done !!!

NOD32's On Demand Scanner would have detected and disinfected the other 9,999 Gibe-infected files --- just like it detected and disinfected Mr Feinstein's "a few virus-created files in the Windows directory".

In his review, Ken Feinstein states that NOD32 doesn't scan ZIP files.

Obviously Mr Feinstein didn't bother looking at NOD32 very closely!

NOD32 does scan ZIP files --- and it scans RAR, ARJ, LZH, and LHA files too !!!

In the "Ability to catch wild and currently circulating viruses" section of his April 2002 CNet / ZDNet review, amateur antivirus program tester Ken Feinstein rates NOD32 as "Poor".

NOD32 was awarded ICSA Certification --- with 100% "in the wild" virus detection --- in the same month he wrote his "Virus Underdogs" review --- and ICSA tests against thousands of real viruses --- not just a few pretend viruses!

According to Virus Bulletin's professional reviewers, NOD32 is the only antivirus program in the world which has not missed a single "in the wild" virus in a Virus Bulletin VB100% test since it was first tested in May 1988 !!!

Obviously Mr Feinstein didn't bother looking up any of the many professional reviews of NOD32, or he would have wondered why his own detection figures were so ridiculously low and would have checked into it further.

It's Ken Feinstein's review --- not NOD32's ability to catch wild and currently circulating viruses --- which is "Poor" !!!

Now we start getting into Ripley's Believe It or Not territory !!!

According to Ken Feinstein, NOD32 "couldn't even detect Kakworm-infected e-mail within a mailbox file when we manually scanned the entire file."

Mr Feinstein's statement that he manually scanned a mailbox file with NOD32 is very interesting !!!

Either this test was never performed or the tester had no idea what he was doing !!! 

The NOD32 Antivirus System does scan email --- the POP3 Scanner scans icoming email on arrival, and AMON monitors incoming email attachments --- but --- NOD32 doesn't scan mailboxes AT ALL !!!

Even if we ignore all the other mistakes in the review, this alone shows that ( giving him the benefit of the doubt and assuming he tested NOD32 at all ) Ken Feinstein's testing methodology was at the very least sloppy and amateurish!

His "review" and its ridiculous "findings" cannot possibly be taken seriously !!!

( You can read Ken Feinstein's April 2002 review of NOD32 here )

Addendum - 08 May 2002

What you read on CNet's website now is not the original review!

After we complained about Ken Feinstein's review, CNet added the following text :

But NOD32's track record with the 200 or so circulating wild viruses tops the charts. In our tests, we also check how well an antivirus program handles the current viruses in the WildList as an indicator of a program's performance. And Virus Bulletin's 100% Award, handed out only to programs that spot every virus making the rounds, shows that NOD32 performs as well as Norton AntiVirus.

"as well as Norton AntiVirus" ???

Let's take a look at the real facts and figures!

NOD32 is way out in front of Norton AntiVirus in VB100% Awards !!!

• NOD32 has won 17 out of 18 submissions!
  
  
• Norton AntiVirus has won 16 out of 22 submissions!

According to Virus Bulletin's published tests on thousands of real viruses :

• between February 2000 to February 2002, NOD32 missed a total of 227 viruses!
  
  
• between February 2000 to February 2002, Norton AntiVirus missed a total of 1,597 viruses!

Virus Bulletin's figures clearly show that NOD32 performs significantly better than Norton AntiVirus --- against real viruses !!!

Addendum - 10 May 2002

CNet has added the following text to their "corrections" page :

In some of our original reviews of antivirus products, we acknowledged the test results of a British publication, Virus Bulletin, while in other reviews, we did not. We have updated all the reviews where appropriate to include links to the Virus Bulletin 100% list. The additions do not change any product ratings.

"The additions do not change any product ratings" indicates that, despite all the evidence to the contrary, CNet still thinks their review of NOD32 was fair and and factual. They don't seem the least bit inclined to admit their review and its ratings were extremely unprofessional and grossly misleading --- nor do they seem inclined to apologize to us or their readers for publishing such blatant hogwash.

The fact is, apart from using a fatally flawed and unfair testing methodology ( a testing methodology which CNet has known is fatally flawed and unfair for nearly two years, but which they continue to use !!! ) to produce a ridiculously low virus detection rating for NOD32, the review contained a number of demonstrably false statements about NOD32's features and performance!

CNet's sloppy and amateurish review unfairly trashed NOD32 !!!

A couple of lines of weasel words do not repair the damage caused to NOD32's fine reputation by CNet's "review" !!!

A couple of lines of weasel words do not compensate us for the sales we undoubtedly missed thanks to CNet's "review" !!!

How about a little fair play, Cnet ???

Here are just a few of the many feedback comments on the review, from CNet's website

" The Best! Period! "
You need a knowledgeable reviewer and a credible hypothesis before undertaking any type of technical review. Unfortunately, both the criteria are missing! This rating that CNet has published looks entirely bogus. Good results are made to look bad, and bad results are made to look good.

" Simply The Best "
I see by the ratio of "Thumbs Up" to "Thumbs Down" votes that there are a lot of avid NOD32 fans out there. I'm not one of them (yet) but I'm VERY impressed with the program and we will be purchasing a licence for our network soon. We were hit badly by a Klez Worm which came in E-mail last week. NAV 2002 detected hundreds of earlier Klez Worms in E-mail, but it missed this one. The Worm destroyed NAV 2002 and infected thousands of files across the network with the Elkhorn Virus. I was never happy with the bloated NAV 2002 anyway, so I took this opportunity to go looking for a replacement anti-virus program. I found a free fix on the NOD32 web site, and I cleared up the Klez Worm and the thousands of Elkhorn Virus infections in less than one hour. I installed the free trial version, and it has performed flawlessly ever since. Incidentally, the comment "NOD32 missed 47 viruses and caused my PC to crash" defies logic. Unless the complainant is a "virus collector", it's doubtful that any scanner in the world would miss 47 viruses on his PC, let alone the only scanner in the world which appears to have an umblemished detection record in Virus Bulletin tests.
Dr. Raymond Jamieson, Ph.D.

" 6/10 ?   You've got to be joking! "
Your sloppy review of NOD32 and the other 4 programs is highly misleading. You link to NAV on every page of the review. The whole thing reads like one big Norton ad. Your own help.com used to warn against using simulated viruses to test anti-virus, but that item has been removed. How convenient for Ken Feinstein! Why should I believe your ridiculous 6/10 when ICSA and VIRUS BULLETIN rate NOD32 10/10, and NOD32 holds the world record for VB-100%25 awards ? You people owe the NOD32 guys and the other anti-virus companies an apology for publishing such rubbish!

" The Best "
I never did trust magazine reviews of software, and CNet's review is no exception. I trust much more laboratories such as Virus Bulletin, on which NOD32 does very well. As one user states, they use real world viruses. Anytime I see Rosenthal or any such synthetic viruses that were used to form a review, the review loses all credibility. Give me a real world test with real world viruses any day. Symantec's NAV also does very well in Virus Bulletin tests, but it is a bloated resource hog whose scan is as slow as a snail. NOD32 takes up little resources and scans very quickly, the fastest in fact. A real review would consider such issues instead of how the interface looks.

" No Bloatware for me "
NOD32 not only outperforms in terms of efficiency and system performance any of the "top" anti-virus programs its heuristics are 10 years ahead of the competition. In our test of 1200 PCs NOD32 takes care of the lower end PCs. It can run on a 386DX! NOD32 is written in machine code. Other ant-virus programs are written in higher level languages, making the PC "think" more than it should. Two thumbs up from me.

" What's your agenda, Cnet ? "
Considering the opinions of respected Anti Virus testing organizations and my own experience, I must question the intentions of Cnet and this seemingly biased report. I have used NOD32 for quite some time now and it has stopped every virus and worm that has hit my inbox -- including the ONE Cnet said it failed to detect. Cnet, what do you hope to gain by this obviously incorrect report ? You certainly have lost any respect I ever had for your "opinions".

" Hey CNET, read this "
I wonder if CNET would care to comment on this quote from the ICSA Certification Lab ? "NOD32 ICSA Certification is yet another mark of quality of ESET's sophisticated product. Complete detection of all virus samples of NOD32 system was combined with impressive scanning detection rate and surprisingly low system footprint," commented Larry Bridwell, ICSA Labs Content Security Programs Manager, after completion of the tests.

( You can read a lot more feedback and add your own comments here )

( Our Snake Oil page is recommended reading. You can read it here )

The bottom line is
  Who has more credibility in the antivirus world --- CNet and Ken Feinstein, or ICSA and Virus Bulletin ?