Security vendor cheats in anti-virus test

The largest security vendor in China, Qihoo 360, is set to lose all its current certifications after being caught cheating in independent anti-virus lab tests, by submitting products that differed from what it makes available to users.

The deception came to light after investigations by AV-Comparatives, AV-TEST and Virus Bulletin found that Qihoo 360 submitted products for testing with the Chinese company's own QVM detection engine disabled.

The submitted products ran only the detection engine provided by Romanian anti-virus developer BitDefender.

A joint statement by the three testing labs provided to iTnews said that the default configuration for all versions of Qihoo's security products in its main markets appeared to be the opposite, with the QVM engine active and BitDefender disabled.

"According to all test data, this would provide a considerably lower level of protection and a higher likelihood of false positives," the three testing labs stated.

Users can adjust settings in Qihoo's products to change which detection engine is used. However, as most users leave the settings unchanged, testing labs insist on using products with their default configurations, so as to best represent real-world usage.

The Chinese company confirmed to the labs that "some settings had been adjusted for testing" and eventually admitted that the setting was not what would be available on a standard download of Qihoo 360.

John Hawes, chief of operations at Virus Bulletin, condemned Qihoo 360's attempt at gaming the testing results.

“This sort of thing doesn’t really help anyone. Independent tests serve both users and developers, showing which products are performing best and highlighting areas where developers need to work harder. If the products being tested aren’t those being used in the real world, nobody’s getting any useful information,” Hawes said.

Chief executive of AV-Comparatives Andreas Clementi said, "misuse of [independent] tests for marketing purposes will, in the long run, result in more successful malware attacks, making internet users less secure."

Qihoo 360 was formed in 2005 and apart from security products, makes web browsers and sells advertising through internet portals.

It is listed on the New York Stock Exchange and for the fourth quarter of its fiscal 2014 year, reported [pdf] 509 million monthly active users of its PC-based products and services.

The company said it had boosted the number of users of its mobile security product from 475 million to 744 million in the December 2013-2014 year.

However, in 2012, Qihoo was accused of fudging its user numbers by a faction of the hacktivist Anonymous collective, which claimed to have found comScore data that showed the actual figures were around half of what was reported.