1. The value of Multi-Scanning Benny Czarny CEO OPSWAT, Inc.

2. Why Multi-Scanning ?What are the threats we are up against ?What is the capability of our solution?

3. What are the threats we are up against ? Differences in reporting the total amount of threatsSource: McAfeeSource: Av-Test.org

4. What are the threats we are up against ? Differences in detection rate for new malwareSource: McAfeeSource: Av-Test.org

5. What is the capability of our solution ? Measuring the quality of anti-malware engines Detection coverage Response time Operating system compatibility Amount of false positive Other metrics

6. What is the capability of our solution ? Measuring the quality of anti-malware engines November 2010 February 2011 August 2011AV Comparatives 97.6 % 95.8 % 92.1 %AV Test 97 % 99 % 96 % AMTSO mission is to develop and publish standards and best practices for testing of anti- malware products

7. Why Multi-Scanning ? Conclusions No current answer about the amount of threats No clear answer about the quality of anti-malware engines

8. Multi-ScanningCan we quantify advantages anddisadvantages of multi-scanning?

9. Multi-ScanningAdvantages Disadvantages Improve malware Increase amount of detection False Positives Decrease detection time Decrease of an outbreak performance Increase resiliency to Costly antivirus engines vulnerability

10. Advantages - Improve malware detection Measuring detection coverage 100% Antivirus 1 97.2% Detection Rate: Antivirus 2 92.1% Detection Rate:Source: www.av-comparatives.org

11. Advantages - Improve malware detection Threats detected by Antivirus A and Antivirus B Malware sharing programs between vendors In the wild 3rd party sites e.g metascan-online.com virustotal.com jotti.comSource: www.av-comparatives.org

12. Advantages - Improve malware detection Factors affecting detection rate of a single antivirus Quality of software code Malware detection engine Signature database Update frequency Location of the analysts Other factors

13. Advantages - Improve malware detection Software reliability modelsProvide developers and managers with reasonably accurate quantitativeestimates of the softwares reliabilityFailure rate, N, can be made.N = F*K* ( *Number of lines of source code)WhenF is the programs linear execution frequencyK is the defect exposure ratio

14. Advantages - Improve malware detection Software reliability modelSource: AV-Test.org

15. Advantages - Improve malware detectionDefects Assuming linear growth of malware

16. Advantages - Improve malware detection AV-Test.orgs Malware CollectionSource: AV-Test.org

17. Advantages - Improve malware detection Assuming exponential growth of malwareDefects

18. Advantages - Improve malware detection Antivirus 1 QA defects not detected by Antivirus 2 And unique samples Shared samples Antivirus 2 QA defects not detected by Antivirus 1 And unique samplesSource: www.av-comparatives.org

19. Advantages - Improve malware detection ProbabilityP( A ) = Probability of Antivirus A to Detect a virusP( B ) = Probability of Antivirus B to Detect a virusThe probability that Antivirus A or Antivirus B detect a virus P(A B) = P(A) + P(B) - P(A B)

20. Advantages - Decrease detection time of an outbreak Source: AV-Test.org Malware Name Malware Name Time Difference FromAV 1 W32/Bredolab/Genreic2 Zero-hour - No detectionAV 2 Win32.Bredolab-BC [Trj] 24.28 hrs. Win32.Bredolab-BN(Trj) 2.10 hrs.AV 3 Agent2.ABYO (Trojan horse) 10.18 hrs. Win32/Cryptor 3.52 hrs.AV 4 - No detection Win32/Bredolab.Cgeneric Zero-hourAV 5 Trojan.Agent-130266 40.82 hrs. - No detectionAV 6 Trojan.Botnetlog.II 3.68 hrs. Trojan.Botnetlog.140 13.17 hrs.AV 7 Win32/TrojanDownloader.Bredolab.AA trojan 2.35 hrs. Win32/Kryptik.BHT trojan (variant) Zero-hourAV 8 Gen:Trojan Heur.bqW@yzoXKwacdf Zero-hour Trojan Downloader.Bredolab CK 20.03 hrs.AV 9 Trojan.Win32.Bredolab 2.55 hrs. Downloader Delphi 1.90 hrs.AV 10 - No detection - No detectionAV 11 Backdoor.Win32.Bredolab.bge 6.70 hrs. Backdoor.Win32.Bredolab.btw 14.52 hrs.AV 13 Generic Dropper.Ir(trojan) 28.83 hrs. - No detectionAV 14 TrojanDownloader:Win32/Bredolab X 11.62 hrs. - No detectionAV 15 W32/Obfuscated D Zero-hour - No detectionAV 16 Trj/Sinowal WRW 76.48 hrs. - No detectionAV 17 Trojan.Win32.GenericSIF369E9 71.27 hrs. - No detectionAV 18 - No detection - No detectionAV 19 - No detection Trojan.Win32.Bredolab.Gen2(v) Zero-hourAV 20 Trojan.Fraudload.Gen!Pac.5(mutant) 4.05 hrs. TrojanFraudload Gen!Pac 5 (mutant) Zero-hour

21. Advantages - Decrease detection time of an outbreak Theoretical average time to decrease the detection of an outbreak Number of engines Average time to respond to an outbreak

22. Advantages - Decrease detection time of an outbreak Example handling a specific outbreak with 1-30 antivirus engines605040302010 Amount of engines Average time to respond to an outbreak 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

23. Advantages - Increase resiliency to antivirus engines vulnerability Vulnerabilities of selected 4 engines Number of advisories on the selected AVs. In a 3 years2.5 21.5 10.5 0 AV 1 AV 2 AV 3 AV 4

24. Advantages - Increase resiliency to antivirus engines vulnerability Known and Known and unknown unknown Vulnerabilities in Vulnerabilities in Antivirus 1 Antivirus 2Source: www.av-comparatives.org

25. Advantages - Increase resiliency to antivirus engines vulnerabilityP(A) = the probability of one Antivirus A to encounter a vulnerabilityP(B) = the probability of one Antivirus A to encounter a vulnerabilityP(A B) = P(A)*P(B)The Challenge - The vulnerability will not effect the multiscanner software

26. Disadvantages of Multi-Scanning Increased amount of false positives Decreased performance Costly

27. Disadvantages - Increased amount of false positives Measuring detection coverage Antivirus 1 False Positives: Antivirus 2 False PositivesSource: www.av-comparatives.org

28. Disadvantages - Increased amount of false positives Antivirus 1 Antivirus 2 8 false positives 10 false positives 14 AbsoluteBlue package AbsoluteBlue packageAzarus package Win32:Malware-genTrojan.Generic.6304836 Win32:Malware-gen DateCalc packageBuchdruck package Azarus package Win32:Trojan-genGen:Variant.Zbot.29 Trojan.Generic.6304836 DB2EXE packageIntrapact package Buchdruck package Win32:Malware-genGen:Trojan.Heur.VP2.fm0@a5Koffgi Gen:Variant.Zbot.29 Fiman packageShellex package DateCalc package Win32:Malware-genGen:Variant.Kazy.17493 Win32:Trojan-gen FTPcontrol packageSkriptum package DB2EXE package Win32:Malware-gen Win32:Malware-gen Joshua packageExploit.CVE-2011-0977.Gen Fiman package Win32:Malware-genVirtualization package Sardu packageGen:Trojan.Heur.KT.4.bq8@aqLITyf Win32:Malware-gen Win32:Dropper-FRUWinnerTw package FTPcontrol package Shannel packageGen:Variant.Kazy.18603 Win32:Malware-gen Win32:FasecWoodMahjongg package Intrapact package ShellPicture packageGen:Variant.Kazy.14979 Gen:Trojan.Heur.VP2.fm0@a5Koffgi Win32:Malware-gen Joshua package xComposer package Win32:Malware-gen Win:32:SMorph ShellPicture package Win32:Malware-gen Virtualization package Gen:Trojan.Heur.KT.4.bq8@aqLITyf WinnerTw package Gen:Variant.Kazy.18603 WoodMahjongg package Gen:Variant.Kazy.14979 xComposer package Test performed August 2011 Win:32:SMorph Source: www.av-comparatives.org

29. Disadvantages - Increased amount of false positivesP(A) = Probability of Antivirus A to Detect a false positiveP(B) = Probability of Antivirus B to Detect a false positiveThe probability that Antivirus A or Antivirus B reports a falsepositiveP(A B) = P(A) + P(B) - P(A B)

30. Disadvantages - Increase amount of False Positives How can white list engine helpP(A) = Probability of Antivirus A to Detect a virusP(B) = Probability of Antivirus B to Detect a virusP(C) = Probability of White list Engine to miss a threatThe probability that Antivirus A or Antivirus B detect a virus P(A B) = P(A) + P(B) - P(A B)- P(C)

31. Disadvantages - Decreased performance AssumptionMulti-scanning Engine 1 Engine 2 Engine 3 Engine 4 Engine 5 Engine 6 Engine 7 0 5 10 15 20 25 30 35 40 Time

32. Disadvantages - Decreased Performance Way to increase performance Reduce Redundant tasks such as Open archives Detect file types Use different engines based on their capabilities to detect threats in different file types Usage of distributed computing Usage of multicore processing Force scanning in memory

33. Disadvantages - Decreased performance Reality Presumed Speed 1 engine 3 engines 8 engines Actual Speed SYSTEM PROFILE OS: Windows Server 2008 R2 CPU: Intel Xeon 2.13GHz 8cores RAM: 8GBPDF EXE JPG OTHER

34. Disadvantages - Costly Linear increased bandwidth consumption Increase in hardware requirements IT training Compliance checks is becoming more complex The solution cost more

35. ConclusionThe argument for multi-scanning isclear though it is difficult tomeasure its advantages.

36. ReferencesAV-test.comAV-Comperatives.comwww.metascan-online.comAMTSOSoftware system defect content prediction from development processand product characteristics - Harris instituteMcAfee