Compliance Cautions - NDSS Symposium
Transcript of Compliance Cautions - NDSS Symposium
![Page 1: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/1.jpg)
Compliance CautionsINVESTIGATING SECURITY ISSUES ASSOCIATED WITH U.S. DIGITAL-SECURITY STANDARDSR O C K S T E V EN S , K E V I N H A L L I D AY, M I C H E L L E M A Z U R E K / / U N I V E RS I T Y O F M A RY L A N DJ O S I A H D Y K S T R A , J A M E S C H A P M A N , A L E X FA R M E R / / I N D E P E N D E N T R E S E A R C H E R SW E N D Y K N OX E V E R E T T E / / L E V I AT H A N S E C U R I T Y G R O U PG A R R E T T B L A D O W / / D R A G O S , I N C .
![Page 2: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/2.jpg)
What are compliance standards?Series of controls or policies that establish a baseline of security
![Page 3: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/3.jpg)
Why use compliance standards?Mandatory to provide critical services or access to sensitive data
![Page 4: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/4.jpg)
How is it enforced?Audits Financial sanctionsPrivilege revocation
![Page 5: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/5.jpg)
Vendor @ RSAC20 selling compliance
![Page 6: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/6.jpg)
So what’s the problem?False sense of security
Never intended to be used as a checklist
![Page 7: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/7.jpg)
Even if you had perfect compliance, what else could go wrong?
First empirical evaluation of compliance standards for security issues that exist because of perfect compliance
![Page 8: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/8.jpg)
Audited StandardsInternal Revenue Services Publication 1075
Payment Card Industry Data Security Standard
North American Electric Reliability Corporation Critical Infrastructure Protection 007-6
![Page 9: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/9.jpg)
Our methodology
Audit
External expert evaluation
Disclose
![Page 10: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/10.jpg)
Audit intentLeverage real-world experienceMatch to exploitation in the wildDetermine root cause
![Page 11: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/11.jpg)
Determining risk estimates
Unlikely Seldom Occasional Likely Frequent
Catastrophic M H H E E
Critical L M H H E
Moderate L L M M H
Negligible L L L L M
ProbabilitySe
verit
y
Low | Medium | High | Extremely High
![Page 12: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/12.jpg)
External expert evaluationRecruited CISOs and compliance authors
Validate findings
Challenge our assumptionsProvide additional context
![Page 13: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/13.jpg)
Disclose findingsInform authors/councils
Inform users of standards
Exercise existing vuln disclosure processes
![Page 14: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/14.jpg)
Results
![Page 15: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/15.jpg)
Audit: By the numbers3 standards
148 issues 4 root causes
![Page 16: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/16.jpg)
Audit: Root causesData vulnerability
Unenforceable
Under-defined processAmbiguous specification
![Page 17: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/17.jpg)
Data vulnerabilityPCI DSS only protects enclave with cardholder data
![Page 18: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/18.jpg)
Data vulnerabilityElectric grid standards allows for variable security based on power production levels
![Page 19: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/19.jpg)
UnenforceableIRS P1075 requires multiple forms of physical security to protect data
*and*
Authorizes telework/remote access to data
![Page 20: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/20.jpg)
Under-defined processIRS P1075 mandates a network component inventory
*but*
Never establishes “ground truth”
![Page 21: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/21.jpg)
Ambiguous specificationIRS P1075: access control policies to be evaluated every 3 years.
By whom?
PCI DSS: all issues identified during a pentest must be addressed.
By when? Priority?
![Page 22: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/22.jpg)
External expert evaluation“Checklist compliance” confirmed
36/49 issues confirmed
10 plausible3 rejected (kinda)
![Page 23: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/23.jpg)
Disclosure attempts
Community knowledge
Top-level reporting
Direct reportingCVEs
US-CERT
National Vulnerability Database
MITRE Corp“Each issue that requires a separate patch can get a CVE”
![Page 24: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/24.jpg)
Disclosure attempts
Community knowledge
Top-level reporting
Direct reportingCVEs
NIST discussions on checklists
DHS “cease communications”
![Page 25: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/25.jpg)
Disclosure attempts
Community knowledge
Top-level reporting
Direct reportingCVEs
PCI Council made updates based on findings
IRS ignored all calls/texts/emails
![Page 26: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/26.jpg)
RecommendationsMake checklists
Solidify language to eliminate ambiguity
Orgs should conduct self-assessmentsBetter disclosure process
![Page 27: Compliance Cautions - NDSS Symposium](https://reader031.fdocuments.net/reader031/viewer/2022020910/6200857a344b983ea64772e3/html5/thumbnails/27.jpg)
SummaryPerfect compliance != perfect security◦ Ambiguous specifications and under-defined processes ◦ Lack of reporting makes fixing known problems harder
First study to empirically identify issues associated with complianceDeveloped methodology for assessing other frameworks
> Questions / Feedback? [email protected] | @ada95ftw