Security certification in payment card industry: Testbeds, measurements, and recommendations

Sazzadur Rahaman, Gang Wang, Danfeng Yao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

25 Scopus citations


The massive payment card industry (PCI) involves various entities such as merchants, issuer banks, acquirer banks, and card brands. Ensuring security for all entities that process payment card information is a challenging task. The PCI Security Standards Council requires all entities to be compliant with the PCI Data Security Standard (DSS), which specifies a series of security requirements. However, little is known regarding how well PCI DSS is enforced in practice. In this paper, we take a measurement approach to systematically evaluate the PCI DSS certification process for e-commerce websites. We develop an e-commerce web application testbed, BuggyCart, which can flexibly add or remove 35 PCI DSS related vulnerabilities. Then we use the testbed to examine the capability and limitations of PCI scanners and the rigor of the certification process. We find that there is an alarming gap between the security standard and its real-world enforcement. None of the 6 PCI scanners we tested are fully compliant with the PCI scanning guidelines, issuing certificates to merchants that still have major vulnerabilities. To further examine the compliance status of real-world e-commerce websites, we build a new lightweight scanning tool named PciCheckerLite and scan 1,203 e-commerce websites across various business sectors. The results confirm that 86% of the websites have at least one PCI DSS violation that should have disqualified them as non-compliant. Our in-depth accuracy analysis also shows that PciCheckerLite's output is more precise than w3af. We reached out to the PCI Security Council to share our research results to improve the enforcement in practice.

Original languageEnglish (US)
Title of host publicationCCS 2019 - Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security
PublisherAssociation for Computing Machinery
Number of pages18
ISBN (Electronic)9781450367479
StatePublished - Nov 6 2019
Externally publishedYes
Event26th ACM SIGSAC Conference on Computer and Communications Security, CCS 2019 - London, United Kingdom
Duration: Nov 11 2019Nov 15 2019

Publication series

NameProceedings of the ACM Conference on Computer and Communications Security
ISSN (Print)1543-7221


Conference26th ACM SIGSAC Conference on Computer and Communications Security, CCS 2019
Country/TerritoryUnited Kingdom


  • Data Breach
  • Data Security Standard
  • E-commerce
  • Internet Measurement
  • Payment Card Industry
  • Testbed
  • Web Security
  • Website Scanning

ASJC Scopus subject areas

  • Software
  • Computer Networks and Communications

Cite this