This page is an attempt to organize awesome publications and researchers on security metrics.

Before 2005

Towards a classification of security metrics
Villarrubia, Carlos, Eduardo Fernández-Medina, Mario Piattini
Proceedings of the 2nd International Workshop on Security in Information Systems, 2004.

2005

On the Effectiveness of Distributed Worm Monitoring
Moheeb Abu Rajab, Fabian Monrose, Andreas Terzis
Proceedings of the 2005 USENIX Security Symposium, USENIX Security 2005

2006

A Framework for the Evaluation of Intrusion Detection Systems
Alvaro A. Cárdenas, John S. Baras, Karl Seamon
Proceedings of the 2006 IEEE Symposium on Security and Privacy, S&P 2006

2007

Optimal security hardening using multi-objective optimization on attack tree models of networks
Rinku Dewri, Nayot Poolsappasit, Indrajit Ray, L. Darrell Whitley
Proceedings of the 2007 ACM Conference on Computer and Communications Security, CCS 2007

2008

Measurement and Classification of Humans and Bots in Internet Chat
Steven Gianvecchio, Mengjun Xie, Zhengyu Wu, Haining Wang
Proceedings of the 2008 USENIX Security Symposium, USENIX Security 2008

2009

Quantified security is a weak hypothesis: A critical survey of results and assumptions
Verendel Vilhelm
Proceedings of the 2009 workshop on New security paradigms workshop, 2009.

On cellular botnets: measuring the impact of malicious devices on a cellular network core
Patrick Traynor, Michael Lin, Machigar Ongtang, Vikhyath Rao, Trent Jaeger, Patrick D. McDaniel, Thomas La Porta
Proceedings of the 2009 ACM Conference on Computer and Communications Security, CCS 2009

Automatic Discovery and Quantification of Information Leaks
Michael Backes, Boris Köpf, Andrey Rybalchenko
Proceedings of the 2009 IEEE Symposium on Security and Privacy, S&P 2009

2010

Testing metrics for password creation policies by attacking large sets of revealed passwords
Matt Weir, Sudhir Aggarwal, Michael P. Collins, Henry Stern
Proceedings of the 2010 ACM Conference on Computer and Communications Security, CCS 2010

Sidebuster: automated detection and quantification of side-channel leaks in web application development
Kehuan Zhang, Zhou Li, Rui Wang, XiaoFeng Wang, Shuo Chen
Proceedings of the 2010 ACM Conference on Computer and Communications Security, CCS 2010

2011

Measuring and Analyzing Search-Redirection Attacks in the Illicit Online Prescription Drug Trade
Nektarios Leontiadis, Tyler Moore, Nicolas Christin
Proceedings of the 2011 USENIX Security Symposium, USENIX Security 2011

2012

Guess Again (and Again and Again): Measuring Password Strength by Simulating Password-Cracking Algorithms
Patrick Gage Kelley, Saranga Komanduri, Michelle L. Mazurek, Richard Shay, Timothy Vidas, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor, Julio López
Proceedings of the 2012 IEEE Symposium on Security and Privacy, S&P 2012

Impact of Spam Exposure on User Engagement
Anirban Dasgupta, Kunal Punera, Justin M. Rao, Xuanhui Wang
Proceedings of the 2012 USENIX Security Symposium, USENIX Security 2012

How Does Your Password Measure Up? The Effect of Strength Meters on Password Creation
Blase Ur, Patrick Gage Kelley, Saranga Komanduri, Joel Lee, Michael Maass, Michelle L. Mazurek, Timothy Passaro, Richard Shay, Timothy Vidas, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor
Proceedings of the 2012 USENIX Security Symposium, USENIX Security 2012

Using probabilistic generative models for ranking risks of Android apps
Hao Peng, Christopher S. Gates, Bhaskar Pratim Sarma, Ninghui Li, Yuan Qi, Rahul Potharaju, Cristina Nita-Rotaru, Ian M. Molloy
Proceedings of the 2012 ACM Conference on Computer and Communications Security, CCS 2012

How secure are power network signature based time stamps?
Wei-Hong Chuang, Ravi Garg, Min Wu
Proceedings of the 2012 ACM Conference on Computer and Communications Security, CCS 2012

Towards a bayesian network game framework for evaluating DDoS attacks and defense
Guanhua Yan, Ritchie Lee, Alex Kent, David H. Wolpert
Proceedings of the 2012 ACM Conference on Computer and Communications Security, CCS 2012

2013

Measuring the Practical Impact of DNSSEC Deployment
Wilson Lian, Eric Rescorla, Hovav Shacham, Stefan Savage
Proceedings of the 2013 USENIX Security Symposium, USENIX Security 2013

WHYPER: Towards Automating Risk Assessment of Mobile Applications
Rahul Pandita, Xusheng Xiao, Wei Yang, William Enck, Tao Xie
Proceedings of the 2013 USENIX Security Symposium, USENIX Security 2013

Quantifying the security of graphical passwords: the case of android unlock patterns
Sebastian Uellenbeck, Markus Dürmuth, Christopher Wolf, Thorsten Holz
Proceedings of the 2013 ACM Conference on Computer and Communications Security, CCS 2013

Impact of integrity attacks on real-time pricing in smart grids
Rui Tan, Varun Badrinath Krishna, David K. Y. Yau, Zbigniew Kalbarczyk
Proceedings of the 2013 ACM Conference on Computer and Communications Security, CCS 2013

2014

Quantifying Information Flow for Dynamic Secrets
Piotr Mardziel, Mário S. Alvim, Michael W. Hicks, Michael R. Clarkson
Proceedings of the 2014 IEEE Symposium on Security and Privacy, S&P 2014

Targeted Threat Index: Characterizing and Quantifying Politically-Motivated Targeted Malware
Seth Hardy, Masashi Crete-Nishihata, Katharine Kleemola, Adam Senft, Byron Sonne, Greg Wiseman, Phillipa Gill, Ronald J. Deibert
Proceedings of the 2014 USENIX Security Symposium, USENIX Security 2014

Structural Data De-anonymization: Quantification, Practice, and Implications
Shouling Ji, Weiqing Li, Mudhakar Srivatsa, Raheem A. Beyah
Proceedings of the 2014 ACM Conference on Computer and Communications Security, CCS 2014

Decide Now or Decide Later? Quantifying the Tradeoff between Prospective and Retrospective Access Decisions
Wen Zhang, You Chen, Thaddeus Cybulski, Daniel Fabbri, Carl A. Gunter, Patrick Lawlor, David M. Liebovitz, Bradley A. Malin
Proceedings of the 2014 ACM Conference on Computer and Communications Security, CCS 2014

AutoCog: Measuring the Description-to-permission Fidelity in Android Applications
Zhengyang Qu, Vaibhav Rastogi, Xinyi Zhang, Yan Chen, Tiantian Zhu, Zhong Chen
Proceedings of the 2014 ACM Conference on Computer and Communications Security, CCS 2014

Every Second Counts: Quantifying the Negative Externalities of Cybercrime via Typosquatting
Mohammad Taha Khan, Xiang Huo, Zhou Li, Chris Kanich
Proceedings of the 2014 ACM Conference on Computer and Communications Security, CCS 2014

2015


Proceedings of the 2015 IEEE Symposium on Security and Privacy, S&P 2015

Measuring the Longitudinal Evolution of the Online Anonymous Marketplace Ecosystem
Kyle Soska, Nicolas Christin
Proceedings of the 2015 USENIX Security Symposium, USENIX Security 2015

A Measurement Study on Co-residence Threat inside the Cloud
Zhang Xu, Haining Wang, Zhenyu Wu
Proceedings of the 2015 USENIX Security Symposium, USENIX Security 2015

Control-Flow Bending: On the Effectiveness of Control-Flow Integrity
Nicholas Carlini, Antonio Barresi, Mathias Payer, David A. Wagner, Thomas R. Gross
Proceedings of the 2015 USENIX Security Symposium, USENIX Security 2015

Measuring Real-World Accuracies and Biases in Modeling Password Guessability
Blase Ur, Sean M. Segreti, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor, Saranga Komanduri, Darya Kurilova, Michelle L. Mazurek, William Melicher, Richard Shay
Proceedings of the 2015 USENIX Security Symposium, USENIX Security 2015

2016

A Survey on Systems Security Metrics
Pendleton Marcus, Richard Garcia-Lebron, Jin-Hee Cho, and Shouhuai Xu
ACM Computing Surveys, CSUR 2016.

Towards Measuring and Mitigating Social Engineering Software Download Attacks
Terry Nelms, Roberto Perdisci, Manos Antonakakis, Mustaque Ahamad
Proceedings of the 2016 USENIX Security Symposium, USENIX Security 2016

2017

Towards Evaluating the Robustness of Neural Networks
Nicholas Carlini, David A. Wagner
Proceedings of the 2017 IEEE Symposium on Security and Privacy, S&P 2017

XHOUND: Quantifying the Fingerprintability of Browser Extensions
Oleksii Starov, Nick Nikiforakis
Proceedings of the 2017 IEEE Symposium on Security and Privacy, S&P 2017

Global Measurement of DNS Manipulation
Paul Pearce, Ben Jones, Frank Li, Roya Ensafi, Nick Feamster, Nicholas Weaver, Vern Paxson
Proceedings of the 2017 USENIX Security Symposium, USENIX Security 2017

Measuring HTTPS Adoption on the Web
Adrienne Porter Felt, Richard Barnes, April King, Chris Palmer, Chris Bentzel, Parisa Tabriz
Proceedings of the 2017 USENIX Security Symposium, USENIX Security 2017

2018

When Your Fitness Tracker Betrays You: Quantifying the Predictability of Biometric Features Across Contexts
Simon Eberz, Giulio Lovisotto, Andrea Patane, Marta Kwiatkowska, Vincent Lenders, Ivan Martinovic
Proceedings of the 2018 IEEE Symposium on Security and Privacy, S&P 2018

Unveiling and Quantifying Facebook Exploitation of Sensitive Personal Data for Advertising Purposes
José González Cabañas, Ángel Cuevas, Rubén Cuevas
Proceedings of the 2018 USENIX Security Symposium, USENIX Security 2018

Better managed than memorized? Studying the Impact of Managers on Password Strength and Reuse
Sanam Ghorbani Lyastani, Michael Schilling, Sascha Fahl, Michael Backes, Sven Bugiel
Proceedings of the 2018 USENIX Security Symposium, USENIX Security 2018

Measuring Information Leakage in Website Fingerprinting Attacks and Defenses
Shuai Li, Huajun Guo, Nicholas Hopper
Proceedings of the 2018 ACM Conference on Computer and Communications Security, CCS 2018

Asking for a Friend: Evaluating Response Biases in Security User Studies
Elissa M. Redmiles, Ziyun Zhu, Sean Kross, Dhruv Kuchhal, Tudor Dumitras, Michelle L. Mazurek
Proceedings of the 2018 ACM Conference on Computer and Communications Security, CCS 2018

2019

Lay Down the Common Metrics: Evaluating Proof-of-Work Consensus Protocols' Security
Ren Zhang, Bart Preneel
Proceedings of the 2019 IEEE Symposium on Security and Privacy, S&P 2019

Does Certificate Transparency Break the Web? Measuring Adoption and Error Rate
Emily Stark, Ryan Sleevi, Rijad Muminovic, Devon O'Brien, Eran Messeri, Adrienne Porter Felt, Brendan McMillion, Parisa Tabriz
Proceedings of the 2019 IEEE Symposium on Security and Privacy, S&P 2019

PhishFarm: A Scalable Framework for Measuring the Effectiveness of Evasion Techniques against Browser Phishing Blacklists
Adam Oest, Yeganeh Safaei, Adam Doupé, Gail-Joon Ahn, Brad Wardman, Kevin Tyers
Proceedings of the 2019 IEEE Symposium on Security and Privacy, S&P 2019

Less is More: Quantifying the Security Benefits of Debloating Web Applications
Babak Amin Azad, Pierre Laperdrix, Nick Nikiforakis
Proceedings of the 2019 USENIX Security Symposium, USENIX Security 2019

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, Dawn Song
Proceedings of the 2019 USENIX Security Symposium, USENIX Security 2019

Evaluating Differentially Private Machine Learning in Practice
Bargav Jayaraman, David Evans
Proceedings of the 2019 USENIX Security Symposium, USENIX Security 2019

Multisketches: Practical Secure Sketches Using Off-the-Shelf Biometric Matching Algorithms
Rahul Chatterjee, M. Sadegh Riazi, Tanmoy Chowdhury, Emanuela Marasco, Farinaz Koushanfar, Ari Juels
Proceedings of the 2019 ACM Conference on Computer and Communications Security, CCS 2019

Quantitative Verification of Neural Networks and Its Security Applications
Teodora Baluta, Shiqi Shen, Shweta Shinde, Kuldeep S. Meel, Prateek Saxena
Proceedings of the 2019 ACM Conference on Computer and Communications Security, CCS 2019

Program-mandering: Quantitative Privilege Separation
Shen Liu, Dongrui Zeng, Yongzhe Huang, Frank Capobianco, Stephen McCamant, Trent Jaeger, Gang Tan
Proceedings of the 2019 ACM Conference on Computer and Communications Security, CCS 2019

Robust Performance Metrics for Authentication Systems
Shridatt Sugrim, Can Liu, Meghan McLean, Janne Lindqvist
Proceedings of the 2019 Annual Network and Distributed System Security Symposium, NDSS 2019

Quantity vs. Quality: Evaluating User Interest Profiles Using Ad Preference Managers
Muhammad Ahmad Bashir, Umar Farooq, Maryam Shahid, Muhammad Fareed Zaffar, Christo Wilson
Proceedings of the 2019 Annual Network and Distributed System Security Symposium, NDSS 2019

Profit: Detecting and Quantifying Side Channels in Networked Applications
Nicolás Rosner, Ismet Burak Kadron, Lucas Bang, Tevfik Bultan
Proceedings of the 2019 Annual Network and Distributed System Security Symposium, NDSS 2019

2020

PhishTime: Continuous Longitudinal Measurement of the Effectiveness of Anti-phishing Blacklists
Adam Oest, Yeganeh Safaei, Penghui Zhang, Brad Wardman, Kevin Tyers, Yan Shoshitaishvili, Adam Doupé
Proceedings of the 2020 USENIX Security Symposium, USENIX Security 2020

A Comprehensive Quality Evaluation of Security and Privacy Advice on the Web
Elissa M. Redmiles, Noel Warford, Amritha Jayanti, Aravind Koneru, Sean Kross, Miraida Morales, Rock Stevens, Michelle L. Mazurek
Proceedings of the 2020 USENIX Security Symposium, USENIX Security 2020

An Ever-evolving Game: Evaluation of Real-world Attacks and Defenses in Ethereum Ecosystem
Shunfan Zhou, Zhemin Yang, Jie Xiang, Yinzhi Cao, Min Yang, Yuan Zhang
Proceedings of the 2020 USENIX Security Symposium, USENIX Security 2020

A Qualitative Study of Dependency Management and Its Security Implications
Ivan Pashchenko, Duc Ly Vu, Fabio Massacci
Proceedings of the 2020 ACM Conference on Computer and Communications Security, CCS 2020

Not All Coverage Measurements Are Equal: Fuzzing by Coverage Accounting for Input Prioritization
Yanhao Wang, Xiangkun Jia, Yuwei Liu, Kyle Zeng, Tiffany Bao, Dinghao Wu, Purui Su
Proceedings of the 2020 Annual Network and Distributed System Security Symposium, NDSS 2020

Withdrawing the BGP Re-Routing Curtain: Understanding the Security Impact of BGP Poisoning through Real-World Measurements
Jared M. Smith, Kyle Birkeland, Tyler McDaniel, Max Schuchard
Proceedings of the 2020 Annual Network and Distributed System Security Symposium, NDSS 2020

2021

SoK: Quantifying Cyber Risk
Daniel W. Woods, Rainer Böhme
Proceedings of the 2021 IEEE Symposium on Security and Privacy, S&P 2021

UNIFUZZ: A Holistic and Pragmatic Metrics-Driven Platform for Evaluating Fuzzers
Yuwei Li, Shouling Ji, Yuan Chen, Sizhuang Liang, Wei-Han Lee, Yueyao Chen, Chenyang Lyu, Chunming Wu, Raheem Beyah, Peng Cheng, Kangjie Lu, Ting Wang
Proceedings of the 2021 USENIX Security Symposium, USENIX Security 2021

Accurately Measuring Global Risk of Amplification Attacks using AmpMap
Soo-Jin Moon, Yucheng Yin, Rahul Anand Sharma, Yifei Yuan, Jonathan M. Spring, Vyas Sekar
Proceedings of the 2021 USENIX Security Symposium, USENIX Security 2021

OblivSketch: Oblivious Network Measurement as a Cloud Service
Shangqi Lai, Xingliang Yuan, Joseph K. Liu, Xun Yi, Qi Li, Dongxi Liu, Surya Nepal
Proceedings of the 2021 Annual Network and Distributed System Security Symposium, NDSS 2021