Abstract
The ability to measure privacy accurately and consistently is key in the development of new privacy protections. However, recent studies have uncovered weaknesses in existing privacy metrics, as well as weaknesses caused by the use of only a single privacy metric. Metrics suites, or combinations of privacy metrics, are a promising mechanism to alleviate these weaknesses, if we can solve two open problems: which metrics should be combined and how. In this article, we tackle the first problem, i.e., the selection of metrics for strong metrics suites, by formulating it as a knapsack optimization problem with both single and multiple objectives. Because solving this problem exactly is difficult due to the large number of combinations and many qualities/objectives that need to be evaluated for each metrics suite, we apply 16 existing evolutionary and metaheuristic optimization algorithms. We solve the optimization problem for three privacy application domains: genomic privacy, graph privacy, and vehicular communications privacy. We find that the resulting metrics suites have better properties, i.e., higher monotonicity, diversity, evenness, and shared value range, than previously proposed metrics suites.
- James Alexander and Jonathan Smith. 2003. Engineering privacy in public: Confounding face recognition. In Proceedings of the 3rd International Workshop on Privacy Enhancing Technologies (PET’03) (LNCS 2760). Springer, Dresden, 88--106.Google Scholar
Cross Ref
- Erman Ayday, Jean Louis Raisaro, Jean-Pierre Hubaux, and Jacques Rougemont. 2013. Protecting and evaluating genomic privacy in medical tests and personalized medicine. In Proceedings of the 12th ACM Workshop on Workshop on Privacy in the Electronic Society (WPES’13). ACM, New York, NY, 95--106. DOI:https://doi.org/10.1145/2517840.2517843Google Scholar
Digital Library
- Thomas Bäck. 2000. Evolutionary Computation 2: Advanced Algorithms and Operators. Taylor 8 Francis Ltd.Google Scholar
- Thomas Bäck, D. B. Fogel, and Z. Michalewicz. 2000. Evolutionary Computation 1: Basic Algorithms and Operators (1st ed.). CRC Press.Google Scholar
- Vitor Basto-Fernandes, Iryna Yevseyeva, André Deutz, and Michael Emmerich. 2017. A survey of diversity oriented optimization: Problems, indicators, and algorithms. In EVOLVE—A Bridge between Probability, Set Oriented Numerics and Evolutionary Computation VII. Springer International Publishing, Cham, 3--23. DOI:https://doi.org/10.1007/978-3-319-49325-1_1Google Scholar
- Elisa Bertino, Dan Lin, and Wei Jiang. 2008. A survey of quantification of privacy preserving data mining algorithms. In Privacy-Preserving Data Mining: Models and Algorithms. Number 34 in Advances in Database Systems. Springer, Chapter 8, 183--205.Google Scholar
- Hans-Georg Beyer and Bernhard Sendhoff. 2008. Covariance matrix adaptation revisited—The CMSA evolution strategy. In Parallel Problem Solving from Nature—PPSN X (Lecture Notes in Computer Science). Springer, Berlin, 123--132.Google Scholar
- Francesco Biscani, Dario Izzo, Wenzel Jakob, Marcus Märtens, Alessio Mereta, Cord Kaldemeyer, Sergey Lyskov, Sylvain Corlay, Benjamin Pritchard, Kishan Manani, Johan Mabille, Tomasz Miąsko, Axel Huebl, jakirkham, hulucc, polygon, Zihao Fu, The Gitter Badger, Merlin Nimier-David, Luka Čehovin Zajc, Jonas Adler, John Travers, Jeongseok Lee, Jakob Jordan, Ivan Smirnov, Huu Nguyen, Felipe Lema, Erik O’Leary, and Andrea Mambrini. 2019. Esa/Pagmo2: Pagmo 2.10. Retrieved from https://zenodo.org/record/2529931/export/hx. DOI:https://doi.org/10.5281/zenodo.2529931Google Scholar
- Julian Blank and Kalyanmoy Deb. 2019. Pymoo—Multi-Objective Optimization in Python. Retrieved from https://pymoo.org.Google Scholar
- Kalyanmoy Deb and Himanshu Jain. 2014. An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, Part I: Solving problems with box constraints. IEEE Trans. Evolution. Comput. 18, 4 (2014), 577--601. DOI:https://doi.org/10.1109/TEVC.2013.2281535Google Scholar
Cross Ref
- K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6, 2 (Apr. 2002), 182--197. DOI:https://doi.org/10.1109/4235.996017Google Scholar
Digital Library
- Claudia Diaz, Stefaan Seys, Joris Claessens, and Bart Preneel. 2003. Towards measuring anonymity. In Privacy Enhancing Technologies, Roger Dingledine and Paul Syverson (Eds.). Number 2482 in Lecture Notes in Computer Science. Springer, Berlin, 54--68.Google Scholar
- Claudia Diaz, Carmela Troncoso, and George Danezis. 2007. Does additional information always reduce anonymity? In Proceedings of the ACM Workshop on Privacy in Electronic Society (WPES’07). ACM, Alexandria, VA, 72--75. DOI:https://doi.org/10.1145/1314333.1314347Google Scholar
Digital Library
- David Eckhoff and Christoph Sommer. 2018. Readjusting the privacy goals in vehicular ad-hoc networks: A safety-preserving solution using non-overlapping time-slotted pseudonym pools. Comput. Commun. 122 (June 2018), 118--128. DOI:https://doi.org/10.1016/j.comcom.2018.03.006Google Scholar
- Michael Emmerich, Nicola Beume, and Boris Naujoks. 2005. An emo algorithm using the hypervolume measure as selection criterion. In Evolutionary Multi-Criterion Optimization. Springer, Berlin, 62--76.Google Scholar
- Félix-Antoine Fortin, François-Michel De Rainville, Marc-André Gardner, Marc Parizeau, and Christian Gagné. 2012. DEAP: Evolutionary algorithms made easy. J. Mach. Learn. Res. 13 (2012), 2171--2175. Issue Jul.Google Scholar
Digital Library
- David E. Goldberg. 1989. Genetic Algorithms in Search, Optimization and Machine Learning. Addison Wesley, Reading, MA.Google Scholar
Digital Library
- David Hadka. 2019. A Free and Open Source Python Library for Multiobjective Optimization: Project-Platypus/Platypus. Retrieved from https://github.com/Project-Platypus/Platypus.Google Scholar
- M. P. Hansen and A. Jaszkiewicz. 1998. Evaluating the Quality of Approximations to the Non-Dominated Set. Technical Report IMM Technical Report IMMREP1998-7. Technical University of Denmark.Google Scholar
- N. Hansen and A. Ostermeier. 1996. Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation. In Proceedings of IEEE International Conference on Evolutionary Computation. 312--317. DOI:https://doi.org/10.1109/ICEC.1996.542381Google Scholar
Cross Ref
- Nikolaus Hansen and Andreas Ostermeier. 2001. Completely derandomized self-adaptation in evolution strategies. Evolution. Comput. 9, 2 (2001), 159--195. DOI:https://doi.org/10.1162/106365601750190398Google Scholar
Digital Library
- Daojing He, S. Chan, and M. Guizani. 2015. Privacy and incentive mechanisms in people-centric sensing networks. IEEE Commun. Mag. 53, 10 (Oct. 2015), 200--206. DOI:https://doi.org/10.1109/MCOM.2015.7295484Google Scholar
Digital Library
- Christian Igel, Nikolaus Hansen, and Stefan Roth. 2007. Covariance matrix adaptation for multi-objective optimization. Evolution. Comput. 15, 1 (2007), 1--28. DOI:https://doi.org/10.1162/evco.2007.15.1.1Google Scholar
Digital Library
- Himanshu Jain and Kalyanmoy Deb. 2014. An evolutionary many-objective optimization algorithm using reference-point based nondominated sorting approach, Part II: Handling constraints and extending to an adaptive approach. IEEE Trans. Evolution. Comput. 18, 4 (Aug. 2014), 602--622. DOI:https://doi.org/10.1109/TEVC.2013.2281534Google Scholar
Cross Ref
- Georgios Kalogridis, Costas Efthymiou, Stojan Z. Denic, Tim A. Lewis, and Rafael Cepeda. 2010. Privacy for smart meters: Towards undetectable appliance load signatures. In Proceedings of the 1st International Conference on Smart Grid Communications (SmartGridComm’10). IEEE, Gaithersburg, MD, 232--237.Google Scholar
Cross Ref
- Hans Kellerer, Ulrich Pferschy, and David Pisinger. 2004. Multidimensional knapsack problems. In Knapsack Problems. Springer, Berlin, 235--283. DOI:https://doi.org/10.1007/978-3-540-24777-7_9Google Scholar
- J. Kennedy and R. Eberhart. 1995. Particle swarm optimization. In Proceedings of the International Conference on Neural Networks (ICNN’95), Vol. 4. 1942--1948. DOI:https://doi.org/10.1109/ICNN.1995.488968Google Scholar
- S. Kukkonen and J. Lampinen. 2005. GDE3: The third evolution step of generalized differential evolution. In Proceedings of the IEEE Congress on Evolutionary Computation, Vol. 1. IEEE, 443--450. DOI:https://doi.org/10.1109/CEC.2005.1554717Google Scholar
- Marco Laumanns, Lothar Thiele, Kalyanmoy Deb, and Eckart Zitzler. 2002. Combining convergence and diversity in evolutionary multiobjective optimization. Evolution. Comput. 10, 3 (Sept. 2002), 263--282. DOI:https://doi.org/10.1162/106365602760234108Google Scholar
Digital Library
- Marco Laumanns, Lothar Thiele, and Eckart Zitzler. 2001. SPEA2: Improving the Strength Pareto Evolutionary Algorithm. TIK-Report 103. Eidgenössische Technische Hochschule Zürich (ETH).Google Scholar
- Averill M. Law and W. David Kelton. 2000. Simulation Modelling and Analysis, 3rd ed. McGraw-Hill Education, Boston, MA.Google Scholar
- Longmei Li, Iryna Yevseyeva, Vitor Basto-Fernandes, Heike Trautmann, Ning Jing, and Michael Emmerich. 2017. Building and using an ontology of preference-based multiobjective evolutionary algorithms. In Evolutionary Multi-Criterion Optimization. Vol. 10173. Springer International Publishing, Cham, 406--421. DOI:https://doi.org/10.1007/978-3-319-54157-0_28Google Scholar
- Miqing Li and Xin Yao. 2019. Quality evaluation of solution sets in multiobjective optimisation: A survey. ACM Comput. Surv. 52, 2 (Mar. 2019), 26:1--26:38. DOI:https://doi.org/10.1145/3300148Google Scholar
Digital Library
- Silvano Martello, David Pisinger, and Paolo Toth. 2000. New trends in exact algorithms for the 0--1 knapsack problem. Eur. J. Oper. Res. 123, 2 (2000), 325--332. DOI:https://doi.org/10.1016/S0377-2217(99)00260-XGoogle Scholar
Cross Ref
- Silvano Martello and Paolo Toth. 1990. Knapsack Problems: Algorithms and Computer Implementations. John Wiley 8 Sons.Google Scholar
Digital Library
- Kaisa Miettinen. 1998. Nonlinear Multiobjective Optimization. Springer Science 8 Business Media.Google Scholar
- Rida E. Moustafa. 2011. Parallel coordinate and parallel coordinate density plots. Wiley Interdisc. Rev.: Comput. Stat. 3, 2 (2011), 134--148. DOI:https://doi.org/10.1002/wics.145Google Scholar
Digital Library
- Arvind Narayanan and Vitaly Shmatikov. 2009. De-anonymizing social networks. In Proceedings of the IEEE Symposium on Security and Privacy. IEEE, Oakland, CA, 173--187. DOI:https://doi.org/10.1109/SP.2009.22Google Scholar
Digital Library
- A. J. Nebro, J. J. Durillo, J. Garcia-Nieto, C. A. Coello Coello, F. Luna, and E. Alba. 2009. SMPSO: A new PSO-based metaheuristic for multi-objective optimization. In Proceedings of the IEEE Symposium on Computational Intelligence in Milti-Criteria Decision-Making. IEEE, 66--73. DOI:https://doi.org/10.1109/MCDM.2009.4938830Google Scholar
- J. A. Nelder and R. Mead. 1965. A simplex method for function minimization. Comput. J. 7, 4 (1965), 308--313. DOI:https://doi.org/10.1093/comjnl/7.4.308Google Scholar
Cross Ref
- Simon Oya, Carmela Troncoso, and Fernando Pérez-González. 2017. Back to the drawing board: Revisiting the design of optimal location privacy-preserving mechanisms. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS’17). ACM, Dallas, TX, 1959--1972. DOI:https://doi.org/10.1145/3133956.3134004Google Scholar
Digital Library
- Thomas Harvey Rowan. 1990. Functional Stability Analysis of Numerical Algorithms. Ph.D. Dissertation. University of Texas at Austin.Google Scholar
- Jason R. Schott. 1995. Fault Tolerant Design Using Single and Multicriteria Genetic Algorithm Optimization.Technical Report AFIT/CI/CIA-95-039. Air Force Inst of Tech Wright-Patterson AFB OH.Google Scholar
- Richard M. Shiffrin and Robert M. Nosofsky. 1994. Seven plus or minus two: A commentary on capacity limitations. Psychol. Rev. 101, 2 (1994), 357--361. DOI:https://doi.org/10.1037/0033-295X.101.2.357Google Scholar
Cross Ref
- Reza Shokri, George Theodorakopoulos, Jean-Yves Le Boudec, and Jean-Pierre Hubaux. 2011. Quantifying location privacy. In Proceedings of the IEEE Symposium on Security and Privacy (S8P’11). IEEE, 247--262. DOI:https://doi.org/10.1109/SP.2011.18Google Scholar
Digital Library
- Margarita Reyes Sierra and Carlos A. Coello Coello. 2005. Improving PSO-based multi-objective optimization using crowding, mutation and -dominance. In Evolutionary Multi-Criterion Optimization (Lecture Notes in Computer Science). Springer, Berlin, 505--519.Google Scholar
- Rainer Storn and Kenneth Price. 1997. Differential evolution—A simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11, 4 (1997), 341--359. DOI:https://doi.org/10.1023/A:1008202821328Google Scholar
Digital Library
- Paul Syverson. 2013. Why I’m not an entropist. In Proceedings of the 17th International Workshop on Security Protocols (LNCS 7028). Springer, Cambridge, UK, 213--230.Google Scholar
- El-Ghazali Talbi. 2009. Metaheuristics: From Design to Implementation. John Wiley 8 Sons.Google Scholar
Digital Library
- Chris Tofallis. 2014. Add or multiply? A tutorial on ranking and choosing with multiple criteria. INFORMS Trans. Edu. 14, 3 (2014), 109--119. DOI:https://doi.org/10.1287/ited.2013.0124Google Scholar
Digital Library
- Evangelos Triantaphyllou. 2000. Multi-Criteria Decision Making Methods: A Comparative Study. Springer U.S.Google Scholar
Cross Ref
- David A. Van Veldhuizen and Gary B. Lamont. 1998. Evolutionary computation and convergence to a Pareto front. In Late Breaking Papers at the Genetic Programming 1998 Conference. Omni Press, 221--228.Google Scholar
- Isabel Wagner. 2017. Evaluating the strength of genomic privacy metrics. ACM Trans. Priv. Secur. 20, 1 (Jan. 2017), 2:1--2:34. DOI:https://doi.org/10.1145/3020003Google Scholar
Digital Library
- Isabel Wagner and David Eckhoff. 2018. Technical privacy metrics: A systematic survey. ACM Comput. Surv. 51, 3 (2018), 57:1--57:38. DOI:https://doi.org/10.1145/3168389Google Scholar
Digital Library
- Isabel Wagner and Iryna Yevseyeva. 2020. Privacy Metrics Suites for Genomic Privacy, Vehicular Communications Privacy, and Graph Privacy (Version 1.0.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.3350563Google Scholar
- Simon Wessing. 2017. Evoalgos: Modular Evolutionary Algorithms. Retrieved from https://ls11-www.cs.tu-dortmund.de/people/swessing/evoalgos/doc/.Google Scholar
- Iryna Yevseyeva, Andreia P. Guerreiro, Michael T. M. Emmerich, and Carlos M. Fonseca. 2014. A portfolio optimization approach to selection in multiobjective evolutionary algorithms. In Proceedings of the Conference on Parallel Problem Solving from Nature (PPSN’14). Vol. 8672. Springer International Publishing, Cham, 672--681. DOI:https://doi.org/10.1007/978-3-319-10762-2_66Google Scholar
- Q. Zhang and H. Li. 2007. MOEA/D: A multiobjective evolutionary algorithm based on decomposition. IEEE Trans. Evolution. Comput. 11, 6 (Dec. 2007), 712--731. DOI:https://doi.org/10.1109/TEVC.2007.892759Google Scholar
- Y. Zhao and I. Wagner. 2019. On the strength of privacy metrics for vehicular communication. IEEE Trans. Mobile Comput. 18, 2 (Feb. 2019), 390--403. DOI:https://doi.org/10.1109/TMC.2018.2830359Google Scholar
Digital Library
- Yuchen Zhao and Isabel Wagner. 2020. Using metrics suites to improve the measurement of privacy in graphs. IEEE Trans. Depend. Secure Comput. (2020). DOI:https://doi.org/10.1109/TDSC.2020.2980271Google Scholar
Cross Ref
- Eckart Zitzler and Simon Künzli. 2004. Indicator-based selection in multiobjective search. In Proceedings of the Conference on Parallel Problem Solving from Nature (PPSN’04). Vol. 3242. Springer, Berlin, 832--842.Google Scholar
Cross Ref
- Eckart Zitzler and Lothar Thiele. 1998. Multiobjective optimization using evolutionary algorithms—A comparative case study. In Proceedings of the Conference on Parallel Problem Solving from Nature (PPSN’98). Vol. 1498. Springer, Berlin, 292--301. DOI:https://doi.org/10.1007/BFb0056872Google Scholar
Cross Ref
- E. Zitzler and L. Thiele. Nov./1999. Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach. IEEE Trans. Evolution. Comput. 3, 4 (Nov. 1999), 257--271. DOI:https://doi.org/10.1109/4235.797969Google Scholar
Digital Library
Index Terms
Designing Strong Privacy Metrics Suites Using Evolutionary Optimization
Recommendations
Technical Privacy Metrics: A Systematic Survey
The goal of privacy metrics is to measure the degree of privacy enjoyed by users in a system and the amount of protection offered by privacy-enhancing technologies. In this way, privacy metrics contribute to improving user privacy in the digital world. ...
POSTER: Evaluating Privacy Metrics for Graph Anonymization and De-anonymization
ASIACCS '18: Proceedings of the 2018 on Asia Conference on Computer and Communications SecurityMany modern communication systems generate graph data, for example social networks and email networks. Such graph data can be used for recommender systems and data mining. However, because graph data contains sensitive information about individuals, ...
Evaluating the Strength of Genomic Privacy Metrics
The genome is a unique identifier for human individuals. The genome also contains highly sensitive information, creating a high potential for misuse of genomic data (for example, genetic discrimination). In this article, we investigate how genomic ...






Comments