Abstract
We present Parallely, a programming language and a system for verification of approximations in parallel message-passing programs. Parallely's language can express various software and hardware level approximations that reduce the computation and communication overheads at the cost of result accuracy.
Parallely's safety analysis can prove the absence of deadlocks in approximate computations and its type system can ensure that approximate values do not interfere with precise values. Parallely's quantitative accuracy analysis can reason about the frequency and magnitude of error. To support such analyses, Parallely presents an approximation-aware version of canonical sequentialization, a recently proposed verification technique that generates sequential programs that capture the semantics of well-structured parallel programs (i.e., ones that satisfy a symmetric nondeterminism property). To the best of our knowledge, Parallely is the first system designed to analyze parallel approximate programs.
We demonstrate the effectiveness of Parallely on eight benchmark applications from the domains of graph analytics, image processing, and numerical analysis. We also encode and study five approximation mechanisms from literature. Our implementation of Parallely automatically and efficiently proves type safety, reliability, and accuracy properties of the approximate benchmarks.
Supplemental Material
- R. J. Lipton. 1975. Reduction: A Method of Proving Properties of Parallel Programs. Commun. ACM 18 (1975).Google Scholar
- J. L. Peterson. 1977. Petri nets. ACM Computing Surveys (CSUR) 3 (1977).Google Scholar
- G. Agha and C. Hewitt. 1985. Concurrent Programming Using Actors: Exploiting Large-Scale Parallelism. Technical Report. Cambridge, MA, USA.Google Scholar
- G. Agha. 1986. An Overview of Actor Languages. In OOPWORK.Google Scholar
- P. Godefroid. 1996. Partial-Order Methods for the Verification of Concurrent Systems: An Approach to the State-Explosion Problem. Springer-Verlag.Google Scholar
Digital Library
- G. Smith and D. Volpano. 1998. Secure Information Flow in a Multi-threaded Imperative Language. In POPL.Google Scholar
- F. Huch. 1999. Verification of Erlang programs using abstract interpretation and model checking.Google Scholar
- R. Milner. 1999. Communicating and mobile systems: the pi calculus. Cambridge university press.Google Scholar
Digital Library
- L. Page, S. Brin, R. Motwani, and T. Winograd. 1999. The PageRank citation ranking: Bringing order to the web. Technical Report.Google Scholar
- A. Gaffar, O. Mencer, W. Luk, P. Cheung, and N. Shirazi. 2002. Floating-point bitwidth analysis via automatic differentiation. In FPT.Google Scholar
- J. Dean and S. Ghemawat. 2004. MapReduce: Simplified data processing on large clusters. OSDI (2004).Google Scholar
- C. Flanagan and P. Godefroid. 2005. Dynamic Partial-order Reduction for Model Checking Software. In POPL.Google Scholar
- S. F. Siegel. 2005. Efficient Verification of Halting Properties for MPI Programs with Wildcard Receives. In VMCAI.Google Scholar
- S. F. Siegel and G. S. Avrunin. 2005. Modeling Wildcard-free MPI Programs for Verification. In PPoPP.Google Scholar
- M. Rinard. 2006. Probabilistic accuracy bounds for fault-tolerant computations that discard tasks. In ICS.Google Scholar
- L.-Å. Fredlund and H. Svensson. 2007. McErlang: a model checker for a distributed functional programming language. In ICFP.Google Scholar
- W. Osborne, R. Cheung, J. Coutinho, W. Luk, and O. Mencer. 2007. Automatic accuracy-guaranteed bit-width optimization for fixed and floating-point systems. In FPL.Google Scholar
- M. Rinard. 2007. Using Early Phase Termination to Eliminate Load Imbalances at Barrier Synchronization Points. In OOPSLA.Google Scholar
- A. Lal and T. Reps. 2008. Reducing concurrent analysis under a context bound to sequential analysis. In International Conference on Computer Aided Verification. 37–51.Google Scholar
- S. Chakradhar, A. Raghunathan, and J. Meng. 2009. Best-Effort Parallel Execution Framework for Recognition and Mining Applications. In IPDPS.Google Scholar
- S. La Torre, P. Madhusudan, and G. Parlato. 2009. Reducing context-bounded concurrent reachability to sequential reachability. In International Conference on Computer Aided Verification. 477–492.Google Scholar
- W. Baek and T. M. Chilimbi. 2010. Green: A Framework for Supporting Energy-Conscious Programming using Controlled Approximation. In PLDI.Google Scholar
Digital Library
- S. Misailovic, S. Sidiroglou, H. Hoffmann, and M. Rinard. 2010. Quality of service profiling. In ICSE.Google Scholar
- J. Ansel, Y. Wong, C. Chan, M. Olszewski, A. Edelman, and S. Amarasinghe. 2011. Language and compiler support for auto-tuning variable-accuracy algorithms. In CGO.Google Scholar
- C. Bienia. 2011. Benchmarking modern multiprocessors.Google Scholar
- S. Chaudhuri, S. Gulwani, R. Lublinerman, and S. Navidpour. 2011. Proving Programs Robust. In ESEC/FSE.Google Scholar
- H. Hoffmann, S. Sidiroglou, M. Carbin, S. Misailovic, A. Agarwal, and M. Rinard. 2011. Dynamic Knobs for Responsive Power-Aware Computing. In ASPLOS.Google Scholar
- S. Liu, K. Pattabiraman, T. Moscibroda, and B. G. Zorn. 2011. Flikker: saving DRAM refresh-power through critical data partitioning. (2011).Google Scholar
- S. Misailovic, D. Roy, and M. Rinard. 2011. Probabilistically Accurate Program Transformations. In SAS.Google Scholar
- B. Recht, C. Re, S. Wright, and F. Niu. 2011. Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In Advances in neural information processing systems.Google Scholar
- A. Sampson, W. Dietl, E. Fortuna, D. Gnanapragasam, L. Ceze, and D. Grossman. 2011. EnerJ: Approximate data types for safe and general low-power computation. In PLDI.Google Scholar
- S. Sidiroglou, S. Misailovic, H. Hoffmann, and M. Rinard. 2011. Managing Performance vs. Accuracy Trade-offs With Loop Perforation. In FSE.Google Scholar
- S. F. Siegel and G. Gopalakrishnan. 2011. Formal Analysis of Message Passing. In VMCAI.Google Scholar
- A. Udupa, K. Rajan, and W. Thies. 2011. ALTER: Exploiting Breakable Dependences for Parallelization. In PLDI.Google Scholar
Digital Library
- M. Carbin, D. Kim, S. Misailovic, and M. Rinard. 2012. Proving Acceptability Properties of Relaxed Nondeterministic Approximate Programs. In PLDI.Google Scholar
- L. Renganarayana, V. Srinivasan, R. Nair, and D. Prener. 2012. Programming with relaxed synchronization. In Relax Workshop.Google Scholar
- Z. Zhu, S. Misailovic, J. Kelner, and M. Rinard. 2012. Randomized Accuracy-Aware Program Transformations for Efficient Approximate Computations. In POPL.Google Scholar
- M. Carbin, D. Kim, S. Misailovic, and M. Rinard. 2013a. Verified integrity properties for safe approximate program transformations. In PEPM.Google Scholar
- M. Carbin, S. Misailovic, and M. C. Rinard. 2013b. Verifying Quantitative Reliability for Programs That Execute on Unreliable Hardware. In OOPSLA.Google Scholar
- E. D’Osualdo, J. Kochems, and C.-H. L. Ong. 2013. Automatic verification of Erlang-style concurrency. In International Static Analysis Symposium.Google Scholar
Cross Ref
- S. Misailovic, D. Kim, and M. Rinard. 2013. Parallelizing Sequential Programs With Statistical Accuracy Tests. ACM TECS Special Issue on Probabilistic Embedded Computing (2013).Google Scholar
- C. Rubio-González, C. Nguyen, H. Nguyen, J. Demmel, W. Kahan, K. Sen, D. Bailey, C. Iancu, and D. Hough. 2013. Precimonious: Tuning assistant for floating-point precision. In SC.Google Scholar
Digital Library
- P. Abdulla, S. Aronis, B. Jonsson, and K. Sagonas. 2014. Optimal Dynamic Partial Order Reduction. In POPL.Google Scholar
- J. Ansel, S. Kamil, K. Veeramachaneni, J. Ragan-Kelley, J. Bosboom, U. M. O’Reilly, and S. Amarasinghe. 2014. Opentuner: An extensible framework for program autotuning. In PACT.Google Scholar
Digital Library
- A. Desai, P. Garg, and P. Madhusudan. 2014. Natural Proofs for Asynchronous Programs Using Almost-synchronous Reductions. In OOPSLA.Google Scholar
- S. Misailovic, M. Carbin, S. Achour, Z. Qi, and M. C. Rinard. 2014. Chisel: Reliability- and Accuracy-aware Optimization of Approximate Computational Kernels. In OOPSLA.Google Scholar
Digital Library
- M. Samadi, D. A. Jamshidi, J. Lee, and S. Mahlke. 2014. Paraprox: Pattern-based Approximation for Data Parallel Applications. In ASPLOS.Google Scholar
Digital Library
- E. Schkufza, R. Sharma, and A. Aiken. 2014. Stochastic optimization of floating-point programs with tunable precision. In PLDI.Google Scholar
- S. Achour and M. Rinard. 2015. Energy Efficient Approximate Computation with Topaz. In OOPSLA.Google Scholar
- M. Ahmad, F. Hijaz, Q. Shi, and O. Khan. 2015. CRONO: A Benchmark Suite for Multithreaded Graph Algorithms Executing on Futuristic Multicores. In IISWC.Google Scholar
- S. Blom, S. Darabi, and M. Huisman. 2015. Verification of loop parallelisations. In International Conference on Fundamental Approaches to Software Engineering. 202–217.Google Scholar
- B. Boston, A. Sampson, D. Grossman, and L. Ceze. 2015. Probability type inference for flexible approximate programming. (2015).Google Scholar
- S. Campanoni, G. Holloway, G.-Y. Wei, and D. Brooks. 2015. HELIX-UP: Relaxing program semantics to unleash parallelization. In CGO.Google Scholar
- Y. Ding, J. Ansel, K. Veeramachaneni, X. Shen, U. M. O’Reilly, and S. Amarasinghe. 2015. Autotuning Algorithmic Choice for Input Sensitivity. In PLDI.Google Scholar
- I. Goiri, R. Bianchini, S. Nagarakatte, and T. Nguyen. 2015. ApproxHadoop: Bringing Approximations to MapReduce Frameworks. In ASPLOS.Google Scholar
- A. Sampson, A. Baixo, B. Ransford, T. Moreau, J. Yip, L. Ceze, and M. Oskin. 2015. ACCEPT: A Programmer-Guided Compiler Framework for Practical Approximate Computing. Technical Report.Google Scholar
- M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, and X. Zheng. 2016. TensorFlow: A System for Large-Scale Machine Learning. In OSDI.Google Scholar
- R. Akram, M. M. U. Alam, and A. Muzahid. 2016. Approximate Lock: Trading off Accuracy for Performance by Skipping Critical Sections. In ISSRE.Google Scholar
- A. Bakst, K. v. Gleissenthall, R. G. Kici, and R. Jhala. 2017. Verifying Distributed Programs via Canonical Sequentialization. In OOPSLA.Google Scholar
- R. Boyapati, J. Huang, P. Majumder, K. H. Yum, and E. J. Kim. 2017. APPROX-NoC: A Data Approximation Framework for Network-On-Chip Architectures. In ISCA.Google Scholar
- A. Canino and Y. D. Liu. 2017. Proactive and Adaptive Energy-aware Programming with Mixed Typechecking. In PLDI.Google Scholar
- W.-F. Chiang, M. Baranowski, I. Briggs, A. Solovyev, G. Gopalakrishnan, and Z. Rakamarić. 2017. Rigorous floating-point mixed-precision tuning. In POPL.Google Scholar
- M. Huisman. 2017. A Verification Technique for Deterministic Parallel Programs. In PPDP.Google Scholar
- V. Magron, G. Constantinides, and A. Donaldson. 2017. Certified Roundoff Error Bounds Using Semidefinite Programming. ACM Trans. Math. Software 43, 4 (Jan. 2017).Google Scholar
Digital Library
- S. Mitra, M. K. Gupta, S. Misailovic, and S. Bagchi. 2017. Phase-aware optimization in approximate computing. In CGO.Google Scholar
- B. Nongpoh, R. Ray, S. Dutta, and A. Banerjee. 2017. AutoSense: A Framework for Automated Sensitivity Analysis of Program Data. IEEE Transactions on Software Engineering 43 (2017).Google Scholar
- A. Yazdanbakhsh, D. Mahajan, H. Esmaeilzadeh, and P. Lotfi-Kamran. 2017. AxBench: A Multiplatform Benchmark Suite for Approximate Computing. IEEE Design Test 34, 2 (April 2017).Google Scholar
Cross Ref
- F. Betzel, K. Khatamifard, H. Suresh, D. J. Lilja, J. Sartori, and U. Karpuzcu. 2018. Approximate Communication: Techniques for Reducing Communication Bottlenecks in Large-Scale Parallel Systems. ACM Computing Surveys (CSUR) 51 (2018).Google Scholar
Digital Library
- B. Boston, Z. Gong, and M. Carbin. 2018. Leto: verifying application-specific hardware fault tolerance with programmable execution models. In OOPSLA.Google Scholar
- E. Darulova, A. Izycheva, F. Nasir, F. Ritter, H. Becker, and R. Bastian. 2018. Daisy-Framework for Analysis and Optimization of Numerical Programs (Tool Paper). In TACAS.Google Scholar
- E. A. Deiana, V. St-Amour, P. A. Dinda, N. Hardavellas, and S. Campanoni. 2018. Unconventional Parallelization of Nondeterministic Applications. In ASPLOS.Google Scholar
- S. He, S. K. Lahiri, and Z. Rakamarić. 2018. Verifying relative safety, accuracy, and termination for program approximations. Journal of Automated Reasoning 60, 1 (2018).Google Scholar
Digital Library
- S. K. Khatamifard, I. Akturk, and U. R. Karpuzcu. 2018. On Approximate Speculative Lock Elision. IEEE Transactions on Multi-Scale Computing Systems 2 (2018).Google Scholar
- J. Lidman and S. A. Mckee. 2018. Verifying Reliability Properties Using the Hyperball Abstract Domain. ACM Transactions on Programming Languages and Systems (TOPLAS) 40, 1 (2018), 3.Google Scholar
Digital Library
- P. Stanley-Marbell and M. Rinard. 2018. Perceived-Color Approximation Transforms for Programs that Draw. IEEE Micro 38, 4 (2018), 20–29.Google Scholar
Digital Library
- J. R. Stevens, A. Ranjan, and A. Raghunathan. 2018. AxBA: an approximate bus architecture framework. In ICCAD.Google Scholar
- R. Xu, J. Koo, R. Kumar, P. Bai, S. Mitra, S. Misailovic, and S. Bagchi. 2018. Videochef: efficient approximation for streaming video processing pipelines. In USENIX ATC.Google Scholar
- V. Fernando, A. Franques, S. Abadal, S. Misailovic, and J. Torrellas. 2019a. Replica: A Wireless Manycore for CommunicationIntensive and Approximate Data. In ASPLOS.Google Scholar
Digital Library
- V. Fernando, K. Joshi, D. Marinov, and S. Misailovic. 2019c. Identifying Optimal Parameters for Randomized Approximate Algorithms. In Workshop on Approximate Computing Across the Stack.Google Scholar
- V. Fernando, K. Joshi, and S. Misailovic. 2019b. Appendix to Parallely https://vimuth.github.io/parallely/appendix.pdf.Google Scholar
- K. Gleissenthall, R. G. Kici, A. Bakst, D. Stefan, and R. Jhala. 2019. Pretend Synchrony. In POPL.Google Scholar
- K. Joshi, V. Fernando, and S. Misailovic. 2019. Statistical Algorithmic Profiling for Randomized Approximate Programs. In ICSE.Google Scholar
- E. Michael, D. Woos, T. Anderson, M. D. Ernst, and Z. Tatlock. 2019. Teaching Rigorous Distributed Systems With Efficient Model Checking. In EuroSys.Google Scholar
Index Terms
Verifying safety and accuracy of approximate parallel programs via canonical sequentialization
Recommendations
Verifying Relative Safety, Accuracy, and Termination for Program Approximations
Approximate computing is an emerging area for trading off the accuracy of an application for improved performance, lower energy costs, and tolerance to unreliable hardware. However, developers must ensure that the leveraged approximations do not ...
Verifying parallel programs with MPI-spin
PVM/MPI'07: Proceedings of the 14th European conference on Recent Advances in Parallel Virtual Machine and Message Passing InterfaceStandard testing and debugging techniques are notoriously ineffective when applied to parallel programs, due to the numerous sources of nondeterminism arising from parallelism. MPI-Spin, an extension of the model checker Spin for verifying and debugging ...
Formal Sequentialization of Distributed Systems via Program Rewriting
Formal sequentialization is introduced as a rewriting process for the reduction of parallelism and internal communication statements of distributed imperative programs. It constructs an equivalence proof in an implicit way, via the application of ...






Comments