Abstract
We present smooth interpretation, a method to systematically approximate numerical imperative programs by smooth mathematical functions. This approximation facilitates the use of numerical search techniques like gradient descent for program analysis and synthesis. The method extends to programs the notion of Gaussian smoothing, a popular signal-processing technique that filters out noise and discontinuities from a signal by taking its convolution with a Gaussian function.
In our setting, Gaussian smoothing executes a program according to a probabilistic semantics; the execution of program P on an input x after Gaussian smoothing can be summarized as follows: (1) Apply a Gaussian perturbation to x -- the perturbed input is a random variable following a normal distribution with mean x. (2) Compute and return the expected output of P on this perturbed input. Computing the expectation explicitly would require the execution of P on all possible inputs, but smooth interpretation bypasses this requirement by using a form of symbolic execution to approximate the effect of Gaussian smoothing on P. The result is an efficient but approximate implementation of Gaussian smoothing of programs.
Smooth interpretation has the effect of attenuating features of a program that impede numerical searches of its input space -- for example, discontinuities resulting from conditional branches are replaced by continuous transitions. We apply smooth interpretation to the problem of synthesizing values of numerical control parameters in embedded control applications. This problem is naturally formulated as one of numerical optimization: the goal is to find parameter values that minimize the error between the resulting program and a programmer-provided behavioral specification. Solving this problem by directly applying numerical optimization techniques is often impractical due to the discontinuities in the error function. By eliminating these discontinuities, smooth interpretation makes it possible to search the parameter space efficiently by means of simple gradient descent. Our experiments demonstrate the value of this strategy in synthesizing parameters for several challenging programs, including models of an automated gear shift and a PID controller.
- R. Alur, C. Courcoubetis, N. Halbwachs, T. A. Henzinger, P.-H. Ho, X. Nicollin, A. Olivero, J. Sifakis, and S. Yovine. The algorithmic analysis of hybrid systems. Theor. Comput. Sci., 138(1):3--34, 1995. Google Scholar
Digital Library
- D. Andre and S. Russell. State abstraction for programmable reinforcement learning agents. In AAAI/IAAI, pages 119--125, 2002. Google Scholar
Digital Library
- C. Bishop. Neural Networks for Pattern Recognition. 1995. Google Scholar
Digital Library
- S. Chaudhuri, S. Gulwani, and R. Lublinerman. Continuity analysis of programs. In POPL, 2010. Google Scholar
Digital Library
- P. Cousot and R. Cousot. Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In POPL, 1977. Google Scholar
Digital Library
- R. DeMillo and R. Lipton. Defining software by continuous, smooth functions. IEEE Transactions on Software Engineering, 17(4):383--384, 1991. Google Scholar
Digital Library
- A. Donzé, B. Krogh, and A. Rajhans. Parameter synthesis for hybrid systems with an application to Simulink models. In HSCC, 2009. Google Scholar
Digital Library
- B. Fischer and J. Schumann. AutoBayes: A system for generating data analysis programs from statistical models. Journal of Functional Programming, 13(03):483--508, 2003. Google Scholar
Digital Library
- G. Frehse, S. Jha, and B. Krogh. A counterexample-guided approach to parameter synthesis for linear hybrid automata. In HSCC, pages 187--200, 2008. Google Scholar
Digital Library
- S. Gulwani and G. Necula. Discovering affine equalities using random interpretation. In POPL, 2003. Google Scholar
Digital Library
- T. Henzinger and H. Wong-Toi. Using HyTech to synthesize control parameters for a steam boiler. In Formal Methods for Industrial Applications, pages 265--282, 1995. Google Scholar
Digital Library
- D. Kozen. Semantics of probabilistic programs. J. Comput. Syst. Sci., 22(3):328--350, 1981.Google Scholar
Cross Ref
- D. Monniaux. Abstract interpretation of probabilistic semantics. In SAS, pages 322--339, 2000. Google Scholar
Digital Library
- N. Nisan and M. Szegedy. On the degree of Boolean functions as real polynomials. Computational Complexity, 4(4):301--313, 1994. Google Scholar
Digital Library
- D. Parnas. Software aspects of strategic defense systems. Communications of the ACM, 28(12):1326--1335, 1985. Google Scholar
Digital Library
- A. Di Pierro and H. Wiklicky. Probabilistic abstract interpretation and statistical testing. In PAPM-PROBMIV, pages 211--212, 2002. Google Scholar
Digital Library
- J. Russ. The image processing handbook. CRC Press, 2007.Google Scholar
- M. Smith. Probabilistic abstract interpretation of imperative programs using truncated normal distributions. Electron. Notes Theor. Comput. Sci., 220(3):43--59, 2008. Google Scholar
Digital Library
- A. Solar-Lezama, R. M. Rabbah, R. Bodík, and K. Ebcioglu. Programming by sketching for bit-streaming programs. In PLDI, pages 281--294, 2005. Google Scholar
Digital Library
- A. Solar-Lezama, L. Tancau, R. Bodik, V. Saraswat, and S. Seshia. Combinatorial sketching for finite programs. In ASPLOS '06, 2006. Google Scholar
Digital Library
- G. N. Vanderplaats. Numerical optimization techniques. Springer-Verlag, 1987.Google Scholar
Cross Ref
Index Terms
Smooth interpretation
Recommendations
Smooth interpretation
PLDI '10: Proceedings of the 31st ACM SIGPLAN Conference on Programming Language Design and ImplementationWe present smooth interpretation, a method to systematically approximate numerical imperative programs by smooth mathematical functions. This approximation facilitates the use of numerical search techniques like gradient descent for program analysis and ...
Bridging boolean and quantitative synthesis using smoothed proof search
POPL '14We present a new technique for parameter synthesis under boolean and quantitative objectives. The input to the technique is a "sketch" --- a program with missing numerical parameters --- and a probabilistic assumption about the program's inputs. The ...
Bridging boolean and quantitative synthesis using smoothed proof search
POPL '14: Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming LanguagesWe present a new technique for parameter synthesis under boolean and quantitative objectives. The input to the technique is a "sketch" --- a program with missing numerical parameters --- and a probabilistic assumption about the program's inputs. The ...







Comments