skip to main content
research-article

Exact Learning Algorithms, Betting Games, and Circuit Lower Bounds

Published:01 November 2013Publication History
Skip Abstract Section

Abstract

This article extends and improves the work of Fortnow and Klivans [2009], who showed that if a circuit class C has an efficient learning algorithm in Angluin’s model of exact learning via equivalence and membership queries [Angluin 1988], then we have the lower bound EXPNP not C. We use entirely different techniques involving betting games [Buhrman et al. 2001] to remove the NP oracle and improve the lower bound to EXP not C. This shows that it is even more difficult to design a learning algorithm for C than the results of Fortnow and Klivans [2009] indicated. We also investigate the connection between betting games and natural proofs, and as a corollary the existence of strong pseudorandom generators.

Our results also yield further evidence that the class of Boolean circuits has no efficient exact learning algorithm. This is because our separation is strong in that it yields a natural proof [Razborov and Rudich 1997] against the class. From this we conclude that an exact learning algorithm for Boolean circuits would imply that strong pseudorandom generators do not exist, which contradicts widely believed conjectures from cryptography. As a corollary we obtain that if strong pseudorandom generators exist, then there is no exact learning algorithm for Boolean circuits.

References

  1. Aizenstein, H., Hegeds, T., Hellerstein, L., and Pitt, L. 1998. Complexity theoretic hardness results for query learning. Comput. Complex. 7, 1, 19--53. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Angluin, D. 1988. Queries and concept learning. Mach. Learn. 2, 4, 319--342. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Buhrman, H. and Homer, S. 1992. Superpolynomial circuits, almost sparse oracles and the exponential hierarchy. In Proceedings of the 12th Conference on Foundations of Software Technology and Theoretical Computer Science. Springer, 116--127. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Buhrman, H., van Melkebeek, D., Regan, K. W., Sivakumar, D., and Strauss, M. 2001. A generalization of resource-bounded measure, with application to the bpp vs. exp problem. SIAM J. Comput. 30, 2, 576--601. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Fortnow, L. and Klivans, A. R. 2009. Efficient learning algorithms yield circuit lower bounds. J. Comput. Syst. Sci. 75, 1, 27--36. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Hitchcock, J. M. 2007. Online learning and resource-bounded dimension: Winnow yields new lower bounds for hard sets. SIAM J. Comput. 36, 6, 1696--1708. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Karp, R. M. and Lipton, R. J. 1980. Some connections between nonuniform and uniform complexity classes. In Proceedings of the 12th Annual ACM Symposium on Theory of Computing. 302--309. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Kearns, M. and Valiant, L. 1994. Cryptographic limitations on learning boolean formulae and finite automata. J. ACM 41, 1, 67--95. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Kharitonov, M. 1995. Cryptographic lower bounds for learnability of boolean functions on the uniform distribution. J. Comput. Syst. Sci. 50, 3, 600--610. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Lindner, W., Schuler, R., and Watanabe, O. 2000. Resource-bounded measure and learnability. Theory Comput. Syst. 33, 2, 151--170.Google ScholarGoogle ScholarCross RefCross Ref
  11. Littlestone, N. 1988. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Mach. Learn. 2, 4, 285--318. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Lutz, J. H. 1992. Almost everywhere high nonuniform complexity. J. Comput. Syst. Sci. 44, 2, 220--258. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Lutz, J. H. 1997. The quantitative structure of exponential time. In Complexity Theory Retrospective II, L. A. Hemaspaandra and A. L. Selman Eds., Springer, 225--254. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Merkle, W., Miller, J. S., Nies, A., Reimann, J., and Stephan, F. 2006. Kolmogorov-loveland randomness and stochasticity. Ann. Pure Appl. Logic 138, 1--3, 183--210.Google ScholarGoogle ScholarCross RefCross Ref
  15. Muchnik, A. A., Semenov, A. L., and Uspensky, V. A. 1998. Mathematical metaphysics of randomness. Theor. Comput. Sci. 207, 2, 263--317. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Razborov, A. A. and Rudich, S. 1997. Natural proofs. J. Comput. System Sci. 55, 1, 24--35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Regan, K. W., Sivakumar, D., and Cai, J. 1995. Pseudorandom generators, measure theory, and natural proofs. In Proceedings of the 36th Symposium on Foundations of Computer Science. IEEE Computer Society, Los Alamitos, CA, 26--35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Toda, S. 1991. On the computational power of pp and _p. SIAM J. Comput. 20, 5, 865--877. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Valiant, L. G. 1984. A Theory of the Learnable. Comm. ACM 27, 11, 1134--1142. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Exact Learning Algorithms, Betting Games, and Circuit Lower Bounds

    Recommendations

    Reviews

    Marius Zimand

    It is natural to expect that the more complex a concept is, the harder it is to learn. This paper improves this type of correlation for concepts that are represented by a class of circuits . It shows that if is learnable by an efficient algorithm that makes membership and equivalence queries, then there exists a set in EXP that cannot be computed by a circuit of type . Previously, this was known for the larger complexity class EXP . We recall that a membership query for learning an unknown circuit C is of the type "What is the value of C on x __?__" An equivalence query is of the type "Is C =d__?__" (to which the teacher says YES or gives an input x on which C and d differ). The proof given in this paper uses efficient martingales of a certain type. The authors show that a learning algorithm for C can be transformed into a martingale that succeeds on C . On the other hand, it is known that there exist sets in EXP on which no martingale (of the type considered in this paper) succeeds. Online Computing Reviews Service

    Access critical reviews of Computing literature here

    Become a reviewer for Computing Reviews.

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Computation Theory
      ACM Transactions on Computation Theory  Volume 5, Issue 4
      November 2013
      103 pages
      ISSN:1942-3454
      EISSN:1942-3462
      DOI:10.1145/2539126
      Issue’s Table of Contents

      Copyright © 2013 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 1 November 2013
      • Revised: 1 August 2013
      • Accepted: 1 August 2013
      • Received: 1 February 2013
      Published in toct Volume 5, Issue 4

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!