Abstract
This article extends and improves the work of Fortnow and Klivans [2009], who showed that if a circuit class C has an efficient learning algorithm in Angluin’s model of exact learning via equivalence and membership queries [Angluin 1988], then we have the lower bound EXPNP not C. We use entirely different techniques involving betting games [Buhrman et al. 2001] to remove the NP oracle and improve the lower bound to EXP not C. This shows that it is even more difficult to design a learning algorithm for C than the results of Fortnow and Klivans [2009] indicated. We also investigate the connection between betting games and natural proofs, and as a corollary the existence of strong pseudorandom generators.
Our results also yield further evidence that the class of Boolean circuits has no efficient exact learning algorithm. This is because our separation is strong in that it yields a natural proof [Razborov and Rudich 1997] against the class. From this we conclude that an exact learning algorithm for Boolean circuits would imply that strong pseudorandom generators do not exist, which contradicts widely believed conjectures from cryptography. As a corollary we obtain that if strong pseudorandom generators exist, then there is no exact learning algorithm for Boolean circuits.
- Aizenstein, H., Hegeds, T., Hellerstein, L., and Pitt, L. 1998. Complexity theoretic hardness results for query learning. Comput. Complex. 7, 1, 19--53. Google Scholar
Digital Library
- Angluin, D. 1988. Queries and concept learning. Mach. Learn. 2, 4, 319--342. Google Scholar
Digital Library
- Buhrman, H. and Homer, S. 1992. Superpolynomial circuits, almost sparse oracles and the exponential hierarchy. In Proceedings of the 12th Conference on Foundations of Software Technology and Theoretical Computer Science. Springer, 116--127. Google Scholar
Digital Library
- Buhrman, H., van Melkebeek, D., Regan, K. W., Sivakumar, D., and Strauss, M. 2001. A generalization of resource-bounded measure, with application to the bpp vs. exp problem. SIAM J. Comput. 30, 2, 576--601. Google Scholar
Digital Library
- Fortnow, L. and Klivans, A. R. 2009. Efficient learning algorithms yield circuit lower bounds. J. Comput. Syst. Sci. 75, 1, 27--36. Google Scholar
Digital Library
- Hitchcock, J. M. 2007. Online learning and resource-bounded dimension: Winnow yields new lower bounds for hard sets. SIAM J. Comput. 36, 6, 1696--1708. Google Scholar
Digital Library
- Karp, R. M. and Lipton, R. J. 1980. Some connections between nonuniform and uniform complexity classes. In Proceedings of the 12th Annual ACM Symposium on Theory of Computing. 302--309. Google Scholar
Digital Library
- Kearns, M. and Valiant, L. 1994. Cryptographic limitations on learning boolean formulae and finite automata. J. ACM 41, 1, 67--95. Google Scholar
Digital Library
- Kharitonov, M. 1995. Cryptographic lower bounds for learnability of boolean functions on the uniform distribution. J. Comput. Syst. Sci. 50, 3, 600--610. Google Scholar
Digital Library
- Lindner, W., Schuler, R., and Watanabe, O. 2000. Resource-bounded measure and learnability. Theory Comput. Syst. 33, 2, 151--170.Google Scholar
Cross Ref
- Littlestone, N. 1988. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Mach. Learn. 2, 4, 285--318. Google Scholar
Digital Library
- Lutz, J. H. 1992. Almost everywhere high nonuniform complexity. J. Comput. Syst. Sci. 44, 2, 220--258. Google Scholar
Digital Library
- Lutz, J. H. 1997. The quantitative structure of exponential time. In Complexity Theory Retrospective II, L. A. Hemaspaandra and A. L. Selman Eds., Springer, 225--254. Google Scholar
Digital Library
- Merkle, W., Miller, J. S., Nies, A., Reimann, J., and Stephan, F. 2006. Kolmogorov-loveland randomness and stochasticity. Ann. Pure Appl. Logic 138, 1--3, 183--210.Google Scholar
Cross Ref
- Muchnik, A. A., Semenov, A. L., and Uspensky, V. A. 1998. Mathematical metaphysics of randomness. Theor. Comput. Sci. 207, 2, 263--317. Google Scholar
Digital Library
- Razborov, A. A. and Rudich, S. 1997. Natural proofs. J. Comput. System Sci. 55, 1, 24--35. Google Scholar
Digital Library
- Regan, K. W., Sivakumar, D., and Cai, J. 1995. Pseudorandom generators, measure theory, and natural proofs. In Proceedings of the 36th Symposium on Foundations of Computer Science. IEEE Computer Society, Los Alamitos, CA, 26--35. Google Scholar
Digital Library
- Toda, S. 1991. On the computational power of pp and _p. SIAM J. Comput. 20, 5, 865--877. Google Scholar
Digital Library
- Valiant, L. G. 1984. A Theory of the Learnable. Comm. ACM 27, 11, 1134--1142. Google Scholar
Digital Library
Index Terms
Exact Learning Algorithms, Betting Games, and Circuit Lower Bounds
Recommendations
Nondeterministic circuit lower bounds from mildly derandomizing Arthur-Merlin games
In several settings, derandomization is known to follow from circuit lower bounds that themselves are equivalent to the existence of pseudorandom generators. This leaves open the question whether derandomization implies the circuit lower bounds that are ...
Efficient learning algorithms yield circuit lower bounds
We describe a new approach for understanding the difficulty of designing efficient learning algorithms. We prove that the existence of an efficient learning algorithm for a circuit class C in Angluin's model of exact learning from membership and ...
Efficient learning algorithms yield circuit lower bounds
COLT'06: Proceedings of the 19th annual conference on Learning TheoryWe describe a new approach for understanding the difficulty of designing efficient learning algorithms. We prove that the existence of an efficient learning algorithm for a circuit class C in Angluin’s model of exact learning from membership and ...








Comments