Concepts inStochastic search using the natural gradient
Probability distribution
In probability theory, a probability mass, probability density, or probability distribution is a function that describes the probability of a random variable taking certain values. For a more precise definition one needs to distinguish between discrete and continuous random variables. In the discrete case, one can easily assign a probability to each possible value: when throwing a die, each of the six values 1 to 6 has the probability 1/6.
more from Wikipedia
Parameter
Parameter (from Ancient Greek παρά also “para” meaning “beside, subsidiary” and μέτρον also “metron” meaning “measure”) can be interpreted in mathematics, logic, linguistics, environmental science and other disciplines. In its common meaning, the term is used to identify a characteristic, a feature, a measurable factor that can help in defining a particular system.
more from Wikipedia
Natural evolution strategy
Natural evolution strategies (NES) are a family of numerical optimization algorithms for black-box problems. Similar in spirit to evolution strategies, they iteratively update the (continuous) parameters of a search distribution by following the natural gradient towards higher expected fitness.
more from Wikipedia
Approximation
An approximation is a representation of something that is not exact, but still close enough to be useful. Although approximation is most often applied to numbers, it is also frequently applied to such things as mathematical functions, shapes, and physical laws. Approximations may be used because incomplete information prevents use of exact representations. Many problems in physics are either too complex to solve analytically, or impossible to solve using the available analytical tools.
more from Wikipedia
Information geometry
Information geometry is a branch of mathematics that applies the techniques of differential geometry to the field of probability theory. Probability distributions for a statistical model form the points of a Riemannian manifold. Fisher information is used as the Riemannian metric. Information geometry reached maturity through the work of Shun'ichi Amari and other Japanese mathematicians in the 1980s.
more from Wikipedia
Fisher information
In mathematical statistics and information theory, the Fisher information (sometimes simply called information) can be defined as the variance of the score, or as the expected value of the observed information. In Bayesian statistics, the asymptotic distribution of the posterior mode depends on the Fisher information and not on the prior. The role of the Fisher information in the asymptotic theory of maximum-likelihood estimation was emphasized by the statistician R.A.
more from Wikipedia
Stochastic optimization
Stochastic optimization (SO) methods are optimization methods that generate and use random variables. For stochastic problems, the random variables appear in the formulation of the optimization problem itself, which involve random objective functions or random constraints, for example. Stochastic optimization methods also include methods with random iterates.
more from Wikipedia
Algorithm
In mathematics and computer science, an algorithm Listen/ˈælɡərɪðəm/ (originating from al-Khwārizmī, the famous mathematician Muḥammad ibn Mūsā al-Khwārizmī) is a step-by-step procedure for calculations. Algorithms are used for calculation, data processing, and automated reasoning. More precisely, an algorithm is an effective method expressed as a finite list of well-defined instructions for calculating a function.
more from Wikipedia