Concepts inComputing sparse Hessians with automatic differentiation
Automatic differentiation
In mathematics and computer algebra, automatic differentiation (AD), sometimes alternatively called algorithmic differentiation, is a set of techniques to numerically evaluate the derivative of a function specified by a computer program. AD exploits the fact that every computer program, no matter how complicated, executes a sequence of elementary arithmetic operations (addition, subtraction, multiplication, division, etc. ) and elementary functions (exp, log, sin, cos, etc.).
more from Wikipedia
Hessian matrix
In mathematics, the Hessian matrix (or simply the Hessian) is the square matrix of second-order partial derivatives of a function; that is, it describes the local curvature of a function of many variables. The Hessian matrix was developed in the 19th century by the German mathematician Ludwig Otto Hesse and later named after him. Hesse himself had used the term "functional determinants".
more from Wikipedia
Sparse matrix
In the subfield of numerical analysis, a sparse matrix is a matrix populated primarily with zeros. The term itself was coined by Harry M. Markowitz. Conceptually, sparsity corresponds to systems which are loosely coupled. Consider a line of balls connected by springs from one to the next; this is a sparse system. By contrast, if the same line of balls had springs connecting each ball to all other balls, the system would be represented by a dense matrix.
more from Wikipedia
Hermitian adjoint
In mathematics, specifically in functional analysis, each linear operator on a Hilbert space has a corresponding adjoint operator. Adjoints of operators generalize conjugate transposes of square matrices to (possibly) infinite-dimensional situations. If one thinks of operators on a Hilbert space as "generalized complex numbers", then the adjoint of an operator plays the role of the complex conjugate of a complex number.
more from Wikipedia
Matrix multiplication
In mathematics, matrix multiplication is a binary operation that takes a pair of matrices, and produces another matrix. This term may refer to a number of different ways to multiply matrices, but most commonly refers to the matrix product. This article will use the following notational conventions. Matrices are represented by capital letters in bold, vectors in lowercase bold, and entries of vectors and matrices are italic (since they are scalars).
more from Wikipedia
Fisher information
In mathematical statistics and information theory, the Fisher information (sometimes simply called information) can be defined as the variance of the score, or as the expected value of the observed information. In Bayesian statistics, the asymptotic distribution of the posterior mode depends on the Fisher information and not on the prior. The role of the Fisher information in the asymptotic theory of maximum-likelihood estimation was emphasized by the statistician R.A.
more from Wikipedia
Analysis of algorithms
In computer science, the analysis of algorithms is the determination of the amount of resources (such as time and storage) necessary to execute them. Most algorithms are designed to work with inputs of arbitrary length. Usually the efficiency or running time of an algorithm is stated as a function relating the input length to the number of steps or storage locations (space complexity).
more from Wikipedia