Concepts inComputing sparse Hessians with automatic differentiation
Matrix multiplication
In mathematics, matrix multiplication is a binary operation that takes a pair of matrices, and produces another matrix. This term may refer to a number of different ways to multiply matrices, but most commonly refers to the matrix product. This article will use the following notational conventions. Matrices are represented by capital letters in bold, vectors in lowercase bold, and entries of vectors and matrices are italic (since they are scalars).
more from Wikipedia
Hessian matrix
In mathematics, the Hessian matrix (or simply the Hessian) is the square matrix of second-order partial derivatives of a function; that is, it describes the local curvature of a function of many variables. The Hessian matrix was developed in the 19th century by the German mathematician Ludwig Otto Hesse and later named after him. Hesse himself had used the term "functional determinants".
more from Wikipedia
Automatic differentiation
In mathematics and computer algebra, automatic differentiation (AD), sometimes alternatively called algorithmic differentiation, is a set of techniques to numerically evaluate the derivative of a function specified by a computer program. AD exploits the fact that every computer program, no matter how complicated, executes a sequence of elementary arithmetic operations (addition, subtraction, multiplication, division, etc. ) and elementary functions (exp, log, sin, cos, etc.).
more from Wikipedia
Matrix (mathematics)
In mathematics, a matrix (plural matrices, or less commonly matrixes) is a rectangular array of numbers, symbols, or expressions. The individual items in a matrix are called its elements or entries. An example of a matrix with six elements is Matrices of the same size can be added or subtracted element by element. The rule for matrix multiplication is more complicated, and two matrices can be multiplied only when the number of columns in the first equals the number of rows in the second.
more from Wikipedia
Analysis of algorithms
In computer science, the analysis of algorithms is the determination of the amount of resources (such as time and storage) necessary to execute them. Most algorithms are designed to work with inputs of arbitrary length. Usually the efficiency or running time of an algorithm is stated as a function relating the input length to the number of steps or storage locations (space complexity).
more from Wikipedia
Hermitian adjoint
In mathematics, specifically in functional analysis, each linear operator on a Hilbert space has a corresponding adjoint operator. Adjoints of operators generalize conjugate transposes of square matrices to (possibly) infinite-dimensional situations. If one thinks of operators on a Hilbert space as "generalized complex numbers", then the adjoint of an operator plays the role of the complex conjugate of a complex number.
more from Wikipedia
Fisher information
In mathematical statistics and information theory, the Fisher information (sometimes simply called information) can be defined as the variance of the score, or as the expected value of the observed information. In Bayesian statistics, the asymptotic distribution of the posterior mode depends on the Fisher information and not on the prior. The role of the Fisher information in the asymptotic theory of maximum-likelihood estimation was emphasized by the statistician R.A.
more from Wikipedia
Algorithm
In mathematics and computer science, an algorithm Listen/ˈælɡərɪðəm/ (originating from al-Khwārizmī, the famous mathematician Muḥammad ibn Mūsā al-Khwārizmī) is a step-by-step procedure for calculations. Algorithms are used for calculation, data processing, and automated reasoning. More precisely, an algorithm is an effective method expressed as a finite list of well-defined instructions for calculating a function.
more from Wikipedia