Edit on GitHub

Exponential Family Distributions

Exponential Family

All the distributions provided by this package are members of the exponential family of distribution, i.e. they have the follotwing canonical form:

\[p(x) = \exp \{ \eta^\top T(x) - A(\eta) + B(x) \}\]

where:

  • $\eta$ is the natural parameter (scalar or vector)
  • $T(x)$ is the sufficient statistic (scalar or vector)
  • $A(\eta)$ is the log-normalizer (scalar)
  • $B(x)$ is the base measure (scalar)

Practically, the package provide the following abstract type

Which represents the supter-type of members of the exponential family.

Parameterization

All subtypes of Distribution have the following form:

struct MyDistribution{P<:AbstractParam} <: Distribution
    param::P
end

This particular form allows each distribution to be agnostic to their concrete parameterization. The parameter type inherits from:

and supports the following methods

ExpFamilyDistributions.realform — Function
realform(param)

Returns the vector of parameters as stored in param. Note that this function is just an accessor of the internal storage of the parameter, modifying the returned value should modify the parameter accordingly.

See also: naturalform.

source

Distribution interface

Each subtype of [Distribution] implements the following interface:

ExpFamilyDistributions.kldiv — Function
kldiv(q::T, p::T[, μ = gradlognorm(q)]) where T<:Distribution

Compute the KL-divergence between two distributions of the same type (i.e. kldiv(Normal, Normal), kldiv(Dirichlet, Dirichlet), ...). You can specify directly the expectation of the sufficient statistics μ.

source
ExpFamilyDistributions.splitgrad — Function
splitgrad(p, μ)

Split the gradient of the log-normalizer into its "standard" components. For instance, for the Normal distribution, the output will be the expected value of $x$ and $xxᵀ$.

source

Utilities

The package also provides the following utility functions: