Visualizing statistical models in Minkowski space: an analytical coordinate embedding

Dimensionality reduction techniques are often used to provide a lower dimensional description of high dimensional data. Quinn et. al [1] proposed an intensive isometric embedding, InPCA in visualizing probabilistic models manifold with the Bhattacharyya distance. It was observed that the InPCA manifolds form a hierarchy of cross-sectional spans that shrinks geometrically in Minkowski space, allowing the use of only a few principal components to capture most of the variation. Here, we show that for a large class of multiparameter models that takes the form of exponential families, a different intensive embedding- the isKL embedding, built on symmetrized Kullback Liebler divergence generates an explicitly and analytically tractable embedding in a Minkowski space of dimension equal to twice the number of parameters. In principle, this technique not only offers a great dimensionality reduction, it also allows one to uncover a hidden exponential family that describes an experiment or a simulation if the isKL embedding gives a cutoff after N+N dimensions. We will discuss the optimization of isKL embedding in producing a good visualization with several statistical models.

[1] Quinn, et al. PNAS (2019):201817218.