maxent_toolbox: Maximum Entropy Toolbox for MATLAB

Introduction

Maximum entropy toolbox for MATLAB is a free, open-source toolbox for finding the maximum entropy distribution of training data, based on a set of constraints or observables over the data.

Maximum entropy models give the mathematically minimal probabilistic models of the states or configurations of a systems, given the mean values of some set of observed functions. Since the entropy of a distribution measures the randomness or lack of interaction among different variables, the minimally structured distribution given a set of observables is the distribution with the maximal entropy that is consistent with these observables. These models are used approximate probability distributions of discrete states from limited data, where standard approaches (such as counting the states) cannot be applied.

This toolbox is intended for learning probability distributions of binary activity patterns, i.e. patterns in the form 1000110100. Examples of problems where one might want to learn such distributions are characterising the joint activity of neural populations (0/1 denoting if a certain neuron fired within a small time window) or gene expression profiles (0/1 denoting if a certain gene is being expressed). The toolbox takes as an input a set of samples of activity patterns and learns a model of the probability over all states, thus extrapolating to the entire distribution over all possible activity patterns.

Mathematically, in its discrete form, if x_i are the elements of the system (here variables taking discrete values), then the maximum entropy model for p(x₁,x₂…x_n) which is consistent with the average values of a set of functions f₁(x)...f_k(x) has a unique solution in the form:

$\hat{p}(x)=\exp[\sum_{i=1}^{k}\lambda_i{f_{i}(\vec{x})}])$

where λ₁...λ_k are the model parameters (or Lagrange multipliers) which can be found numerically.

The user can choose between several variants of maximum entropy models, each relying on a different set of observables or constraints. The maximum entropy models currently supported by this package are

Independent model, which uses <x_i>_data as constraints
Pairwise maximum entropy model: <x_i>_data and <x_ix_j>_data
K-synchrony model: <Σ_ix_i>_data
K-Pairwise model
Arbitrary set of high-order correlations
RP (Random Projection) model
Any combination of the above models

The toolbox automatically switches between exhaustive solutions for small (<30) groups of variables and Markov Chain Monte Carlo (MCMC) methods for larger groups and can be used to learn distributions of up to several hundreds of binary variables. The software is provided as an installable toolbox for MATLAB, and most of the code is written in heavily optimized C++ precompiled for Windows (64 bit), OS X and Linux (CentOS).

Download

Download latest version packaged as an installable MATLAB toolbox with mex files precompiled for Windows 64bit, MacOS and Linux (CentOS 64bit).
Download most recent source code.

Install

Click on the downloaded .mltbx file to install it as a toolbox for MATLAB. To uninstall, open MATLAB and navigate to Add-Ons -> Manage Add-Ons.

Documentation

If this is the first time you are using the toolbox, start by reading the quick start guide or take a look at some example code (which you can also run by typing "maxent_example" after installing the toolbox).

Experienced users can refer to the function reference for details about each of the available functions in the toolbox.

You can also check out answers to some frequently asked questions.

Build

Build instructions for Windows, MacOS and Unix. Note that the toolbox installer already contains precompiled binaries, so building the toolbox is only necessary if you use a nonstandard operating system or want to make changes in the code.

Cite

If you use this toolbox as part of a published academic work, please cite it as:

Ori Maoz and Elad Schneidman. maxent_toolbox: Maximum Entropy Toolbox for MATLAB, version 1.0.2. 2017. doi: 10.5281/zenodo.191625. URL https://orimaoz.github.io/maxent_toolbox.

The corresponding BibTeX citation:

@misc{Maoz2017_191625,
author = {Maoz, Ori and Schneidman, Elad},
title = {maxent{\_}toolbox: Maximum Entropy Toolbox for MATLAB, version 1.0.2},
year = {2017},
version = {1.02},
publisher = {Zenodo},
doi = {10.5281/zenodo.191625},
url = {https://orimaoz.github.io/maxent_toolbox}
}

License

The Maximum Entropy Toolbox for MATLAB is free and open source, distributed under the MIT license.

Authors and Contributors

Ori Maoz, orimaoz@gmail.com