Getting Started¶

Installation¶

Install InferPy from PyPI:

$ python -m pip install inferpy

For further details, check the Installation section.

30 seconds to InferPy¶

The core data structure of InferPy is a probabilistic model, defined as a set of random variables with a conditional independence structure. A random variable is an object parameterized by a set of tensors.

Let’s look at a simple non-linear probabilistic component analysis model (NLPCA). Graphically the model can be defined as follows,

We start by importing the required packages and defining the constant parameters in the model.

import inferpy as inf
import tensorflow as tf

# number of components
k = 1
# size of the hidden layer in the NN
d0 = 100
# dimensionality of the data
dx = 2
# number of observations (dataset size)
N = 1000

A model can be defined by decorating any function with @inf.probmodel. The model is fully specified by the variables defined inside this function:

@inf.probmodel
def nlpca(k, d0, dx, decoder):

    with inf.datamodel():
        z = inf.Normal(tf.ones([k])*0.5, 1., name="z")    # shape = [N,k]
        output = decoder(z,d0,dx)
        x_loc = output[:,:dx]
        x_scale = tf.nn.softmax(output[:,dx:])
        x = inf.Normal(x_loc, x_scale, name="x")   # shape = [N,d]

The construct with inf.datamodel(), which resembles the plateau notation, will replicate N times the variables enclosed, where N is the data size.

In the previous model, the input argument decoder must be a function implementing a neural network. This can be defined outside the model as follows.

def decoder(z,d0,dx):
    h0 = tf.keras.layers.Dense(d0, activation=tf.nn.relu)
    h1 = tf.keras.layers.Dense(2 * dx)
    return h1(h0(z))

Now, we can instantiate our model and obtain samples (from the prior distributions).

# create an instance of the model
m = nlpca(k,d0,dx, decoder)

# Sample from priors
samples = m.prior().sample()

In variational inference, we need to define a Q-model as follows.

@inf.probmodel
def qmodel(k):
    with inf.datamodel():
        qz_loc = inf.Parameter(tf.ones([k])*0.5, name="qz_loc")
        qz_scale = tf.math.softplus(inf.Parameter(tf.ones([k]),name="qz_scale"))
        qz = inf.Normal(qz_loc, qz_scale, name="z")

Afterwards, we define the parameters of the inference algorithm and fit the model to the data.

# set the inference algorithm
VI = inf.inference.VI(qmodel(k), epochs=5000)

# learn the parameters
m.fit({"x": x_train}, VI)

The inference method can be further configured. But, as in Keras, a core principle is to try to make things reasonably simple, while allowing the user the full control if needed.

Finally, we might extract the posterior of z, which is basically the hidden representation of the data.

#extract the hidden representation
hidden_encoding = m.posterior("z", data={"x":x_train})
print(hidden_encoding.sample())