CGVB

Data Type: Double

The momentum weight $\beta_1$ associated with the $\bar{g}$ component to update variational parameters in each VB iteration in Algorithm 7,

Must be a number between $0$ and $1$.

Default: 0.9

Example: 'GradWeight1',0.95

Data Type: Double

The momentum weight $\beta_2$ associated with the $\bar{v}$ component to update variational parameters in each VB iteration in Algorithm 7.

Must be a number between $0$ and $1$.

Default: 0.9

Example: 'GradWeight2',0.95

Data Type: Double | Positive

The maximum value of $\bar{g}$ in Algorithm 7 to prevent the exploding gradient problem that occurs when the gradient gets too large, thus making the optimization for the model parameters (e.g., using gradient descent) highly unstable.

Default: 100

Example: 'GradientMax',10

Data Type: Column vector

The column vector of initial values of variational parameters. For example, we can use the point estimation of model parameters from MLE to initialize the VB techniques.

Default: None

Example: 'InitValue',zeros(D,1)

Data Type: True | False

Flag to plot the smoothed lowerbound over iterations to quicly check the convergence of the VB algorithm.

Default: true

Example: 'LBPlot',false

Data Type: Double | Between 0 and 1

The fixed learning rate $\epsilon_0$ to update the variational parameters in each VB iteration in Algorithm 7.

Must be a number between $0$ and $1$.

Default: 0.01

Example: 'LearningRate',0.001

Data Type: Integer | Positive

Maximum number of VB iterations for early stopping. If the 'BatchSize' argument is specified, users have to use the 'MaxEpoch' argument to specify the maximum number of iterations instead.

Default: 1000

Example: 'MaxIter',1000

Data Type: Integer | Positive

Number of consecutive times that the validation loss, or lowerbound, is allowed to be larger than or equal to the previously smallest loss, or lowerbound, before the training is stopped, used as an early stopping criterion. This is denoted as $P$ in Algorithm 7.

Default: 20

Example: 'MaxPatience',100

Data Type: Integer | Positive

Number of Monte Carlo samples needed to estimate the gradient of the lower bound. This is denoted as $S$ in Algorithm 7.

Default: 50

Example: 'NumSample',100

Data Type: Integer | Positive

Number of model parameters.

• If the handle of the function calculating the $h(\theta)$ and $\nabla_\theta h(\theta)$ terms is provided, users have to specify a value for this argument.
• If a model object is specified, users have to set the number of parameters using the NumParams property of the model class. See how to define a custom model as a Maltab class object.

Default: None

Example: 'NumParams',{'Normal',[0,10]}

Data Type: true | false

Flag to save variational parameters in each VB iteration.

Default: false

Example: 'SaveParams',true

Data Type: struct

Additional settings that could be used to define custom models as function handlers.

It’s the most efficient o define these additional setting as a struct. This struct then will be passed to the function handlers as an input. See how to define custom model as function handler.

Default: None

Example: 'Setting',prior with prior is a struct whose fields are prior distribution name and parameters, e.g. prior.name = 'Normal' and prior.params = [0,1].

Data Type: double

The constant factor $c$ to scale the initial values for the lower triangular matrix $L$.

Default: 0.1

Example: 'SigInitScale',0.5

Data Type: double

The constant factor $\sigma$ to scale the convariance matrix of the normal distribution used to initialize the variational mean.

Only specify this argument when the argument 'InitMethod' is set to 'Random'.

Default: 0.01

Example: 'StdForInit',0.04

Data Type: Integer | Positive

The iteration to start reducing learning rate, which is denoted as $\tau$ in Algorithm 7.

By default, this is set as 'MaxIter'/2 or 'MaxEpoch'/2.

Must be smaller than 'MaxIter' or 'MaxEpoch'.

Default: 'MaxIter'/2 or 'MaxEpoch'/2

Example: 'StepAdaptive',300

Data Type: string | cell array of strings

The VB algorithm uses lowerbound to access the convergence of the training phase.

However, users can also calculate the predictive scores evaluated on the training data over VB iterations. Users can specify a single metric, defined as a string, or multiple metrics, defined as a cell array of strings.

Available score metrics:

For the PPS:

• If the models are specified as function handlers, users have to also specify function handlers to the argument 'LogLikFunc' to compute the log-likelihood of the custom models.
• If the models are specified as class objects, users have to define a method named logLik() to compute the log-likelihood of the custom models.

Default: None

Example: 'TrainingLoss',{'PPS','MSE'}

Data Type: double between 0 and 1 | Integer

Number of observations of training data are used as validation data. The number of observations can be specified as a percentage (a number between 0 and 1) of training data or an integer smaller than the number of training observations.

Note: This option is only available for cross-sectional (tabular) data.

Default: None

Example: 'Prior',0.1 or 'Prior',1000

Data Type: string | cell array of strings

Calculate the predictive scores evaluated on the validation data over VB iterations. Users can specify a single metric, defined as a string, or multiple metrics, defined as a cell array of strings.

Available score metrics:

For the PPS:

• If the models are specified as function handlers, users have to also specify function handlers to the argument 'LogLikFunc' to compute the log-likelihood of the custom models.
• If the models are specified as class objects, users have to define a method named logLik() to compute the log-likelihood of the custom models.

Note: This option is only available for cross-sectional (tabular) data.

Default: None

Example: 'ValidationLoss','MSE'

Data Type: True | False

By default, the index of the current iteration and lowerbound are shown in every iteration. Set 'Verbose' to be false to turn off these messages.

Default: true

Example: 'Verbose',false

Data Type: Integer | Positive

Size of moving average window that is used to smooth the lowerbound. Denoted as $t_W$ in Algorithm 7.

Default: 50

Example: 'WindowSize',100