Module reference#

Gaussian process models#

Gaussian process classes for quantile regression.

class gpytorch_qr.models.QuantileGP(variational_strategy, mean_module, covar_module)[source]#

Base class for Gaussian process quantile regression.

Parameters:
variational_strategygpytorch.variational.VariationalStrategy
mean_modulegpytorch.means.Mean
covar_modulegpytorch.kernels.Kernel

Notes

Input predictors are expected to have shape (*B, N, D), where *B are optional batch shapes (e.g., for cross validation), N is the number of data points and D is the number of input dimensions.

Quantiles are task dimension with shape T, constructed by combination of L latent GPs.

  • variational_strategy must wrap a variational distribution with batch shape (*B, L).

  • mean_module and covar_module must have batch shape (*B, L).

  • Posterior is gpytorch.distributions.MultitaskMultivariateNormal with batch shape (*B) and event shape (N, T).

  • MLL loss is a tensor of shape (*B).

forward(x)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

abstractmethod joint_quantile_posterior(x)[source]#

Joint posterior over quantiles at input locations.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
torch.distributions.Distribution
marginal_quantile_posterior(x)[source]#

Marginal posterior over quantiles at input locations.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
torch.distributions.Distribution
mean_quantiles(x)[source]#

Predict quantiles by analytical posterior mean.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

mean_quantiles_mc(x, num_samples=10)[source]#

Posterior mean of quantiles by Monte Carlo approximation.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

num_samplesint, default=10

The number of Monte Carlo samples.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

mean_quantiles_delta(x)[source]#

Posterior mean of quantiles by 0th-order delta method.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

quantile_quantiles(x, q)[source]#

Analytic quantile of quantile posterior.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

qtorch.Tensor with shape (q,)

The quantile levels.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

quantile_quantiles_mc(x, q, num_samples=10)[source]#

Quantile of quantile posterior by Monte Carlo approximation.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

qtorch.Tensor with shape (q,)

The quantile levels.

num_samplesint, default=10

The number of Monte Carlo samples.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

class gpytorch_qr.models.DirectQuantileGP(variational_strategy, mean_module, covar_module)[source]#

Gaussian process quantile regression with direct quantile representation.

Notes

The task dimension of the output GP is structured as

[*Q_1, *Q_2, ..., *Q_k]

where Q_i contains quantiles for the i-th output dimension.

joint_quantile_posterior(x)[source]#

Joint posterior over quantiles at input locations.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
torch.distributions.Distribution
marginal_quantile_posterior(x)[source]#

Marginal posterior over quantiles at input locations.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
torch.distributions.Distribution
mean_quantiles(x)[source]#

Predict quantiles by analytical posterior mean.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

mean_quantiles_delta(x)[source]#

Posterior mean of quantiles by 0th-order delta method.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

quantile_quantiles(x, q)[source]#

Analytic quantile of quantile posterior.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

qtorch.Tensor with shape (q,)

The quantile levels.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

class gpytorch_qr.models.CenterGapQuantileGP(variational_strategy, mean_module, covar_module, num_quantiles, central_quantile_idx, quantile_levels=None)[source]#

Gaussian process quantile regression with center-gap quantile representation.

Parameters:
variational_strategy
mean_module
covar_module
num_quantileslist of int

The number of quantiles in each output dimension.

central_quantile_idxlist of int

Indices of central quantiles in each output dimension.

quantile_levelslist of torch.Tensor, optional

Quantile levels for each output dimension. Required for posterior prediction when gpytorch_qr.settings.quantile_gap_lower_bound is nonzero.

Notes

The task dimension of the output GP is structured as

[c_1, c_2, ..., c_k,  *L_1, *U_1,  *L_2, *U_2,  ...,  *L_k, *U_k]

where c_i is the central quantile for the i-th output dimension, L_i contains the pre-softplus-transformed lower gaps, and U_i contains the pre-softplus-transformed upper gaps.

joint_quantile_posterior(x)[source]#

Joint posterior over quantiles at input locations.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
torch.distributions.Distribution
mean_quantiles_delta(x)[source]#

Posterior mean of quantiles by 0th-order delta method.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

Asymmetric laplace distribution#

Asymmetric Laplace distributions for Bayesian quantile regression.

class gpytorch_qr.distributions.AsymmetricLaplace(loc, scale, asymmetry, validate_args=None)[source]#

Asymmetric Laplace distribution.

Parameters:
loctorch.Tensor

Location parameter of the distribution.

scaletorch.Tensor

Scale of the distribution.

asymmetrytorch.Tensor

Asymmetry of the distribution.

property mean#

Returns the mean of the distribution.

property mode#

Returns the mode of the distribution.

property variance#

Returns the variance of the distribution.

property stddev#

Returns the standard deviation of the distribution.

expand(batch_shape, _instance=None)[source]#

Return a new distribution instance with batch dimensions expanded.

rsample(sample_shape=())[source]#

Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched.

log_prob(value)[source]#

Returns the log of the probability density/mass function evaluated at value.

Args:

value (Tensor):

cdf(value)[source]#

Returns the cumulative density/mass function evaluated at value.

Args:

value (Tensor):

icdf(value)[source]#

Returns the inverse cumulative density/mass function evaluated at value.

Args:

value (Tensor):

entropy()[source]#

Returns entropy of distribution, batched over batch_shape.

Returns:

Tensor of shape batch_shape.

Asymmetric laplace likelihood#

Asymmetric Laplace distributions likelihoods for quantile regression.

class gpytorch_qr.likelihoods.AsymmetricLaplaceLikelihood(kappa, noise_prior=None, noise_constraint=None, batch_shape=(), **kwargs)[source]#

Likelihood with homoscedastic asymmetric Laplace distribution.

Parameters:
kappatorch.Tensor

The asymmetry parameters of the distribution.

noise_prior

Prior for noise parameter \(\sigma^2\).

noise_constraint

Constraint for noise parameter \(\sigma^2\).

batch_shape: torch.Size, default=torch.Size()

The batch shape of the learned noise parameter.

See also

gpytorch.likelihoods.GaussianLikelihood

Gaussian distribution equivalent of this likelihood.

property noise#

The squared ALD quantile-loss scale \(\lambda^2\).

property raw_noise#

The unconstrained parameter corresponding to noise.

class gpytorch_qr.likelihoods.MultitaskAsymmetricLaplaceLikelihood(kappa, num_tasks, rank=0, batch_shape=(), task_prior=None, noise_prior=None, noise_constraint=None, has_global_noise=True, has_task_noise=True)[source]#

Homoscedastic asymmetric Laplace likelihood for multitask models.

Parameters:
kappatorch.Tensor

Asymmetry parameter of the asymmetric Laplace distribution.

num_tasksint

Number of tasks.

rankint, default=0

Rank of the task noise covariance. Only 0 is supported.

batch_shapetorch.Size, default=torch.Size()

Batch shape of the learned noise parameters.

task_prior

Prior over the task noise covariance when rank > 0.

noise_prior

Prior over the global or task-specific squared ALD scales.

noise_constraint

Constraint for squared ALD scales. Defaults to GreaterThan(1e-4).

has_global_noisebool, default=True

Whether to include a shared squared ALD scale.

has_task_noisebool, default=True

Whether to include task-specific squared ALD scales.

See also

gpytorch.likelihoods.MultitaskGaussianLikelihood

Multitask Gaussian distribution equivalent of this likelihood.

property noise#

The shared squared ALD scale.

property task_noises#

The independent task-specific squared ALD scales.

class gpytorch_qr.likelihoods.DirectQuantilesLikelihood(quantile_levels, rank=0, batch_shape=(), task_prior=None, noise_prior=None, noise_constraint=None, has_global_noise=True, has_task_noise=True)[source]#

Likelihood for single-output GPQR with direct quantile representation.

Parameters:
quantile_levelstorch.Tensor with shape (*B, Q)

The quantile levels.

rank
batch_shape
task_prior
noise_prior
noise_constraint
has_global_noise
has_task_noise
expected_log_prob(observations, function_dist, *args, **kwargs)[source]#

Expected log probability of the observed data under the ALD likelihood.

Parameters:
observationstorch.Tensor with shape (*B, N)

The observed response variables.

function_disttorch.distributions.Distribution

Latent GP posterior at the observed locations.

Returns:
torch.Tensor with shape (*B, N, Q)

The expected log probability of the observed data under the ALD likelihood.

class gpytorch_qr.likelihoods.MultioutputDirectQuantilesLikelihood(quantile_levels, rank=0, batch_shape=(), task_prior=None, noise_prior=None, noise_constraint=None, has_global_noise=True, has_task_noise=True)[source]#

Likelihood for multi-output GPQR with direct quantile representation.

Parameters:
quantile_levels
rank
batch_shape
task_prior
noise_prior
noise_constraint
has_global_noise
has_task_noise
expected_log_prob(observations, function_dist, *args, **kwargs)[source]#

Expected log probability of the observed data under the ALD likelihood.

Parameters:
observationstorch.Tensor with shape (*B, N, D)

The observed response variables.

function_disttorch.distributions.Distribution

Latent GP posterior at the observed locations.

Returns:
torch.Tensor with shape (*B, N, Q)

The expected log probability of the observed data under the ALD likelihood.

Notes

The last dimension of observations is the output dimension, which is not the task dimension. The task dimension is the sum of all number of quantiles across all output dimensions, i.e., T = Q_1 + Q_2 + ... + Q_D.

class gpytorch_qr.likelihoods.CenterGapQuantilesLikelihood(quantile_levels, central_quantile_idx, rank=0, batch_shape=(), task_prior=None, noise_prior=None, noise_constraint=None, has_global_noise=True, has_task_noise=True)[source]#

Likelihood for single-output GPQR with center-gap quantile representation.

Parameters:
quantile_levelstorch.Tensor with shape (*B, Q)

The quantile levels.

central_quantile_idxint or torch.Tensor with shape (*B)

The index of the central quantile in the quantile levels.

rank
batch_shape
task_prior
noise_prior
noise_constraint
has_global_noise
has_task_noise
expected_log_prob(observations, function_dist, *args, **kwargs)[source]#

Expected log probability of the observed data under the ALD likelihood.

Parameters:
observationstorch.Tensor with shape (*B, N)

The observed response variables.

function_disttorch.distributions.Distribution

Latent GP posterior at the observed locations.

Returns:
torch.Tensor with shape (*B, N, Q)

The expected log probability of the observed data under the ALD likelihood.

class gpytorch_qr.likelihoods.MultioutputCenterGapQuantilesLikelihood(quantile_levels, central_quantile_idxs, rank=0, batch_shape=(), task_prior=None, noise_prior=None, noise_constraint=None, has_global_noise=True, has_task_noise=True)[source]#

Likelihood for multi-output GPQR with center-gap quantile representation.

Parameters:
quantile_levels
central_quantile_idxs
rank
batch_shape
task_prior
noise_prior
noise_constraint
has_global_noise
has_task_noise
expected_log_prob(observations, function_dist, *args, **kwargs)[source]#

Expected log probability of the observed data under the ALD likelihood.

Parameters:
observationstorch.Tensor with shape (*B, N, D)

The observed response variables.

function_disttorch.distributions.Distribution

Latent GP posterior at the observed locations.

Returns:
torch.Tensor with shape (*B, N, Q)

The expected log probability of the observed data under the ALD likelihood.

Notes

The last dimension of observations is the output dimension, which is not the task dimension. The task dimension is the sum of all number of quantiles across all output dimensions, i.e., T = Q_1 + Q_2 + ... + Q_D.

Variational strategies#

Variational strategies for GPQR.

class gpytorch_qr.variational.CenterGapLMCVariationalStrategy(base_variational_strategy, num_tasks, num_latents, latent_dim=-1, jitter_val=None, num_central_latents=None, num_quantiles=None)[source]#

A special LMC variational strategy for the center-gap representation.

This class enforces independence between the central quantiles and the gap functions.

Parameters:
base_variational_strategy
num_tasks
num_latents
latent_dim
jitter_val
num_central_latentsint, optional

The number of latent functions that are linearly combined to model the central quantiles. If not passed, defaults to len(num_quantiles).

num_quantileslist of int, optional

The number of quantiles in each output dimension. Its sum must equal num_tasks. If not passed, defaults to [num_tasks].

Notes

Input GP

The input T latent GPs are split to

[c_1, c_2, ..., c_k,  g_1, g_2, ..., g_{T-k}]

where:

  • c_i are linearly combined to form central quantiles for each output dimension.

  • g_j are linearly combined to form the gap functions between quantiles.

  • c_i and g_j are not combined with each other, i.e., central quantiles and gap functions are independent.

The number of c_i, i.e., k, is num_central_latents.

Output GP

The output multitask GPs are structured as

[C_1, C_2, ..., C_q,  *L_1, *U_1,  *L_2, *U_2,  ...,  *L_q, *U_q]

where:

  • C_i is the central quantile for i-th output dimension,

  • L_i contains pre-softplus-transformed lower gaps for i-th output dimension,

  • U_i contains pre-softplus-transformed upper gaps for i-th output dimension.

The number of output dimensions, i.e., q, is len(num_quantiles). The sum 1 + len(L_i) + len(U_i) equals num_quantiles[i] for each output dimension i.

construct_lmc_mask(T, Q)[source]#

Construct a mask to restrict the learnable LMC coefficients.

Parameters:
Tint

The number of latent functions.

Qint

The number of quantiles.

Returns:
lmc_masktorch.Tensor with shape (T, Q)

A binary mask of the same shape as the LMC coefficients. 1 indicates learnable coefficients, and 0 indicates fixed coefficients.

Utilities#

Utility functions.

gpytorch_qr.utils.centergap_to_quantiles(central, lower_gaps, upper_gaps, quantile_level_offsets=None)[source]#

Convert center-gap representation samples to quantiles.

Parameters:
centraltorch.Tensor with shape (…, 1)

The central quantile values.

lower_gapstorch.Tensor with shape (…, L)

Pre-softplus-transformed lower gap values.

upper_gapstorch.Tensor with shape (…, U)

Pre-softplus-transformed upper gap values.

quantile_level_offsetstorch.Tensor, optional

Quantile levels relative to the central quantile, with shape broadcastable to (..., Q). The active gpytorch_qr.settings.quantile_gap_lower_bound multiplies these offsets, adding lower_bound * diff(q) to each adjacent gap.

Returns:
quantilestorch.Tensor with shape (…, Q)

Quantile values. (Q = L + U + 1) The quantiles are ordered in increasing order along the quantile dimension. Their dtype is controlled by gpytorch_qr.settings.quantile_reconstruction_dtype.

class gpytorch_qr.utils.CenterGapToQuantileTransform(Qs, Ls, quantile_level_offsets=None)[source]#

Transformation from center-gap distribution to quantile distribution.

Parameters:
Qslist of int

The number of quantiles for each task, i.e., [Q_1, Q_2, ..., Q_k].

Lslist of int

The number of lower quantiles in center-gap representation for each task, i.e., [L_1, L_2, ..., L_k].

Notes

The input distribution’s quantile dimension should be laid out as:

[c_1, c_2, ..., c_k,  *L_1, *U_1,  *L_2, *U_2,  ...,  *L_k, *U_k]

where:

  • c_i is the central quantile for i-th output dimension,

  • L_i contains pre-softplus-transformed lower gaps for i-th output dimension,

  • U_i contains pre-softplus-transformed upper gaps for i-th output dimension.

The output distribution’s quantile dimension is laid out as:

[*Q_1, *Q_2, ..., *Q_k].
log_abs_det_jacobian(x, y)[source]#

Computes the log det jacobian log |dy/dx| given input and output.

gpytorch_qr.utils.transform_centergap_posterior(posterior, Qs, Ls, quantile_level_offsets=None)[source]#

Convert the center-gap posterior to quantile posterior.

Parameters:
posteriorgpytorch.distributions.MultitaskMultivariateNormal

The center-gap posterior distribution. Event shape must be (N, Q_1 + Q_2 + ..., Q_k), where Q_i is the number of quantiles for i-th output dimension.

Qslist of int

The number of quantiles for each task, i.e., [Q_1, Q_2, ..., Q_k].

Lslist of int

The number of lower quantiles in center-gap representation for each task, i.e., [L_1, L_2, ..., L_k].

quantile_level_offsetstorch.Tensor, optional

Quantile levels relative to their central quantiles, with trailing size sum(Qs).

Returns:
quantile_posteriortorch.distributions.TransformedDistribution

Posterior over quantiles, obtained by applying CenterGapToQuantileTransform to a batched gpytorch.distributions.MultitaskMultivariateNormal.

Notes

Input and output distribution has specific structure in the quantile dimension. See CenterGapToQuantileTransform for details.

Settings#

Runtime settings for GPyTorch-QR.

class gpytorch_qr.settings.enforce_strict_quantile_order(state=True)[source]#

Enforce representably distinct center-gap quantile predictions.

When enabled, each predicted quantile is at least the next representable floating-point value above the preceding quantile. The correction is applied independently to each output dimension and only to model posterior predictions, not to the training likelihood.

This is an inference-oriented, dtype-dependent correction. It guarantees strict order for finite predictions but does not preserve a requested mathematical gap when that gap is smaller than one ULP.

Parameters:
statebool, default=True

Whether to enable strict quantile ordering.

class gpytorch_qr.settings.quantile_gap_lower_bound(value)[source]#

Set the minimum center-gap quantile slope.

Within this context, every adjacent center-gap quantile difference is transformed as

\[\operatorname{softplus}(\mathrm{raw\_gap}) + \mathrm{lower\_bound} \, \Delta q.\]

The setting is read when center-gap latent values are converted to quantiles, so the context must enclose likelihood evaluation and posterior prediction. Nested contexts restore the previously active value on exit.

Parameters:
valuefloat

A finite, non-negative lower bound. The default outside a context is 0.0.

Examples

>>> import gpytorch_qr
>>> with gpytorch_qr.settings.quantile_gap_lower_bound(1e-4):
...     gpytorch_qr.settings.quantile_gap_lower_bound.value()
0.0001
class gpytorch_qr.settings.quantile_reconstruction_dtype(dtype)[source]#

Set the dtype used to reconstruct center-gap quantiles.

The center, raw gaps, and quantile-level offsets are converted before the softplus, cumulative-sum, and addition operations. The reconstructed quantiles retain the selected dtype. This prevents a small gap from being rounded away when it is added to a large central quantile.

Parameters:
dtypetorch.dtype or None

A real floating-point dtype. None preserves the dtype of the latent center and gaps and is the default.

On this page
Show Source