Module reference#

Gaussian process models#

Gaussian process classes for quantile regression.

class gpytorch_qr.models.QuantileGP(variational_strategy, mean_module, covar_module, latent_dim)[source]#

Base class for Gaussian process quantile regression.

Parameters:
variational_strategygpytorch.variational.VariationalStrategy
mean_modulegpytorch.means.Mean
covar_modulegpytorch.kernels.Kernel
latent_dim{0, -1}

The dimension along which the latent GPs are represented in module batch shape. 0 if quantiles are batches, -1 if quantiles are multitasks.

Notes

Input predictors are expected to have shape (*B, N, D), where *B are optional batch shapes (e.g., for cross validation), N is the number of data points and D is the number of input dimensions.

Quantiles can be either batch dimension or task dimension with shape Q.

Batch quantiles

  • variational_strategy must wrap a variational distribution with batch shape (Q, *B).

  • mean_module and covar_module must have batch shape (Q, *B).

  • Posterior is gpytorch.distributions.MultivariateNormal with batch shape (Q, *B) and event shape (N,).

  • MLL loss is a tensor of shape (Q, *B).

Multitask quantiles

Quantiles are constructed by combination of L latent GPs.

  • variational_strategy must wrap a variational distribution with batch shape (*B, L).

  • mean_module and covar_module must have batch shape (*B, L).

  • Posterior is gpytorch.distributions.MultitaskMultivariateNormal with batch shape (*B) and event shape (N, Q).

  • MLL loss is a tensor of shape (*B).

forward(x)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

abstractmethod joint_quantile_posterior(x)[source]#

Joint posterior over quantiles at input locations.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
torch.distributions.Distribution
marginal_quantile_posterior(x)[source]#

Marginal posterior over quantiles at input locations.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
torch.distributions.Distribution
mean_quantiles(x)[source]#

Predict quantiles by analytical posterior mean.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

mean_quantiles_mc(x, num_samples=10)[source]#

Posterior mean of quantiles by Monte Carlo approximation.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

num_samplesint, default=10

The number of Monte Carlo samples.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

mean_quantiles_delta(x)[source]#

Posterior mean of quantiles by 0th-order delta method.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

quantile_quantiles(x, q)[source]#

Analytic quantile of quantile posterior.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

qtorch.Tensor with shape (q,)

The quantile levels.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

quantile_quantiles_mc(x, q, num_samples=10)[source]#

Quantile of quantile posterior by Monte Carlo approximation.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

qtorch.Tensor with shape (q,)

The quantile levels.

num_samplesint, default=10

The number of Monte Carlo samples.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

class gpytorch_qr.models.DirectQuantileGP(variational_strategy, mean_module, covar_module, latent_dim)[source]#

Gaussian process quantile regression with direct quantile representation.

joint_quantile_posterior(x)[source]#

Joint posterior over quantiles at input locations.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
torch.distributions.Distribution
marginal_quantile_posterior(x)[source]#

Marginal posterior over quantiles at input locations.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
torch.distributions.Distribution
mean_quantiles(x)[source]#

Predict quantiles by analytical posterior mean.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

mean_quantiles_delta(x)[source]#

Posterior mean of quantiles by 0th-order delta method.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

quantile_quantiles(x, q)[source]#

Analytic quantile of quantile posterior.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

qtorch.Tensor with shape (q,)

The quantile levels.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

class gpytorch_qr.models.CenterGapQuantileGP(variational_strategy, mean_module, covar_module, latent_dim, num_lower_quantiles)[source]#

Gaussian process quantile regression with center-gap quantile representation.

Parameters:
variational_strategy
mean_modulegpytorch_qr.centergap.CenterGapMean

Mean module for center-gap representation.

covar_module
latent_dim
num_lower_quantilesint

The number of lower quantiles in center-gap representation.

joint_quantile_posterior(x)[source]#

Joint posterior over quantiles at input locations.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
torch.distributions.Distribution
mean_quantiles_delta(x)[source]#

Posterior mean of quantiles by 0th-order delta method.

Parameters:
xtorch.Tensor with shape (*B, N, D)

The input locations.

Returns:
quantilestorch.Tensor

The predicted quantiles at the input locations.

Asymmetric laplace distribution#

Asymmetric Laplace distributions for quantile regression.

class gpytorch_qr.distributions.ALD(m, lamda, kappa)[source]#

Asymmetric Laplace distribution.

Parameters:
mtorch.Tensor

The location parameter of the distribution.

lamdatorch.Tensor

The scale parameter of the distribution.

kappatorch.Tensor

The quantile level of the distribution.

Attributes:
mtorch.Tensor
lamdatorch.Tensor
kappatorch.Tensor
log_prob(value)[source]#

Log probability of the asymmetric Laplace distribution

icdf(value)[source]#

Inverse CDF of the asymmetric Laplace distribution.

class gpytorch_qr.distributions.BatchQuantileALD(m, lamda, kappa)[source]#

Asymmetric Laplace distribution where quantiles are treated as batches.

Parameters:
mtorch.Tensor with shape (S, Q, *B, N)

The location parameters of the distribution.

lamdatorch.Tensor with shape (Q, *B, 1)

The scale parameters of the distribution for each quantile.

kappatorch.Tensor with shape (Q, *B, 1)

The quantile levels of the distribution.

Attributes:
mtorch.Tensor with shape (S, Q, *B, N)
lamdatorch.Tensor with shape (1, Q, *B, 1)
kappatorch.Tensor with shape (1, Q, *B, 1)

Notes

  • S : the number of samples drawn from the posterior distribution.

  • Q : the number of quantiles.

  • B : additional batches.

  • N : the number of data points.

The posterior distribution have batch shape (Q, *B).

log_prob(value)[source]#

Log probability of the asymmetric Laplace distribution at the given value.

Parameters:
valuetorch.Tensor with shape (*B, N)

Observed response variables at which to evaluate the log probability.

Returns:
logptorch.Tensor with shape (S, Q, *B, N)

The log probability at the given values for each quantile and sample.

class gpytorch_qr.distributions.MultitaskQuantileALD(m, lamda, kappa)[source]#

Asymmetric Laplace distribution where quantiles are treated as tasks.

Parameters:
mtorch.Tensor with shape (S, *B, N, Q)

The location parameters of the distribution.

lamdatorch.Tensor with shape (*B, 1, Q)

The scale parameters of the distribution for each quantile.

kappatorch.Tensor with shape (*B, 1, Q)

The quantile levels of the distribution.

Attributes:
mtorch.Tensor with shape (S, *B, N, Q)
lamdatorch.Tensor with shape (1, *B, 1, Q)
kappatorch.Tensor with shape (1, *B, 1, Q)

Notes

  • S : the number of samples drawn from the posterior distribution.

  • Q : the number of quantiles.

  • B : additional batches.

  • N : the number of data points.

The posterior distribution have batch shape (*B).

log_prob(value)[source]#

Log probability of the asymmetric Laplace distribution at the given value.

Parameters:
valuetorch.Tensor with shape (*B, N)

Observed response variables at which to evaluate the log probability.

Returns:
logptorch.Tensor with shape (S, *B, N, Q)

The log probability at the given values for each quantile and sample.

Asymmetric laplace likelihood#

Asymmetric Laplace distributions likelihoods for quantile regression.

class gpytorch_qr.likelihoods.ALDLikelihood(q, raw_scales=0.0, learn_scales=True)[source]#

Asymmetric Laplace distribution likelihood.

Parameters:
qtorch.Tensor

The quantile levels.

raw_scalestorch.Tensor or scalar, default=0

The initial untransformed scales of the asymmetric Laplace distribution. The actual scales are obtained by applying the positive transformation. If tensor, dimension should be same to q and shape should be broadcastable. Scalar value is repeated to the shape of q.

learn_scalesbool, default=True

Whether to update scales by gradients.

Notes

Whether to broadcast raw_scales is important when learn_scales=True. When the scale is broadcasted, the same scale parameter is shared across and updated along the broadcasted dimension, e.g., across different quantile levels or batches.

Sharing scales across quantiles may be deliberately used to reduce the number of parameters and increase the stability of training. On the other hand, sharing scales across batches usually does not make sense and should be avoided. In general, it is recommended to use independent scale parameters for all channels.

To encourage the use of independent scale parameters, scalar raw_scales is repeated to the shape of q instead of being broadcasted. For example, if q has shape (Q, *B) and raw_scales is Tensor(1) whose shape is (), then it is converted to a tensor of shape (Q, *B) where all values are 1. On the other hand, if raw_scales is Tensor([[1]]) whose shape is (1, 1), then it is broadcasted to shape (Q, *B) and shared across quantiles and batches.

predictive_posterior(gp_posterior)[source]#

Predictive posterior distribution of function values.

Parameters:
gp_posteriorgpytorch.distributions.MultivariateNormal

The joint posterior over latent GPs at input locations.

Returns:
samplestorch.Tensor

Samples drawn from the predictive posterior distribution of function values. The first dimension is the sampling dimension.

Examples

Get 95% predictive intervals:

with torch.no_grad(), gpytorch.settings.num_likelihood_samples(1000):
    pp_dist = gp(x_pred)
ci_lower = pp_dist.quantile(0.025, dim=0)
ci_upper = pp_dist.quantile(0.975, dim=0)
class gpytorch_qr.likelihoods.BatchQuantileGPLikelihood(q, raw_scales=0.0, learn_scales=True)[source]#

Likelihood for BatchQuantileALD with direct representation.

Parameters:
q

The quantile levels. Shape is (Q, *B).

raw_scales

The initial untransformed scales of the asymmetric Laplace distribution. Shape is either () or (Q, *B).

learn_scales
Attributes:
qtorch.Tensor with shape (Q, *B)
raw_scalestorch.Tensor with shape (Q, *B)

Examples

>>> import torch
>>> from torch.distributions import Normal
>>> torch.manual_seed(42)
>>> def mean(x):
...     return torch.cos(x * 2 * 3.14)
>>> def std(x):
...     return x + 0.1
>>> x_range = torch.linspace(0, 1, 10).reshape(-1, 1)
>>> x = x_range.repeat(2, 1)
>>> y = (mean(x) + torch.randn(x.shape).mul(std(x))).squeeze()
>>> q = torch.tensor([0.1, 0.25, 0.5, 0.75, 0.9])
>>> true_quantiles = mean(x_range) + std(x_range) * Normal(0, 1).icdf(q)
>>> from gpytorch.variational import CholeskyVariationalDistribution
>>> from gpytorch.variational import VariationalStrategy
>>> from gpytorch.means import ConstantMean
>>> from gpytorch.kernels import RBFKernel, ScaleKernel
>>> from gpytorch_qr.models import DirectQuantileGP
>>> from gpytorch_qr.likelihoods import BatchQuantileGPLikelihood
>>> class MyGP(DirectQuantileGP):
...     def __init__(self, inducing_points, num_quantiles):
...         N, D = inducing_points.size()
...         variational_distribution = CholeskyVariationalDistribution(
...             N,
...             batch_shape=torch.Size([num_quantiles]),
...         )
...         variational_strategy = VariationalStrategy(
...             self,
...             inducing_points,
...             variational_distribution,
...             learn_inducing_locations=True,
...         )
...         mean = ConstantMean(batch_shape=torch.Size([num_quantiles]))
...         covar = ScaleKernel(
...             RBFKernel(ard_num_dims=D, batch_shape=torch.Size([num_quantiles])),
...             batch_shape=torch.Size([num_quantiles]),
...         )
...         super().__init__(variational_strategy, mean, covar, 0)
>>> inducing_points = torch.linspace(0, 1, 10).reshape(-1, 1)
>>> gp = MyGP(inducing_points, len(q))
>>> likelihood = BatchQuantileGPLikelihood(q)
>>> from gpytorch.mlls import VariationalELBO
>>> gp.train()
>>> likelihood.train()
>>> mll = VariationalELBO(likelihood, gp, num_data=y.numel())
>>> optimizer = torch.optim.Adam(
...     list(gp.parameters()) + list(likelihood.parameters()),
...     lr=0.001,
... )
>>> N = 1  # Set to 1 for faster training; increase for better performance
>>> for _ in range(N):
...     output = gp(x)
...     loss = -mll(output, y).sum()
...     loss.backward()
...     optimizer.step()
...     optimizer.zero_grad()
>>> gp.eval()
>>> x_pred = torch.linspace(0, 2, 100).reshape(-1, 1)
>>> with torch.no_grad():
...     mean_q = gp.mean_quantiles(x_pred)
>>> import matplotlib.pyplot as plt
>>> plt.scatter(x, y, c='gray', marker='.', alpha=0.1)
>>> plt.plot(x_range, true_quantiles, '--', c='k')
>>> plt.plot(x_pred, mean_q.T)
../_images/index-1.png
forward(function_samples)[source]#

Return the ALD distribution for the given function samples.

Parameters:
function_samplestorch.Tensor with shape (S, Q, *B, N)

The function samples drawn from the posterior distributions of quantile functions. S is the number of samples, Q is the number of quantiles, B is the batch shape, and N is the number of data points.

Returns:
BatchQuantileALD
expected_log_prob(observations, function_dist, *args, **kwargs)[source]#

Expected log probability of the observed data under the ALD likelihood.

Parameters:
observationstorch.Tensor with shape (*B, N)

The observed response variables.

function_disttorch.distributions.Distribution

The distribution of the function values at the input locations.

Returns:
torch.Tensor with shape (*B, N)

The expected log probability of the observed data under the ALD likelihood.

class gpytorch_qr.likelihoods.MultitaskQuantileGPLikelihood(q, raw_scales=0.0, learn_scales=True)[source]#

Likelihood for MultitaskQuantileALD with direct representation.

It is recommended to use fewer latent GPs than the number of tasks(=quantiles) to model the correlation structure.

Parameters:
q

The quantile levels. Shape is (*B, Q).

raw_scales

The initial untransformed scales of the asymmetric Laplace distribution. Shape is either () or (*B, Q).

learn_scales
Attributes:
qtorch.Tensor with shape (*B, Q)
raw_scalestorch.Tensor with shape (*B, Q)
forward(function_samples)[source]#

Return the ALD distribution for the given function samples.

Parameters:
function_samplestorch.Tensor with shape (S, *B, N, Q)

The function samples drawn from the posterior distributions of quantile functions. S is the number of samples, Q is the number of quantiles, B is the batch shape, and N is the number of data points.

Returns:
MultitaskQuantileALD
expected_log_prob(observations, function_dist, *args, **kwargs)[source]#

Expected log probability of the observed data under the ALD likelihood.

Parameters:
observationstorch.Tensor with shape (*B, N)

The observed response variables.

function_disttorch.distributions.Distribution

The distribution of the function values at the input locations.

Returns:
torch.Tensor with shape (*B, Q)

The expected log probability of the observed data under the ALD likelihood.

class gpytorch_qr.likelihoods.BatchCenterGapQuantileGPLikelihood(q, central_quantile_index, raw_scales=0.0, learn_scales=True)[source]#

Likelihood for BatchQuantileALD with center-gap representation.

Parameters:
q

The quantile levels. Shape is (Q, *B).

central_quantile_indextorch.Tensor or scalar

The index of the central quantile in the quantile levels. If tensor, it should be broadcastable to the shape of (*B) so that each batch can have different central quantile index.

raw_scales

The initial untransformed scales of the asymmetric Laplace distribution. Shape is either () or (Q, *B).

learn_scales
Attributes:
qtorch.Tensor with shape (Q, *B)
raw_scalestorch.Tensor with shape (Q, *B)

Examples

>>> import torch
>>> from torch.distributions import Normal
>>> torch.manual_seed(42)
>>> def mean(x):
...     return torch.cos(x * 2 * 3.14)
>>> def std(x):
...     return x + 0.1
>>> x_range = torch.linspace(0, 1, 10).reshape(-1, 1)
>>> x = x_range.repeat(2, 1)
>>> y = (mean(x) + torch.randn(x.shape).mul(std(x))).squeeze()
>>> q = torch.tensor([0.1, 0.25, 0.5, 0.75, 0.9])
>>> true_quantiles = mean(x_range) + std(x_range) * Normal(0, 1).icdf(q)
>>> from gpytorch.variational import CholeskyVariationalDistribution
>>> from gpytorch.variational import VariationalStrategy
>>> from gpytorch.means import ConstantMean
>>> from gpytorch.kernels import RBFKernel, ScaleKernel
>>> from gpytorch_qr.means import CenterGapMean
>>> from gpytorch_qr.models import CenterGapQuantileGP
>>> from gpytorch_qr.likelihoods import BatchCenterGapQuantileGPLikelihood
>>> class MyGP(CenterGapQuantileGP):
...     def __init__(self, inducing_points, num_quantiles, num_lower_q):
...         N, D = inducing_points.size()
...         variational_distribution = CholeskyVariationalDistribution(
...             N,
...             batch_shape=torch.Size([num_quantiles]),
...         )
...         variational_strategy = VariationalStrategy(
...             self,
...             inducing_points,
...             variational_distribution,
...             learn_inducing_locations=True,
...         )
...         mean = CenterGapMean(
...             ConstantMean(batch_shape=torch.Size([1])),
...             ConstantMean(batch_shape=torch.Size([num_quantiles - 1])),
...             latent_dim=0,
...         )
...         covar = ScaleKernel(
...             RBFKernel(ard_num_dims=D, batch_shape=torch.Size([num_quantiles])),
...             batch_shape=torch.Size([num_quantiles]),
...         )
...         super().__init__(variational_strategy, mean, covar, 0, num_lower_q)
>>> inducing_points = torch.linspace(0, 1, 10).reshape(-1, 1)
>>> central_q_index = (q - 0.5).abs().argmin().item()
>>> gp = MyGP(inducing_points, len(q), central_q_index)
>>> likelihood = BatchCenterGapQuantileGPLikelihood(q, central_q_index)
>>> from gpytorch.mlls import VariationalELBO
>>> gp.train()
>>> likelihood.train()
>>> mll = VariationalELBO(likelihood, gp, num_data=y.numel())
>>> optimizer = torch.optim.Adam(
...     list(gp.parameters()) + list(likelihood.parameters()),
...     lr=0.001,
... )
>>> N = 1  # Set to 1 for faster training; increase for better performance
>>> for _ in range(N):
...     output = gp(x)
...     loss = -mll(output, y).sum()
...     loss.backward()
...     optimizer.step()
...     optimizer.zero_grad()
>>> gp.eval()
>>> x_pred = torch.linspace(0, 2, 100).reshape(-1, 1)
>>> with torch.no_grad():
...     quantiles = gp.mean_quantiles_mc(x_pred)
>>> import matplotlib.pyplot as plt
>>> plt.scatter(x, y, c='gray', marker='.', alpha=0.1)
>>> plt.plot(x_range, true_quantiles, '--', c='k')
>>> plt.plot(x_pred, quantiles.T)
../_images/index-2.png
forward(function_samples)[source]#

Return the ALD distribution for the given function samples.

Parameters:
function_samplestorch.Tensor with shape (S, Q, *B, N)

The function samples drawn from the posterior distributions of quantile functions. S is the number of samples, Q is the number of quantiles, B is the batch shape, and N is the number of data points.

Returns:
BatchQuantileALD
expected_log_prob(observations, function_dist, *args, **kwargs)[source]#

Expected log probability of the observed data under the ALD likelihood.

Parameters:
observationstorch.Tensor with shape (*B, N)

The observed response variables.

function_disttorch.distributions.Distribution

The distribution of the function values at the input locations.

Returns:
torch.Tensor with shape (*B, N)

The expected log probability of the observed data under the ALD likelihood.

class gpytorch_qr.likelihoods.MultitaskCenterGapQuantileGPLikelihood(q, central_quantile_index, raw_scales=0.0, learn_scales=True)[source]#

Likelihood for MultitaskQuantileALD with center-gap representation.

Latent GPs model the central quantile and the gaps between quantiles separately.

It is recommended to use fewer latent GPs than the number of tasks(=quantiles) to model the correlation structure.

Parameters:
q

The quantile levels. Shape is (*B, Q).

central_quantile_indexint

The index of the central quantile in the quantile levels.

raw_scales

The initial untransformed scales of the asymmetric Laplace distribution. Shape is either () or (*B, Q).

learn_scales
Attributes:
qtorch.Tensor with shape (*B, Q)
raw_scalestorch.Tensor with shape (*B, Q)

Examples

>>> import torch
>>> from torch.distributions import Normal
>>> torch.manual_seed(42)
>>> def mean(x):
...     return torch.cos(x * 2 * 3.14)
>>> def std(x):
...     return x + 0.1
>>> x_range = torch.linspace(0, 1, 10).reshape(-1, 1)
>>> x = x_range.repeat(2, 1)
>>> y = (mean(x) + torch.randn(x.shape).mul(std(x))).squeeze()
>>> q = torch.tensor([0.1, 0.25, 0.5, 0.75, 0.9])
>>> true_quantiles = mean(x_range) + std(x_range) * Normal(0, 1).icdf(q)
>>> from gpytorch.variational import CholeskyVariationalDistribution
>>> from gpytorch.variational import VariationalStrategy
>>> from gpytorch.means import ConstantMean
>>> from gpytorch.kernels import RBFKernel, ScaleKernel
>>> from gpytorch_qr.means import CenterGapMean
>>> from gpytorch_qr.models import CenterGapQuantileGP
>>> from gpytorch_qr.likelihoods import MultitaskCenterGapQuantileGPLikelihood
>>> from gpytorch_qr.variational import CGBlkdiagLmcVariationalStrategy
>>> class MyGP(CenterGapQuantileGP):
...     def __init__(
...         self,
...         inducing_points,
...         num_quantiles,
...         num_lower_q,
...         num_latents,
...         num_lower_latents,
...     ):
...         N, D = inducing_points.size()
...         variational_distribution = CholeskyVariationalDistribution(
...             N,
...             batch_shape=torch.Size([num_latents]),
...         )
...         variational_strategy = CGBlkdiagLmcVariationalStrategy(
...             VariationalStrategy(
...                 self,
...                 inducing_points,
...                 variational_distribution,
...                 learn_inducing_locations=True,
...             ),
...             num_quantiles=num_quantiles,
...             num_latents=num_latents,
...             num_lower_quantiles=num_lower_q,
...             num_lower_latents=num_lower_latents,
...         )
...         mean = CenterGapMean(
...             ConstantMean(batch_shape=torch.Size([1])),
...             ConstantMean(batch_shape=torch.Size([num_latents - 1])),
...             latent_dim=-1,
...         )
...         covar = ScaleKernel(
...             RBFKernel(ard_num_dims=D, batch_shape=torch.Size([num_latents])),
...             batch_shape=torch.Size([num_latents]),
...         )
...         super().__init__(variational_strategy, mean, covar, -1, num_lower_q)
>>> inducing_pts = torch.linspace(0, 1, 10).reshape(-1, 1)
>>> central_q_index = (q - 0.5).abs().argmin().item()
>>> num_latents = len(q) - 2  # recommended to be smaller than q
>>> gp = MyGP(inducing_pts, len(q), central_q_index, num_latents, num_latents // 2)
>>> likelihood = MultitaskCenterGapQuantileGPLikelihood(q, central_q_index)
>>> from gpytorch.mlls import VariationalELBO
>>> gp.train()
>>> likelihood.train()
>>> mll = VariationalELBO(likelihood, gp, num_data=y.numel())
>>> optimizer = torch.optim.Adam(
...     list(gp.parameters()) + list(likelihood.parameters()),
...     lr=0.001,
... )
>>> N = 1  # Set to 1 for faster training; increase for better performance
>>> for _ in range(N):
...     output = gp(x)
...     loss = -mll(output, y)
...     loss.backward()
...     optimizer.step()
...     optimizer.zero_grad()
>>> gp.eval()
>>> x_pred = torch.linspace(0, 2, 100).reshape(-1, 1)
>>> with torch.no_grad():
...     quantiles = gp.mean_quantiles_mc(x_pred)
>>> import matplotlib.pyplot as plt
>>> plt.scatter(x, y, c='gray', marker='.', alpha=0.1)
>>> plt.plot(x_range, true_quantiles, '--', c='k')
>>> plt.plot(x_pred, quantiles)
../_images/index-3.png
forward(function_samples)[source]#

Return the ALD distribution for the given function samples.

Parameters:
function_samplestorch.Tensor with shape (S, *B, N, Q)

The function samples drawn from the posterior distributions of quantile functions. S is the number of samples, Q is the number of quantiles, B is the batch shape, and N is the number of data points.

Returns:
MultitaskQuantileALD
expected_log_prob(observations, function_dist, *args, **kwargs)[source]#

Expected log probability of the observed data under the ALD likelihood.

Parameters:
observationstorch.Tensor with shape (*B, N)

The observed response variables.

function_disttorch.distributions.Distribution

The distribution of the function values at the input locations.

Returns:
torch.Tensor with shape (*B, Q)

The expected log probability of the observed data under the ALD likelihood.

Variational strategies#

Variational strategies for GPQR.

class gpytorch_qr.variational.CGLmcVariationalStrategy(base_variational_strategy, num_quantiles, num_latents, latent_dim=-1, jitter_val=None)[source]#

LMC variational strategy for the center-gap quantile regression model.

This class allows all gaps to be correlated.

Parameters:
base_variational_strategygpytorch.variational.VariationalStrategy

The base variational strategy to wrap.

num_quantilesint

The number of quantiles.

num_latentsint

The number of latent functions.

latent_dimint, optional

The dimension along which the latent functions are defined. Default is -1.

jitter_valfloat, optional

The jitter value to add to the covariance matrix for numerical stability.

Notes

This class modifies the standard LMC coefficients to fit the center-gap representation. The first latent function directly represents the central quantile, and it does not form any linear combinations with the other latent functions. The remaining latent functions are linearly combined to model the gap functions between quantiles.

Subclass can extend construct_lmc_mask() to further restrict the linear combinations, e.g., to model upper and lower gap functions separately by block diagonal matrices.

construct_lmc_mask(shape)[source]#

Construct a mask to restrict the LMC structure.

Parameters:
shapetorch.Size

The shape of the LMC coefficients. Must be ([batch_shape], T - 1, Q - 1), where T is the number of latent functions and Q is the number of quantiles.

Returns:
lmc_masktorch.Tensor with shape shape

A binary mask of the same shape as the LMC coefficients, where 1 indicates the positions of the LMC coefficients to be learned, and 0 indicates the positions of the LMC coefficients to be fixed at 0.

class gpytorch_qr.variational.CGBlkdiagLmcVariationalStrategy(base_variational_strategy, num_quantiles, num_latents, num_lower_quantiles, num_lower_latents, latent_dim=-1, jitter_val=None)[source]#

LMC variational strategy for the center-gap quantile regression model.

This class does not allow correlations between upper and lower gap functions.

Parameters:
base_variational_strategy
num_quantiles
num_latents
num_lower_quantilesint

The number of lower quantiles.

num_lower_latentsint

The number of lower latent functions.

latent_dim
jitter_val
construct_lmc_mask(shape)[source]#

Construct a mask to restrict the LMC structure.

Parameters:
shapetorch.Size

The shape of the LMC coefficients. Must be ([batch_shape], T - 1, Q - 1), where T is the number of latent functions and Q is the number of quantiles.

Returns:
lmc_masktorch.Tensor with shape shape

A binary mask of the same shape as the LMC coefficients, where 1 indicates the positions of the LMC coefficients to be learned, and 0 indicates the positions of the LMC coefficients to be fixed at 0.

Mean modules#

Mean modules.

class gpytorch_qr.means.CenterGapMean(center_mean, gap_mean, latent_dim)[source]#

Mean module for center-gap.

Parameters:
center_meangpytorch.means.Mean

Mean module for the central quantile. If GPQR treats quantiles as batches, this module should have batch shape (1, *B) where B is additional batch shape. If GPQR treats quantiles as task, this module should have batch shape (*B, 1).

gap_meangpytorch.means.Mean

Mean module for the quantile gaps. If GPQR treats quantiles as batches, this module should have batch shape (Q-1, *B) where Q is the number of quantiles and B is additional batch shape. If GPQR treats quantiles as task, this module should have batch shape (*B, L-1) where L is the number of latent GPs.

latent_dim{0, -1}

The dimension along which the latent GPs are represented in module batch shape. 0 if quantiles are batches, -1 if quantiles are tasks.

Notes

If GPQR treats quantiles as batches, input predictors are expected to have shape (1, *B, N, D). If GPQR treats quantiles as tasks, input predictors are expected to have shape (*B, 1, N, D). N is the number of data points and D is the number of input dimensions.

forward(x)[source]#

Compute the mean of center-gap representation.

Parameters:
xtorch.Tensor in shape (1, *B, N, D) or (*B, 1, N, D)

Utilities#

Utility functions.

gpytorch_qr.utils.centergap_to_quantiles(central, lower_gaps, upper_gaps, quantile_dim=-1)[source]#

Convert center-gap representation samples to quantiles.

Parameters:
centraltorch.Tensor with shape (…, 1, …)

The central quantile values.

lower_gapstorch.Tensor with shape (…, L, …)

Pre-transformed lower gap values.

upper_gapstorch.Tensor with shape (…, U, …)

Pre-transformed upper gap values.

quantile_dimint, default=-1

The dimension along which the quantiles are represented.

Returns:
quantilestorch.Tensor with shape (…, Q, …)

Quantile values. (Q = L + U + 1 at quantile_dim) The quantiles are ordered from lowest to highest along the quantile dimension.

class gpytorch_qr.utils.CenterGapToQuantileTransform(L, quantile_dim=-2)[source]#

Bijective transform from center-gap distribution to quantile distribution.

Parameters:
Lint

Number of lower quantile gaps in the center-gap representation.

quantile_dim{-1, -2}

The dimension along which the quantiles are represented.

Notes

If quantile_dim is -1, shape of input tensor is either (N, Q) or (S, N, Q). If quantile_dim is -2, shape of input tensor is either (Q, N) or (S, Q, N). Here, Q is the number of quantiles, N is the number of data points, and S is the number of samples.

The center-gap components along the quantile dimension is ordered as a central quantile, L lower pre-gaps, and U upper pre-gaps (Q = 1 + L + U).

log_abs_det_jacobian(x, y)[source]#

Computes the log det jacobian log |dy/dx| given input and output.

gpytorch_qr.utils.transform_centergap_posterior(posterior, L)[source]#

Convert the center-gap posterior to quantile posterior.

Parameters:
posteriorgpytorch.distributions.MultivariateNormal

The center-gap posterior distribution.

Lint

The number of lower quantiles in center-gap representation.

Returns:
quantile_posteriortorch.distributions.TransformedDistribution

Posterior over quantiles, obtained by applying CenterGapToQuantileTransform to a batched gpytorch.distributions.MultivariateNormal.

Notes

The quantile dimension consists of the central quantile, followed by L lower gaps and U upper gaps, where U = Q - L - 1.

On this page
Show Source