Function for Fitting a Spatial Factor Joint Species Distribution Model

The function sfJSDM fits a spatially-explicit joint species distribution model. This model does not explicitly account for imperfect detection (see sfMsPGOcc()). We use Polya-Gamma latent variables and a spatial factor modeling approach. Currently, models are implemented using a Nearest Neighbor Gaussian Process.

Usage

sfJSDM(formula, data, inits, priors, tuning, 
       cov.model = 'exponential', NNGP = TRUE, 
       n.neighbors = 15, search.type = 'cb', 
       std.by.sp = FALSE, n.factors, n.batch, 
       batch.length, accept.rate = 0.43, n.omp.threads = 1, 
       verbose = TRUE, n.report = 100, 
       n.burn = round(.10 * n.batch * batch.length), n.thin = 1, 
       n.chains = 1, k.fold, k.fold.threads = 1, k.fold.seed = 100, 
       k.fold.only = FALSE, monitors, keep.only.mean.95, shared.spatial = FALSE, ...)

Arguments

formula: a symbolic description of the model to be fit for the model using R's model syntax. Only right-hand side of formula is specified. See example below. Random intercepts are allowed using lme4 syntax (Bates et al. 2015).
data: a list containing data necessary for model fitting. Valid tags are y, covs, coords, range.ind, and grid.index. y is a two-dimensional array with first dimension equal to the number of species and second dimension equal to the number of sites. Note how this differs from other spOccupancy functions in that y does not have any replicate surveys. This is because sfJSDM does not account for imperfect detection. covs is a matrix or data frame containing the variables used in the model, with \(J\) rows for each column (variable). coords is a matrix of the observation coordinates used to estimate the SVCs for each site. coords has two columns for the easting and northing coordinate, respectively. Typically, each site in the data set will have it's own coordinate, such that coords is a \(J \times 2\) matrix and grid.index should not be specified. If you desire to estimate SVCs at some larger spatial level, e.g., if points fall within grid cells and you want to estimate an SVC for each grid cell instead of each point, coords can be specified as the coordinate for each grid cell. In such a case, grid.index is an indexing vector of length J, where each value of grid.index indicates the corresponding row in coords that the given site corresponds to. Note that spOccupancy assumes coordinates are specified in a projected coordinate system. range.ind is a matrix with rows corresponding to species and columns corresponding to sites, with each element taking value 1 if that site is within the range of the corresponding species and 0 if it is outside of the range. This matrix is not required, but it can be helpful to restrict the modeled area for each individual species to be within the realistic range of locations for that species when estimating the model parameters.
inits: a list with each tag corresponding to a parameter name. Valid tags are beta.comm, beta, tau.sq.beta, phi, lambda, sigma.sq.psi, and nu. nu is only specified if cov.model = "matern". sigma.sq.psi is only specified if random intercepts are included in formula. The value portion of each tag is the parameter's initial value. See priors description for definition of each parameter name. Additionally, the tag fix can be set to TRUE to fix the starting values across all chains. If fix is not specified (the default), starting values are varied randomly across chains.
priors: a list with each tag corresponding to a parameter name. Valid tags are beta.comm.normal, tau.sq.beta.ig, phi.unif, nu.unif, and sigma.sq.psi.ig. Community-level occurrence (beta.comm) regression coefficients are assumed to follow a normal distribution. The hyperparameters of the normal distribution are passed as a list of length two with the first and second elements corresponding to the mean and variance of the normal distribution, which are each specified as vectors of length equal to the number of coefficients to be estimated or of length one if priors are the same for all coefficients. If not specified, prior means are set to 0 and prior variances set to 2.73. Community-level variance parameters (tau.sq.beta) are assumed to follow an inverse Gamma distribution. The hyperparameters of the inverse gamma distribution are passed as a list of length two with the first and second elements corresponding to the shape and scale parameters, which are each specified as vectors of length equal to the number of coefficients to be estimated or a single value if priors are the same for all parameters. If not specified, prior shape and scale parameters are set to 0.1. If desired, the species-specific regression coefficients (beta) can also be estimated indepdendently by specifying the tag independent.betas = TRUE. If specified, this will not estimate species-specific coefficients as random effects from a common-community-level distribution, and rather the values of beta.comm and tau.sq.beta will be fixed at the specified initial values. This is equivalent to specifying a Gaussian, independent prior for each of the species-specific effects. The spatial factor model fits n.factors independent spatial processes. The spatial decay phi and smoothness nu parameters for each latent factor are assumed to follow Uniform distributions. The hyperparameters of the Uniform are passed as a list with two elements, with both elements being vectors of length n.factors corresponding to the lower and upper support, respectively, or as a single value if the same value is assigned for all factors. The priors for the factor loadings matrix lambda are fixed following the standard spatial factor model to ensure parameter identifiability (Christensen and Amemlya 2002). The upper triangular elements of the N x n.factors matrix are fixed at 0 and the diagonal elements are fixed at 1. The lower triangular elements are assigned a standard normal prior (i.e., mean 0 and variance 1). sigma.sq.psi is the random effect variance for any random effects, and is assumed to follow an inverse Gamma distribution. The hyperparameters of the inverse-Gamma distribution are passed as a list of length two with first and second elements corresponding to the shape and scale parameters, respectively, which are each specified as vectors of length equal to the number of random intercepts or of length one if priors are the same for all random effect variances.
tuning: a list with each tag corresponding to a parameter name. Valid tags are phi and nu. The value portion of each tag defines the initial variance of the adaptive sampler. We assume the initial variance of the adaptive sampler is the same for each species, although the adaptive sampler will adjust the tuning variances separately for each species. See Roberts and Rosenthal (2009) for details.
cov.model: a quoted keyword that specifies the covariance function used to model the spatial dependence structure among the observations. Supported covariance model key words are: "exponential", "matern", "spherical", and "gaussian".
NNGP: if TRUE, model is fit with an NNGP. If FALSE, a full Gaussian process is used. See Datta et al. (2016) and Finley et al. (2019) for more information. For spatial factor models, only NNGP = TRUE is currently supported.
n.neighbors: number of neighbors used in the NNGP. Only used if NNGP = TRUE. Datta et al. (2016) showed that 15 neighbors is usually sufficient, but that as few as 5 neighbors can be adequate for certain data sets, which can lead to even greater decreases in run time. We recommend starting with 15 neighbors (the default) and if additional gains in computation time are desired, subsequently compare the results with a smaller number of neighbors using WAIC or k-fold cross-validation.
search.type: a quoted keyword that specifies the type of nearest neighbor search algorithm. Supported method key words are: "cb" and "brute". The "cb" should generally be much faster. If locations do not have identical coordinate values on the axis used for the nearest neighbor ordering then "cb" and "brute" should produce identical neighbor sets. However, if there are identical coordinate values on the axis used for nearest neighbor ordering, then "cb" and "brute" might produce different, but equally valid, neighbor sets, e.g., if data are on a grid.
std.by.sp: a logical value indicating whether the covariates are standardized separately for each species within the corresponding range for each species (TRUE) or not (FALSE). Note that if range.ind is specified in data.list, this will result in the covariates being standardized differently for each species based on the sites where range.ind == 1 for that given species. If range.ind is not specified and std.by.sp = TRUE, this will simply be equivalent to standardizing the covariates across all locations prior to fitting the model. Note that the covariates in formula should still be standardized across all locations. This can be done either outside the function, or can be done by specifying scale() in the model formula around the continuous covariates.
n.factors: the number of factors to use in the spatial factor model approach. Typically, the number of factors is set to be small (e.g., 4-5) relative to the total number of species in the community, which will lead to substantial decreases in computation time. However, the value can be anywhere between 1 and N (the number of species in the community).
n.batch: the number of MCMC batches in each chain to run for the Adaptive MCMC sampler. See Roberts and Rosenthal (2009) for details.
batch.length: the length of each MCMC batch to run for the Adaptive MCMC sampler. See Roberts and Rosenthal (2009) for details.
accept.rate: target acceptance rate for Adaptive MCMC. Defaul is 0.43. See Roberts and Rosenthal (2009) for details.
n.omp.threads: a positive integer indicating the number of threads to use for SMP parallel processing. The package must be compiled for OpenMP support. For most Intel-based machines, we recommend setting n.omp.threads up to the number of hyperthreaded cores. Note, n.omp.threads > 1 might not work on some systems.
verbose: if TRUE, messages about data preparation, model specification, and progress of the sampler are printed to the screen. Otherwise, no messages are printed.
n.report: the interval to report Metropolis sampler acceptance and MCMC progress. Note this is specified in terms of batches and not overall samples for spatial models.
n.burn: the number of samples out of the total n.samples to discard as burn-in for each chain. By default, the first 10% of samples is discarded.
n.thin: the thinning interval for collection of MCMC samples. The thinning occurs after the n.burn samples are discarded. Default value is set to 1.
n.chains: the number of chains to run in sequence.
k.fold: specifies the number of k folds for cross-validation. If not specified as an argument, then cross-validation is not performed and k.fold.threads and k.fold.seed are ignored. In k-fold cross-validation, the data specified in data is randomly partitioned into k equal sized subsamples. Of the k subsamples, k - 1 subsamples are used to fit the model and the remaining k samples are used for prediction. The cross-validation process is repeated k times (the folds). As a scoring rule, we use the model deviance as described in Hooten and Hobbs (2015). Cross-validation is performed after the full model is fit using all the data. Cross-validation results are reported in the k.fold.deviance object in the return list.
k.fold.threads: number of threads to use for cross-validation. If k.fold.threads > 1 parallel processing is accomplished using the foreach and doParallel packages. Ignored if k.fold is not specified.
k.fold.seed: seed used to split data set into k.fold parts for k-fold cross-validation. Ignored if k.fold is not specified.
k.fold.only: a logical value indicating whether to only perform cross-validation (TRUE) or perform cross-validation after fitting the full model (FALSE). Default value is FALSE.
monitors: a character vector used to indicate if only a subset of the model model parameters are desired to be monitored. If posterior samples of all parameters are desired, then don't specify the argument (this is the default). When working with a large number of species and/or sites, the full model object can be quite large, and so this argument can be used to only return samples of specific parameters to help reduce the size of this resulting object. Valid tags include beta.comm, tau.sq.beta, beta, z, psi, lambda, theta, w, like (used for WAIC calculation), beta.star, sigma.sq.psi. Note that if all parameters are not returned, subsequent functions that require the model object may not work. We only recommend specifying this option when working with large data sets (e.g., > 100 species and/or > 10,000 sites).
keep.only.mean.95: not currently supported.
shared.spatial: a logical value used to specify whether a common spatial process should be estimated for all species instead of the factor modeling approach. If true, a spatial variance parameter sigma.sq is estimated for the model, which can be specified in the initial values and prior distributions (sigma.sq.ig).
...: currently no additional arguments

Note

Some of the underlying code used for generating random numbers from the Polya-Gamma distribution is taken from the pgdraw package written by Daniel F. Schmidt and Enes Makalic. Their code implements Algorithm 6 in PhD thesis of Jesse Bennett Windle (2013) https://repositories.lib.utexas.edu/handle/2152/21842.

References

Datta, A., S. Banerjee, A.O. Finley, and A.E. Gelfand. (2016) Hierarchical Nearest-Neighbor Gaussian process models for large geostatistical datasets. Journal of the American Statistical Association, doi:10.1080/01621459.2015.1044091 .

Finley, A.O., A. Datta, B.D. Cook, D.C. Morton, H.E. Andersen, and S. Banerjee. (2019) Efficient algorithms for Bayesian Nearest Neighbor Gaussian Processes. Journal of Computational and Graphical Statistics, doi:10.1080/10618600.2018.1537924 .

Finley, A. O., Datta, A., and Banerjee, S. (2020). spNNGP R package for nearest neighbor Gaussian process models. arXiv preprint arXiv:2001.09111.

Polson, N.G., J.G. Scott, and J. Windle. (2013) Bayesian Inference for Logistic Models Using Polya-Gamma Latent Variables. Journal of the American Statistical Association, 108:1339-1349.

Roberts, G.O. and Rosenthal J.S. (2009) Examples of adaptive MCMC. Journal of Computational and Graphical Statistics, 18(2):349-367.

Bates, Douglas, Martin Maechler, Ben Bolker, Steve Walker (2015). Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67(1), 1-48. doi:10.18637/jss.v067.i01 .

Hooten, M. B., and Hobbs, N. T. (2015). A guide to Bayesian model selection for ecologists. Ecological Monographs, 85(1), 3-28.

Christensen, W. F., and Amemiya, Y. (2002). Latent variable analysis of multivariate spatial data. Journal of the American Statistical Association, 97(457), 302-317.

Author

Jeffrey W. Doser doserjef@msu.edu,
Andrew O. Finley finleya@msu.edu

Value

An object of class sfJSDM that is a list comprised of:

beta.comm.samples: a coda object of posterior samples for the community level occurrence regression coefficients.
tau.sq.beta.samples: a coda object of posterior samples for the occurrence community variance parameters.
beta.samples: a coda object of posterior samples for the species level occurrence regression coefficients.
theta.samples: a coda object of posterior samples for the species level correlation parameters.
lambda.samples: a coda object of posterior samples for the latent spatial factor loadings.
psi.samples: a three-dimensional array of posterior samples for the latent occurrence probability values for each species.
w.samples: a three-dimensional array of posterior samples for the latent spatial random effects for each latent factor. Array dimensions correspond to MCMC sample, latent factor, and site. If shared.spatial = TRUE, this is still returned as a three-dimensional array where the first dimension is MCMC sample, second dimension is 1, and third dimension is site.
sigma.sq.psi.samples: a coda object of posterior samples for variances of random intercepts included in the occurrence portion of the model. Only included if random intercepts are specified in formula.
beta.star.samples: a coda object of posterior samples for the occurrence random effects. Only included if random intercepts are specified in formula.
like.samples: a three-dimensional array of posterior samples for the likelihood value associated with each site and species. Used for calculating WAIC.
rhat: a list of Gelman-Rubin diagnostic values for some of the model parameters.
ESS: a list of effective sample sizes for some of the model parameters.
run.time: MCMC sampler execution time reported using proc.time().
k.fold.deviance: vector of scoring rules (deviance) from k-fold cross-validation. A separate value is reported for each species. Only included if k.fold is specified in function call.

The return object will include additional objects used for subsequent prediction and/or model fit evaluation. Note that detection probability estimated values are not included in the model object, but can be extracted using fitted().

Examples

J.x <- 8
J.y <- 8
J <- J.x * J.y
n.rep<- sample(2:4, size = J, replace = TRUE)
N <- 6
# Community-level covariate effects
# Occurrence
beta.mean <- c(0.2)
p.occ <- length(beta.mean)
tau.sq.beta <- c(0.6)
# Detection
alpha.mean <- c(0)
tau.sq.alpha <- c(1)
p.det <- length(alpha.mean)
# Random effects
psi.RE <- list()
p.RE <- list()
# Draw species-level effects from community means.
beta <- matrix(NA, nrow = N, ncol = p.occ)
alpha <- matrix(NA, nrow = N, ncol = p.det)
for (i in 1:p.occ) {
  beta[, i] <- rnorm(N, beta.mean[i], sqrt(tau.sq.beta[i]))
}
for (i in 1:p.det) {
  alpha[, i] <- rnorm(N, alpha.mean[i], sqrt(tau.sq.alpha[i]))
}
alpha.true <- alpha
n.factors <- 3
phi <- rep(3 / .7, n.factors)
sigma.sq <- rep(2, n.factors)
nu <- rep(2, n.factors)

dat <- simMsOcc(J.x = J.x, J.y = J.y, n.rep = n.rep, N = N, beta = beta, alpha = alpha,
                psi.RE = psi.RE, p.RE = p.RE, sp = TRUE, sigma.sq = sigma.sq,
                phi = phi, nu = nu, cov.model = 'matern', factor.model = TRUE,
                n.factors = n.factors)
#> sigma.sq is specified but will be set to 1 for spatial latent factor model

pred.indx <- sample(1:J, round(J * .25), replace = FALSE)
y <- dat$y[, -pred.indx, , drop = FALSE]
# Occupancy covariates
X <- dat$X[-pred.indx, , drop = FALSE]
coords <- as.matrix(dat$coords[-pred.indx, , drop = FALSE])
# Prediction covariates
X.0 <- dat$X[pred.indx, , drop = FALSE]
coords.0 <- as.matrix(dat$coords[pred.indx, , drop = FALSE])
# Detection covariates
X.p <- dat$X.p[-pred.indx, , , drop = FALSE]

y <- apply(y, c(1, 2), max, na.rm = TRUE)
data.list <- list(y = y, coords = coords)
# Priors
prior.list <- list(beta.comm.normal = list(mean = 0, var = 2.72),
                   tau.sq.beta.ig = list(a = 0.1, b = 0.1),
                   nu.unif = list(0.5, 2.5))
# Starting values
inits.list <- list(beta.comm = 0,
                   beta = 0,
                   fix = TRUE,
                   tau.sq.beta = 1)
# Tuning
tuning.list <- list(phi = 1, nu = 0.25)

batch.length <- 25
n.batch <- 5
n.report <- 100
formula <- ~ 1

out <- sfJSDM(formula = formula,
              data = data.list,
              inits = inits.list,
              n.batch = n.batch,
              batch.length = batch.length,
              accept.rate = 0.43,
              priors = prior.list,
              cov.model = "matern",
              tuning = tuning.list,
              n.factors = 3,
              n.omp.threads = 1,
              verbose = TRUE,
              NNGP = TRUE,
              n.neighbors = 5,
              search.type = 'cb',
              n.report = 10,
              n.burn = 0,
              n.thin = 1,
              n.chains = 2)
#> ----------------------------------------
#> 	Preparing to run the model
#> ----------------------------------------
#> covariates (covs) not specified in data.
#> Assuming intercept only model.
#> No prior specified for phi.unif.
#> Setting uniform bounds based on the range of observed spatial coordinates.
#> phi is not specified in initial values.
#> Setting initial value to random values from the prior distribution
#> lambda is not specified in initial values.
#> Setting initial values of the lower triangle to 0
#> nu is not specified in initial values.
#> Setting initial values to random values from the prior distribution
#> w is not specified in initial values.
#> Setting initial value to 0
#> Fixing initial values across all chains
#> ----------------------------------------
#> 	Building the neighbor list
#> ----------------------------------------
#> ----------------------------------------
#> Building the neighbors of neighbors list
#> ----------------------------------------
#> ----------------------------------------
#> 	Model description
#> ----------------------------------------
#> Spatial Factor NNGP JSDM with Polya-Gamma latent
#> variable fit with 48 sites and 6 species.
#> 
#> Samples per chain: 125 (5 batches of length 25)
#> Burn-in: 0 
#> Thinning Rate: 1 
#> Number of Chains: 2 
#> Total Posterior Samples: 250 
#> 
#> Using the matern spatial correlation model.
#> 
#> Using 3 latent spatial factors.
#> Using 5 nearest neighbors.
#> 
#> Source compiled with OpenMP support and model fit using 1 thread(s).
#> 
#> Adaptive Metropolis with target acceptance rate: 43.0
#> ----------------------------------------
#> 	Chain 1
#> ----------------------------------------
#> Sampling ... 
#> Batch: 5 of 5, 100.00%
#> ----------------------------------------
#> 	Chain 2
#> ----------------------------------------
#> Sampling ... 
#> Batch: 5 of 5, 100.00%
summary(out)
#> 
#> Call:
#> sfJSDM(formula = formula, data = data.list, inits = inits.list, 
#>     priors = prior.list, tuning = tuning.list, cov.model = "matern", 
#>     NNGP = TRUE, n.neighbors = 5, search.type = "cb", n.factors = 3, 
#>     n.batch = n.batch, batch.length = batch.length, accept.rate = 0.43, 
#>     n.omp.threads = 1, verbose = TRUE, n.report = 10, n.burn = 0, 
#>     n.thin = 1, n.chains = 2)
#> 
#> Samples per Chain: 125
#> Burn-in: 0
#> Thinning Rate: 1
#> Number of Chains: 2
#> Total Posterior Samples: 250
#> Run Time (min): 0.0104
#> 
#> ----------------------------------------
#> 	Community Level
#> ----------------------------------------
#> Means (logit scale): 
#>                Mean     SD    2.5%     50%  97.5%   Rhat ESS
#> (Intercept) -0.9425 0.6909 -2.3163 -0.9389 0.3925 1.0301 250
#> 
#> Variances (logit scale): 
#>               Mean     SD   2.5%    50%   97.5%   Rhat ESS
#> (Intercept) 4.2243 6.6792 0.6289 2.8683 13.0285 1.6847 250
#> 
#> ----------------------------------------
#> 	Species Level
#> ----------------------------------------
#> Estimates (logit scale): 
#>                    Mean     SD    2.5%     50%   97.5%   Rhat ESS
#> (Intercept)-sp1 -0.7497 0.5126 -1.9118 -0.7306  0.1198 2.1432  19
#> (Intercept)-sp2  1.0455 0.5340  0.0193  1.0149  2.0767 1.3089  19
#> (Intercept)-sp3  0.1149 0.5661 -1.1997  0.1519  1.0835 1.2509  55
#> (Intercept)-sp4 -1.4247 0.5219 -2.4942 -1.3843 -0.4766 1.4222  61
#> (Intercept)-sp5 -3.6508 0.9878 -5.8141 -3.5509 -2.1360 1.9658  23
#> (Intercept)-sp6 -1.9058 0.5387 -3.0428 -1.8938 -0.9686 1.2109  60
#> 
#> ----------------------------------------
#> 	Spatial Covariance
#> ----------------------------------------
#>          Mean     SD    2.5%     50%   97.5%   Rhat ESS
#> phi-1 10.9066 4.4195  6.0673  9.5505 20.9051 8.2874   5
#> phi-2 16.5139 3.6911 10.2840 17.2128 20.9936 1.0491   7
#> phi-3 14.5076 4.1317  6.4794 15.3228 20.2422 2.2817   8
#> nu-1   1.7018 0.2489  1.2593  1.6679  2.1605 1.0012   6
#> nu-2   1.8207 0.3847  1.2394  1.7756  2.4006 1.6481   4
#> nu-3   2.1162 0.2544  1.5330  2.1884  2.4179 1.3254   4