brierScore {ensembleBMA}R Documentation

Brier Scores

Description

Computes climatology, ensemble, logistic, and BMA Brier scores given observation thresholds.

Usage

brierScore( fit, ensembleData, thresholds, dates = NULL, popData = NULL, 
                ...)

Arguments

fit An ensemble BMA model fit for ensembleData.
ensembleData An ensembleData object including ensemble forecasts and observations. It need not be the object used to form fit, although it must include the same ensemble members. If ensembleData includes dates, they must be consistent with fit and dates. If ensembleData does not include dates, they will be inferred from the fit and the dates argument.
thresholds One or more threshold values for the Brier score computations.
dates The dates for which the Brier score will be computed. These dates must be consistent with fit and ensembleData. The default is to use all of the dates in fit.
popData For gamma0 model fits, there is an additional popData argument for providing predictors in the logistic regression for probability of zero precipitation. If popData was supplied to obtain in the modeling for fit, then popData of the same kind must be supplied here.
... Included for generic function compatibility.

Details

There can be a lot of warnings due to logistic fitting near the extremes.

Value

A data frame giving the climatology (empirical distribution of the verifying observations), ensemble (voting), logistic (coefficients determined by logistic regression on the training data), and BMA Brier scores for the specified thresholds.

References

G. W. Brier, Verification of forecasts expressed in terms of probability, Monthly Weather Review, 78:1–3 (1950).

T. Gneiting and A. E. Raftery, Strictly proper scoring rules, prediction and estimation, Journal of the American Statistical Association 102:359–378 (2007).

C. Fraley, A. E. Raftery, T. Gneiting and J. M. Sloughter, ensembleBMA: An R Package for Probabilistic Forecasting using Ensembles and Bayesian Model Averaging, Technical Report No. 516, Department of Statistics, University of Washington, August 2007.

See Also

ensembleBMA

Examples

  data(prcpTest)
                                      
  labels <- c("CENT","AVN","CMCG","ETA","GASP","JMA","NGPS","TCWB","UKMO")
  prcpTestData <- ensembleData( forecasts = prcpTest[ ,labels],
                          dates = prcpTest$date, observations = prcpTest$obs)
## Not run: 
  prcpTestFit <- ensembleBMAgamma0(prcpTestData)
## End(Not run)
 
  hist(prcpTestData$obs)

  brierScore(prcpTestFit, prcpTestData, thresholds = c(0, 5, 10, 15, 20))

[Package ensembleBMA version 2.1 Index]