verification: Proper verification rules
Loading required package: fields
Loading required package: spam
Loading required package: grid
Attaching package: 'grid'
The following object is masked from 'package:timetools': unit
Spam version 1.3-0 (2015-10-24) is loaded. Type 'help( Spam)' or 'demo( spam)' for a short introduction and overview of this package. Help for individual functions is also obtained by adding the suffix '.spam' to the function name, e.g. 'help( chol.spam)'.
Attaching package: 'spam'
The following objects are masked from 'package:base': backsolve, forwardsolve
Loading required package: maps
# maps v3.1: updated 'world': all lakes moved to separate new # # 'lakes' database. Type '?world' or 'news(package="maps")'. #
Attaching package: 'maps'
The following object is masked from 'package:fma': ozone
Loading required package: boot
Loading required package: CircStats
Loading required package: MASS
Attaching package: 'MASS'
The following objects are masked from 'package:fma': cement, housing, petrol
Loading required package: dtw
Loading required package: proxy
Attaching package: 'proxy'
The following object is masked from 'package:spam': as.matrix
The following objects are masked from 'package:stats': as.dist, dist
The following object is masked from 'package:base': as.matrix
Loaded dtw v1.18-1. See ?dtw for help, citation("dtw") for use in publication.
Attaching package: 'verification'
The following object is masked from 'package:tseries': value
||check loss function|
||Conditional Quantile Plot|
||Ranked Probability Score|
||Decompostion of Continuous Ranked Probability Score|
||Discrimination plot dataset.|
||Fractional Skill Score|
||Linear Error in Probability Space (LEPS)|
||Add lines to ROC or attribute diagrams|
||Skill score with measurement error.|
||Multiple Contingency Table Statistics|
||Probability of precipitation (pop) data.|
||An ensemble of precipitation forecasts|
||Time Series Prediction Comparison Test|
||Probablisitic Forecast Dataset.|
||Converts continuous probability values into binned discrete probability forecasts.|
||Quantile Reliability Plot|
||Convert Continuous Forecast Values to Discrete Forecast Values.|
||Reduced centered random variable|
||Area under curve (AUC) calculation for Response Operating Characteristic curve.|
||Relative operating characteristic curve.|
||Ranked Probability Score|
||Verification statistics for a 2 by 2 Contingency Table|
||Percentile bootstrap for 2 by 2 table|
||Forecast Value Function|
> obs<- c(28, 72, 23, 2680) > A <- verify(obs, pred = NULL, frcst.type = "binary", obs.type = "binary")
 " Assume data entered as c(n11, n01, n10, n00) Obs*Forecast"
The forecasts are binary, the observations are binary. The contingency table for the forecast [,1] [,2] [1,] 28 72 [2,] 23 2680 PODy = 0.5489 Std. Err. for POD = 0.06967 TS = 0.2276 Std. Err. for TS = 0.03278 ETS = 0.216 Std. Err. for ETS = 0.03411 FAR = 0.7199 Std. Err. for FAR = 0.03469 HSS = 0.3553 Std. Err. for HSS = 0.04614 PC = 0.9661 Std. Err. for PC = 0.003245 BIAS = 1.961 Odds Ratio = 45.31 Log Odds Ratio = 3.814 Std. Err. for log Odds Ratio = 0.3057 Odds Ratio Skill Score = 0.9568 Std. Err. for Odds Ratio Skill Score = Extreme Dependency Score (EDS) = 0.7396 Std. Err. for EDS = 0.04794 Symmetric Extreme Dependency Score (SEDS) = 0.5935 Std. Err. for SEDS = 0.04391 Extremal Dependence Index (EDI) = 0.7173 Std. Err. for EDI = 0.06167 Symmetric Extremal Dependence Index (SEDI) = 0.7527 Std. Err. for SEDI = 0.06043
> obs <- round(runif(100, 1,5)) > pred <- round(runif(100, 1,5)) > > A <- verify(obs, pred, frcst.type = "cat", obs.type = "cat" ) > summary(A)
The forecasts are categorical, the observations are categorical. Percent Correct = 0.15 Heidke Skill Score = -0.0835 Pierce Skill Score = -0.0862 Gerrity Score = -0.0184 Statistics considering each category in turn. Threat Score 0.0435 0.05 0.0769 0.128 0.087 Bias by cat. Percent correct by cat. 0.78 0.62 0.52 0.59 0.79 Hit Rate (POD) by cat. 0.111 0.08 0.121 0.273 0.182 False Alarm Rate by cat. 0.154 0.2 0.284 0.321 0.135 False Alarm Ratio by cat. 0.933 0.882 0.826 0.806 0.857