Last updated on 2019-04-22 07:46:34 CEST.
| Flavor | Version | Tinstall | Tcheck | Ttotal | Status | Flags |
|---|---|---|---|---|---|---|
| r-devel-linux-x86_64-debian-clang | 3.0.5 | 106.03 | 79.46 | 185.49 | OK | |
| r-devel-linux-x86_64-debian-gcc | 3.0.5 | 87.28 | 64.95 | 152.23 | ERROR | |
| r-devel-linux-x86_64-fedora-clang | 3.0.5 | 264.06 | OK | |||
| r-devel-linux-x86_64-fedora-gcc | 3.0.5 | 244.05 | ERROR | |||
| r-devel-windows-ix86+x86_64 | 3.0.5 | 337.00 | 158.00 | 495.00 | ERROR | |
| r-patched-linux-x86_64 | 3.0.5 | 99.03 | 76.91 | 175.94 | ERROR | |
| r-patched-solaris-x86 | 3.0.5 | 267.40 | OK | |||
| r-release-linux-x86_64 | 3.0.5 | 102.39 | 75.25 | 177.64 | OK | |
| r-release-windows-ix86+x86_64 | 3.0.5 | 260.00 | 154.00 | 414.00 | WARN | |
| r-release-osx-x86_64 | 3.0.5 | OK | ||||
| r-oldrel-windows-ix86+x86_64 | 3.0.5 | 378.00 | 202.00 | 580.00 | OK | |
| r-oldrel-osx-x86_64 | 3.0.5 | OK |
Version: 3.0.5
Check: tests
Result: ERROR
Running ‘testthat.R’ [17s/23s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(bigKRLS)
Loading required package: bigmemory
bigKRLS is authored by Pete Mohanty (Stanford University) and Robert Shaffer (University of Pennsylvania) under License GPL 3.
To get started, check out vignette("bigKRLS_basics").
If you use this package in your research, kindly cite:
Pete Mohanty and Robert Shaffer. 2018. 'Messy Data, Robust Inference? Navigating Obstacles to Inference with bigKRLS.' Political Analysis. Cambridge University Press. DOI=10.1017/pan.2018.33. pages 1-18.
>
> # have to disable R_TESTS environment variable or parallel::makePSOCKcluster hangs.
> Sys.setenv("R_TESTS" = "")
>
>
> test_check("bigKRLS")
..........................
All done. You may wish to use summary() for more detail, predict() for out-of-sample forecasts, or shiny.bigKRLS() to interact with results. For an alternative approach, see help(crossvalidate.bigKRLS). Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
MODEL SUMMARY:
lambda: 0.1306
N: 32
N Effective: 15.17858
R2: 0.9405
R2AME**: 0.8477
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
cyl -0.0733 0.0765 -0.9581 0.3806
disp -0.0037 0.0022 -1.6485 0.1582
hp -0.0145 0.0040 -3.5903 0.0148
drat 1.2786 0.6303 2.0287 0.0963
wt -1.4940 0.3115 -4.7968 0.0045
qsec 0.3678 0.1741 2.1123 0.0864
vs* 1.2772 0.6234 2.0488 0.0938
am* 0.9967 0.5954 1.6739 0.1530
gear 1.0847 0.2867 3.7828 0.0120
carb -0.5951 0.1518 -3.9200 0.0104
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
cyl -0.3219 -0.1071 -0.0182 0.0065 0.0406
disp -0.0108 -0.0080 -0.0040 -0.0011 0.0044
hp -0.0372 -0.0258 -0.0138 -0.0009 0.0038
drat -0.4892 0.6973 1.1559 2.1076 3.4894
wt -3.5756 -1.9904 -1.2308 -0.7641 -0.2853
qsec -0.1250 0.0504 0.2099 0.5900 1.2423
vs* -0.3757 0.8754 1.2843 1.8167 2.6408
am* -0.6929 0.3432 0.8767 1.8618 2.7406
gear -0.4504 -0.1866 0.8420 1.9432 3.4149
carb -1.1902 -1.0159 -0.5844 -0.2336 0.0107
(*) Reported average and percentiles of dy/dx is for discrete change of the dummy variable from min to max (usually 0 to 1)).
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Saving model estimates to:
bigKRLS_test_results
0 matrices saved as big matrices (base R save() may be used safely in this case too).
Smaller, base R elements of the outputted object saved: bigKRLS_test_results/estimates.RData
Total file size approximately 0 megabytes.
.
model estimates will be saved to:
bigKRLS_test_bigmemory_results
................
saving vcovmatyhat to /home/hornik/tmp/R.check/r-devel-gcc/Work/PKGS/bigKRLS.Rcheck/tests/testthat
vcovmatyhat successfully saved to disk (and removed from memory for speed).
..........
saving output to bigKRLS_test_bigmemory_results
writing bigKRLS_test_bigmemory_results/X.txt ...
writing bigKRLS_test_bigmemory_results/K.txt ...
writing bigKRLS_test_bigmemory_results/vcov.est.c.txt ...
writing bigKRLS_test_bigmemory_results/derivatives.txt ...
5 matrices saved as big matrices.
to reload, use syntax like:
load.bigKRLS("/home/hornik/tmp/R.check/r-devel-gcc/Work/PKGS/bigKRLS.Rcheck/tests/testthat/bigKRLS_test_bigmemory_results")
or
load.bigKRLS("/home/hornik/tmp/R.check/r-devel-gcc/Work/PKGS/bigKRLS.Rcheck/tests/testthat/bigKRLS_test_bigmemory_results", newname="my_estimates")
base R elements of the output saved to estimates.RData.
Total file size approximately 0 megabytes.
All done. You may wish to use summary() for more detail, predict() for out-of-sample forecasts, or shiny.bigKRLS() to interact with results. For an alternative approach, see help(crossvalidate.bigKRLS). Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Reading from K.txt
Reading from X.txt
Reading from derivatives.txt
Reading from vcov.est.c.txt
NOTE: vcov.est.fitted not found in .RData or in big matrix file, vcov.est.fitted.txt .
............................
Overview of Model Performance
N: 32
Seed: 123
In Sample Out of Sample
Mean Squared Error (Full Model) 1.125 7.532
Mean Squared Error (Average Marginal Effects Only) 282.697 302.770
Pseudo-R^2 (Full Model) 0.971 0.919
Pseudo-R^2 (Average Marginal Effects Only) 0.860 0.866
N 26.000 6.000
Summary of Training Model:
MODEL SUMMARY:
lambda: 0.0749
N: 26
N Effective: 8.693671
R2: 0.9713
R2AME**: 0.8595
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
cyl -0.0308 0.0738 -0.4177 NaN
disp -0.0013 0.0019 -0.6889 NaN
hp -0.0214 0.0036 -5.9445 NaN
drat 0.7105 0.5552 1.2797 NaN
wt -2.1883 0.2698 -8.1106 NaN
qsec 0.4290 0.1427 3.0069 NaN
vs* 1.6259 0.5522 2.9445 NaN
am* 0.8126 0.5254 1.5466 NaN
gear 1.2481 0.2464 5.0661 NaN
carb -0.6583 0.1271 -5.1802 NaN
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
cyl -0.2678 -0.0583 -0.0249 0.0639 0.1224
disp -0.0157 -0.0097 -0.0034 0.0065 0.0148
hp -0.0532 -0.0355 -0.0238 -0.0039 0.0019
drat -0.8121 0.1667 0.6667 1.2357 3.1801
wt -4.8466 -2.6870 -1.8917 -1.4949 -0.6539
qsec -0.1957 0.1062 0.3431 0.7820 1.2412
vs* 0.1074 1.0071 1.6507 2.2429 3.3543
am* -0.8303 -0.0115 0.7399 1.5441 3.1487
gear -0.4625 -0.0895 1.0600 2.1423 3.4811
carb -1.4582 -1.2111 -0.5200 -0.2011 -0.0732
(*) Reported average and percentiles of dy/dx is for discrete change of the dummy variable from min to max (usually 0 to 1)).
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
......................................
.....................
....................
....................
..................
..................
..................
................
....................
.....................
....................
....................
Overview of Model Performance
N: 250
Kfolds: 4
Seed: 1234
Fold 1 Fold 2 Fold 3 Fold 4
MSE (In Sample) 0.734 0.743 0.695 0.686
MSE (Out of Sample) 1.276 0.873 1.195 1.226
MSE AME (In Sample) 2.695 1.984 2.540 2.209
MSE AME (Out of Sample) 3.177 1.677 3.045 1.936
R2 (In Sample) 0.774 0.791 0.787 0.787
R2 (Out of Sample) 0.648 0.681 0.667 0.659
R2 AME (In Sample) 0.742 0.731 0.732 0.750
R2 AME (Out of Sample) 0.708 0.769 0.734 0.682
MSE denotes Mean Squared Error. AME implies calculations done with Average Marginal Effects only.
Summary of Training Model1:
MODEL SUMMARY:
lambda: 0.899
N: 187
N Effective: 156.3024
R2: 0.7736
R2AME**: 0.7417
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.9589 0.1905 5.0346 0
x2 1.4519 0.1891 7.6767 0
x3 2.0993 0.1798 11.6737 0
x4 2.7440 0.1757 15.6168 0
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.6189 -0.0087 1.0668 1.8372 3.3296
x2 -1.0785 0.6842 1.7075 2.3498 3.0562
x3 -0.3417 1.2507 2.1789 3.1527 4.4687
x4 0.7443 1.9765 2.8437 3.5838 4.4058
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Summary of Training Model2:
MODEL SUMMARY:
lambda: 0.5778
N: 188
N Effective: 151.2168
R2: 0.7908
R2AME**: 0.7309
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.8270 0.2062 4.0110 1e-04
x2 1.5459 0.2060 7.5055 0e+00
x3 2.4301 0.2042 11.9033 0e+00
x4 3.1351 0.2036 15.3949 0e+00
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.6407 -0.2399 0.9918 1.9215 3.0752
x2 -0.4728 0.7705 1.4049 2.2533 3.8459
x3 0.0212 1.5036 2.6631 3.3750 4.4612
x4 0.4207 2.1119 3.3870 4.2646 5.2462
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Summary of Training Model3:
MODEL SUMMARY:
lambda: 0.6276
N: 188
N Effective: 152.5316
R2: 0.787
R2AME**: 0.7324
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.5377 0.2009 2.6770 0.0083
x2 1.4111 0.2024 6.9715 0.0000
x3 2.3743 0.1888 12.5778 0.0000
x4 2.9933 0.1911 15.6618 0.0000
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.8595 -0.2222 0.6688 1.4931 2.4555
x2 -0.8720 0.5198 1.2902 2.5101 3.4258
x3 0.0256 1.2932 2.2869 3.3879 4.8906
x4 0.0714 2.1910 3.0890 4.0984 5.1052
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Summary of Training Model4:
MODEL SUMMARY:
lambda: 0.6245
N: 187
N Effective: 151.8663
R2: 0.7868
R2AME**: 0.7503
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.8540 0.1975 4.3235 0
x2 1.6994 0.1972 8.6181 0
x3 2.1450 0.1909 11.2334 0
x4 3.0553 0.1950 15.6664 0
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.1126 0.2378 0.9325 1.5662 3.0208
x2 -0.2084 1.0239 1.7979 2.5718 3.3397
x3 -0.1887 1.3657 2.3700 2.9835 3.8805
x4 0.6929 2.1339 3.1462 4.0975 5.0257
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
── 1. Failure: Kfolds crossvalidation works (@test_basic_usage.R#148) ─────────
kcv$folds not equal to kcvbig$folds.
2/250 mismatches (average diff: 2)
[1] 2 - 4 == -2
[204] 4 - 2 == 2
── 2. Failure: Kfolds crossvalidation works (@test_basic_usage.R#150) ─────────
sum(kcv$fold_2$tested$predicted) not equal to sum(kcvbig$fold_2$tested$predicted[]).
1/1 mismatches
[1] 296 - 297 == -1.35
── 3. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#157
kcv$R2_is not equal to kcvbig$R2_is.
2/4 mismatches (average diff: 0.00769)
[2] 0.791 - 0.790 == 0.000627
[4] 0.787 - 0.772 == 0.014752
── 4. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#158
kcv$R2_oos not equal to kcvbig$R2_oos.
2/4 mismatches (average diff: 0.00807)
[2] 0.681 - 0.690 == -0.00893
[4] 0.659 - 0.652 == 0.00720
── 5. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#159
kcv$R2AME_is not equal to kcvbig$R2AME_is.
2/4 mismatches (average diff: 0.000358)
[2] 0.731 - 0.731 == 6.97e-05
[4] 0.750 - 0.750 == 6.47e-04
── 6. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#160
kcv$MSE_is not equal to kcvbig$MSE_is.
2/4 mismatches (average diff: 0.0249)
[2] 0.743 - 0.746 == -0.00287
[4] 0.686 - 0.733 == -0.04684
── 7. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#166
kcv$MSE_oos not equal to kcvbig$MSE_oos.
2/4 mismatches (average diff: 0.0209)
[2] 0.873 - 0.849 == 0.0238
[4] 1.226 - 1.244 == -0.0180
── 8. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#167
kcv$MSE_AME_is not equal to kcvbig$MSE_AME_is.
2/4 mismatches (average diff: 0.297)
[2] 1.98 - 2.00 == -0.0115
[4] 2.21 - 2.79 == -0.5824
── 9. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#168
kcv$R2AME_oos not equal to kcvbig$R2AME_oos.
2/4 mismatches (average diff: 0.00162)
[2] 0.769 - 0.769 == -0.00024
[4] 0.682 - 0.685 == -0.00300
── 10. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#16
kcv$MSE_AME_oos not equal to kcvbig$MSE_AME_oos.
2/4 mismatches (average diff: 0.304)
[2] 1.68 - 1.61 == 0.0707
[4] 1.94 - 2.47 == -0.5368
══ testthat results ═══════════════════════════════════════════════════════════
OK: 18 SKIPPED: 0 FAILED: 10
1. Failure: Kfolds crossvalidation works (@test_basic_usage.R#148)
2. Failure: Kfolds crossvalidation works (@test_basic_usage.R#150)
3. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#157)
4. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#158)
5. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#159)
6. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#160)
7. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#166)
8. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#167)
9. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#168)
10. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#169)
Error: testthat unit tests failed
Execution halted
Flavor: r-devel-linux-x86_64-debian-gcc
Version: 3.0.5
Check: tests
Result: ERROR
Running ‘testthat.R’ [23s/25s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(bigKRLS)
Loading required package: bigmemory
bigKRLS is authored by Pete Mohanty (Stanford University) and Robert Shaffer (University of Pennsylvania) under License GPL 3.
To get started, check out vignette("bigKRLS_basics").
If you use this package in your research, kindly cite:
Pete Mohanty and Robert Shaffer. 2018. 'Messy Data, Robust Inference? Navigating Obstacles to Inference with bigKRLS.' Political Analysis. Cambridge University Press. DOI=10.1017/pan.2018.33. pages 1-18.
>
> # have to disable R_TESTS environment variable or parallel::makePSOCKcluster hangs.
> Sys.setenv("R_TESTS" = "")
>
>
> test_check("bigKRLS")
..........................
All done. You may wish to use summary() for more detail, predict() for out-of-sample forecasts, or shiny.bigKRLS() to interact with results. For an alternative approach, see help(crossvalidate.bigKRLS). Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
MODEL SUMMARY:
lambda: 0.1306
N: 32
N Effective: 15.17858
R2: 0.9405
R2AME**: 0.8477
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
cyl -0.0733 0.0765 -0.9581 0.3806
disp -0.0037 0.0022 -1.6485 0.1582
hp -0.0145 0.0040 -3.5903 0.0148
drat 1.2786 0.6303 2.0287 0.0963
wt -1.4940 0.3115 -4.7968 0.0045
qsec 0.3678 0.1741 2.1123 0.0864
vs* 1.2772 0.6234 2.0488 0.0938
am* 0.9967 0.5954 1.6739 0.1530
gear 1.0847 0.2867 3.7828 0.0120
carb -0.5951 0.1518 -3.9200 0.0104
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
cyl -0.3219 -0.1071 -0.0182 0.0065 0.0406
disp -0.0108 -0.0080 -0.0040 -0.0011 0.0044
hp -0.0372 -0.0258 -0.0138 -0.0009 0.0038
drat -0.4892 0.6973 1.1559 2.1076 3.4894
wt -3.5756 -1.9904 -1.2308 -0.7641 -0.2853
qsec -0.1250 0.0504 0.2099 0.5900 1.2423
vs* -0.3757 0.8754 1.2843 1.8167 2.6408
am* -0.6929 0.3432 0.8767 1.8618 2.7406
gear -0.4504 -0.1866 0.8420 1.9432 3.4149
carb -1.1902 -1.0159 -0.5844 -0.2336 0.0107
(*) Reported average and percentiles of dy/dx is for discrete change of the dummy variable from min to max (usually 0 to 1)).
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Saving model estimates to:
bigKRLS_test_results
0 matrices saved as big matrices (base R save() may be used safely in this case too).
Smaller, base R elements of the outputted object saved: bigKRLS_test_results/estimates.RData
Total file size approximately 0 megabytes.
.
model estimates will be saved to:
bigKRLS_test_bigmemory_results
................
saving vcovmatyhat to /data/gannet/ripley/R/packages/tests-devel/bigKRLS.Rcheck/tests/testthat
vcovmatyhat successfully saved to disk (and removed from memory for speed).
..........
saving output to bigKRLS_test_bigmemory_results
writing bigKRLS_test_bigmemory_results/X.txt ...
writing bigKRLS_test_bigmemory_results/K.txt ...
writing bigKRLS_test_bigmemory_results/vcov.est.c.txt ...
writing bigKRLS_test_bigmemory_results/derivatives.txt ...
5 matrices saved as big matrices.
to reload, use syntax like:
load.bigKRLS("/data/gannet/ripley/R/packages/tests-devel/bigKRLS.Rcheck/tests/testthat/bigKRLS_test_bigmemory_results")
or
load.bigKRLS("/data/gannet/ripley/R/packages/tests-devel/bigKRLS.Rcheck/tests/testthat/bigKRLS_test_bigmemory_results", newname="my_estimates")
base R elements of the output saved to estimates.RData.
Total file size approximately 0 megabytes.
All done. You may wish to use summary() for more detail, predict() for out-of-sample forecasts, or shiny.bigKRLS() to interact with results. For an alternative approach, see help(crossvalidate.bigKRLS). Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Reading from K.txt
Reading from X.txt
Reading from derivatives.txt
Reading from vcov.est.c.txt
NOTE: vcov.est.fitted not found in .RData or in big matrix file, vcov.est.fitted.txt .
............................
Overview of Model Performance
N: 32
Seed: 123
In Sample Out of Sample
Mean Squared Error (Full Model) 1.125 7.532
Mean Squared Error (Average Marginal Effects Only) 282.697 302.770
Pseudo-R^2 (Full Model) 0.971 0.919
Pseudo-R^2 (Average Marginal Effects Only) 0.860 0.866
N 26.000 6.000
Summary of Training Model:
MODEL SUMMARY:
lambda: 0.0749
N: 26
N Effective: 8.693671
R2: 0.9713
R2AME**: 0.8595
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
cyl -0.0308 0.0738 -0.4177 NaN
disp -0.0013 0.0019 -0.6889 NaN
hp -0.0214 0.0036 -5.9445 NaN
drat 0.7105 0.5552 1.2797 NaN
wt -2.1883 0.2698 -8.1106 NaN
qsec 0.4290 0.1427 3.0069 NaN
vs* 1.6259 0.5522 2.9445 NaN
am* 0.8126 0.5254 1.5466 NaN
gear 1.2481 0.2464 5.0661 NaN
carb -0.6583 0.1271 -5.1802 NaN
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
cyl -0.2678 -0.0583 -0.0249 0.0639 0.1224
disp -0.0157 -0.0097 -0.0034 0.0065 0.0148
hp -0.0532 -0.0355 -0.0238 -0.0039 0.0019
drat -0.8121 0.1667 0.6667 1.2357 3.1801
wt -4.8466 -2.6870 -1.8917 -1.4949 -0.6539
qsec -0.1957 0.1062 0.3431 0.7820 1.2412
vs* 0.1074 1.0071 1.6507 2.2429 3.3543
am* -0.8303 -0.0115 0.7399 1.5441 3.1487
gear -0.4625 -0.0895 1.0600 2.1423 3.4811
carb -1.4582 -1.2111 -0.5200 -0.2011 -0.0732
(*) Reported average and percentiles of dy/dx is for discrete change of the dummy variable from min to max (usually 0 to 1)).
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
......................................
.....................
....................
....................
................
.................
................
................
......................
......................
......................
....................
Overview of Model Performance
N: 250
Kfolds: 4
Seed: 1234
Fold 1 Fold 2 Fold 3 Fold 4
MSE (In Sample) 0.734 0.743 0.695 0.686
MSE (Out of Sample) 1.276 0.873 1.195 1.226
MSE AME (In Sample) 2.695 1.984 2.540 2.209
MSE AME (Out of Sample) 3.177 1.677 3.045 1.936
R2 (In Sample) 0.774 0.791 0.787 0.787
R2 (Out of Sample) 0.648 0.681 0.667 0.659
R2 AME (In Sample) 0.742 0.731 0.732 0.750
R2 AME (Out of Sample) 0.708 0.769 0.734 0.682
MSE denotes Mean Squared Error. AME implies calculations done with Average Marginal Effects only.
Summary of Training Model1:
MODEL SUMMARY:
lambda: 0.899
N: 187
N Effective: 156.3024
R2: 0.7736
R2AME**: 0.7417
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.9589 0.1905 5.0346 0
x2 1.4519 0.1891 7.6767 0
x3 2.0993 0.1798 11.6737 0
x4 2.7440 0.1757 15.6168 0
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.6189 -0.0087 1.0668 1.8372 3.3296
x2 -1.0785 0.6842 1.7075 2.3498 3.0562
x3 -0.3417 1.2507 2.1789 3.1527 4.4687
x4 0.7443 1.9765 2.8437 3.5838 4.4058
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Summary of Training Model2:
MODEL SUMMARY:
lambda: 0.5778
N: 188
N Effective: 151.2168
R2: 0.7908
R2AME**: 0.7309
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.8270 0.2062 4.0110 1e-04
x2 1.5459 0.2060 7.5055 0e+00
x3 2.4301 0.2042 11.9033 0e+00
x4 3.1351 0.2036 15.3949 0e+00
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.6407 -0.2399 0.9918 1.9215 3.0752
x2 -0.4728 0.7705 1.4049 2.2533 3.8459
x3 0.0212 1.5036 2.6631 3.3750 4.4612
x4 0.4207 2.1119 3.3870 4.2646 5.2462
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Summary of Training Model3:
MODEL SUMMARY:
lambda: 0.6276
N: 188
N Effective: 152.5316
R2: 0.787
R2AME**: 0.7324
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.5377 0.2009 2.6770 0.0083
x2 1.4111 0.2024 6.9715 0.0000
x3 2.3743 0.1888 12.5778 0.0000
x4 2.9933 0.1911 15.6618 0.0000
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.8595 -0.2222 0.6688 1.4931 2.4555
x2 -0.8720 0.5198 1.2902 2.5101 3.4258
x3 0.0256 1.2932 2.2869 3.3879 4.8906
x4 0.0714 2.1910 3.0890 4.0984 5.1052
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Summary of Training Model4:
MODEL SUMMARY:
lambda: 0.6245
N: 187
N Effective: 151.8663
R2: 0.7868
R2AME**: 0.7503
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.8540 0.1975 4.3235 0
x2 1.6994 0.1972 8.6181 0
x3 2.1450 0.1909 11.2334 0
x4 3.0553 0.1950 15.6664 0
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.1126 0.2378 0.9325 1.5662 3.0208
x2 -0.2084 1.0239 1.7979 2.5718 3.3397
x3 -0.1887 1.3657 2.3700 2.9835 3.8805
x4 0.6929 2.1339 3.1462 4.0975 5.0257
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
── 1. Failure: Kfolds crossvalidation works (@test_basic_usage.R#148) ─────────
kcv$folds not equal to kcvbig$folds.
183/250 mismatches (average diff: 1.73)
[1] 2 - 3 == -1
[2] 3 - 2 == 1
[3] 2 - 4 == -2
[4] 4 - 2 == 2
[5] 2 - 3 == -1
[10] 3 - 2 == 1
[12] 2 - 4 == -2
[13] 4 - 2 == 2
[16] 2 - 4 == -2
...
── 2. Failure: Kfolds crossvalidation works (@test_basic_usage.R#149) ─────────
sum(kcv$fold_1$tested$newdata) not equal to sum(kcvbig$fold_1$tested$newdata[]).
1/1 mismatches
[1] 134 - 123 == 11.3
── 3. Failure: Kfolds crossvalidation works (@test_basic_usage.R#150) ─────────
sum(kcv$fold_2$tested$predicted) not equal to sum(kcvbig$fold_2$tested$predicted[]).
1/1 mismatches
[1] 296 - 312 == -15.9
── 4. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#157
kcv$R2_is not equal to kcvbig$R2_is.
4/4 mismatches (average diff: 0.0139)
[1] 0.774 - 0.791 == -0.0173
[2] 0.791 - 0.778 == 0.0125
[3] 0.787 - 0.799 == -0.0115
[4] 0.787 - 0.773 == 0.0141
── 5. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#158
kcv$R2_oos not equal to kcvbig$R2_oos.
4/4 mismatches (average diff: 0.042)
[1] 0.648 - 0.729 == -0.0816
[2] 0.681 - 0.707 == -0.0257
[3] 0.667 - 0.626 == 0.0411
[4] 0.659 - 0.679 == -0.0197
── 6. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#159
kcv$R2AME_is not equal to kcvbig$R2AME_is.
4/4 mismatches (average diff: 0.0128)
[1] 0.742 - 0.727 == 0.01453
[2] 0.731 - 0.730 == 0.00107
[3] 0.732 - 0.756 == -0.02316
[4] 0.750 - 0.738 == 0.01224
── 7. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#160
kcv$MSE_is not equal to kcvbig$MSE_is.
4/4 mismatches (average diff: 0.0738)
[1] 0.734 - 0.588 == 0.1464
[2] 0.743 - 0.759 == -0.0159
[3] 0.695 - 0.739 == -0.0438
[4] 0.686 - 0.776 == -0.0893
── 8. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#166
kcv$MSE_oos not equal to kcvbig$MSE_oos.
4/4 mismatches (average diff: 0.176)
[1] 1.276 - 1.397 == -0.1210
[2] 0.873 - 0.911 == -0.0378
[3] 1.195 - 0.873 == 0.3216
[4] 1.226 - 1.000 == 0.2255
── 9. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#167
kcv$MSE_AME_is not equal to kcvbig$MSE_AME_is.
4/4 mismatches (average diff: 0.514)
[1] 2.69 - 1.84 == 0.8533
[2] 1.98 - 1.93 == 0.0559
[3] 2.54 - 2.20 == 0.3431
[4] 2.21 - 3.01 == -0.8045
── 10. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#16
kcv$R2AME_oos not equal to kcvbig$R2AME_oos.
4/4 mismatches (average diff: 0.0468)
[1] 0.708 - 0.759 == -0.0505
[2] 0.769 - 0.757 == 0.0125
[3] 0.734 - 0.658 == 0.0768
[4] 0.682 - 0.729 == -0.0473
── 11. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#16
kcv$MSE_AME_oos not equal to kcvbig$MSE_AME_oos.
4/4 mismatches (average diff: 0.662)
[1] 3.18 - 2.40 == 0.77362
[2] 1.68 - 1.67 == 0.00907
[3] 3.05 - 2.15 == 0.89983
[4] 1.94 - 2.90 == -0.96532
══ testthat results ═══════════════════════════════════════════════════════════
OK: 18 SKIPPED: 0 FAILED: 11
1. Failure: Kfolds crossvalidation works (@test_basic_usage.R#148)
2. Failure: Kfolds crossvalidation works (@test_basic_usage.R#149)
3. Failure: Kfolds crossvalidation works (@test_basic_usage.R#150)
4. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#157)
5. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#158)
6. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#159)
7. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#160)
8. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#166)
9. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#167)
1. ...
Error: testthat unit tests failed
Execution halted
Flavor: r-devel-linux-x86_64-fedora-gcc
Version: 3.0.5
Check: running tests for arch ‘i386’
Result: ERROR
Running 'testthat.R' [24s]
Running the tests in 'tests/testthat.R' failed.
Complete output:
> library(testthat)
> library(bigKRLS)
Loading required package: bigmemory
bigKRLS is authored by Pete Mohanty (Stanford University) and Robert Shaffer (University of Pennsylvania) under License GPL 3.
To get started, check out vignette("bigKRLS_basics").
If you use this package in your research, kindly cite:
Pete Mohanty and Robert Shaffer. 2018. 'Messy Data, Robust Inference? Navigating Obstacles to Inference with bigKRLS.' Political Analysis. Cambridge University Press. DOI=10.1017/pan.2018.33. pages 1-18.
>
> # have to disable R_TESTS environment variable or parallel::makePSOCKcluster hangs.
> Sys.setenv("R_TESTS" = "")
>
>
> test_check("bigKRLS")
..........................
All done. You may wish to use summary() for more detail, predict() for out-of-sample forecasts, or shiny.bigKRLS() to interact with results. For an alternative approach, see help(crossvalidate.bigKRLS). Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
MODEL SUMMARY:
lambda: 0.1306
N: 32
N Effective: 15.17858
R2: 0.9405
R2AME**: 0.8477
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
cyl -0.0733 0.0765 -0.9581 0.3806
disp -0.0037 0.0022 -1.6485 0.1582
hp -0.0145 0.0040 -3.5903 0.0148
drat 1.2786 0.6303 2.0287 0.0963
wt -1.4940 0.3115 -4.7968 0.0045
qsec 0.3678 0.1741 2.1123 0.0864
vs* 1.2772 0.6234 2.0488 0.0938
am* 0.9967 0.5954 1.6739 0.1530
gear 1.0847 0.2867 3.7828 0.0120
carb -0.5951 0.1518 -3.9200 0.0104
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
cyl -0.3219 -0.1071 -0.0182 0.0065 0.0406
disp -0.0108 -0.0080 -0.0040 -0.0011 0.0044
hp -0.0372 -0.0258 -0.0138 -0.0009 0.0038
drat -0.4892 0.6973 1.1559 2.1076 3.4894
wt -3.5756 -1.9904 -1.2308 -0.7641 -0.2853
qsec -0.1250 0.0504 0.2099 0.5900 1.2423
vs* -0.3757 0.8754 1.2843 1.8167 2.6408
am* -0.6929 0.3432 0.8767 1.8618 2.7406
gear -0.4504 -0.1866 0.8420 1.9432 3.4149
carb -1.1902 -1.0159 -0.5844 -0.2336 0.0107
(*) Reported average and percentiles of dy/dx is for discrete change of the dummy variable from min to max (usually 0 to 1)).
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Saving model estimates to:
bigKRLS_test_results
0 matrices saved as big matrices (base R save() may be used safely in this case too).
Smaller, base R elements of the outputted object saved: bigKRLS_test_results/estimates.RData
Total file size approximately 0 megabytes.
.
model estimates will be saved to:
bigKRLS_test_bigmemory_results
................
saving vcovmatyhat to d:/Rcompile/CRANpkg/local/3.6/bigKRLS.Rcheck/tests_i386/testthat
vcovmatyhat successfully saved to disk (and removed from memory for speed).
..........
saving output to bigKRLS_test_bigmemory_results
writing bigKRLS_test_bigmemory_results/X.txt ...
writing bigKRLS_test_bigmemory_results/K.txt ...
writing bigKRLS_test_bigmemory_results/vcov.est.c.txt ...
writing bigKRLS_test_bigmemory_results/derivatives.txt ...
5 matrices saved as big matrices.
to reload, use syntax like:
load.bigKRLS("d:\RCompile\CRANpkg\local\3.6\bigKRLS.Rcheck\tests_i386\testthat\bigKRLS_test_bigmemory_results")
or
load.bigKRLS("d:\RCompile\CRANpkg\local\3.6\bigKRLS.Rcheck\tests_i386\testthat\bigKRLS_test_bigmemory_results", newname="my_estimates")
base R elements of the output saved to estimates.RData.
Total file size approximately 0 megabytes.
All done. You may wish to use summary() for more detail, predict() for out-of-sample forecasts, or shiny.bigKRLS() to interact with results. For an alternative approach, see help(crossvalidate.bigKRLS). Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Reading from K.txt
Reading from X.txt
Reading from derivatives.txt
Reading from vcov.est.c.txt
NOTE: vcov.est.fitted not found in .RData or in big matrix file, vcov.est.fitted.txt .
............................
Overview of Model Performance
N: 32
Seed: 123
In Sample Out of Sample
Mean Squared Error (Full Model) 1.125 7.532
Mean Squared Error (Average Marginal Effects Only) 282.697 302.770
Pseudo-R^2 (Full Model) 0.971 0.919
Pseudo-R^2 (Average Marginal Effects Only) 0.860 0.866
N 26.000 6.000
Summary of Training Model:
MODEL SUMMARY:
lambda: 0.0749
N: 26
N Effective: 8.693671
R2: 0.9713
R2AME**: 0.8595
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
cyl -0.0308 0.0738 -0.4177 NaN
disp -0.0013 0.0019 -0.6889 NaN
hp -0.0214 0.0036 -5.9445 NaN
drat 0.7105 0.5552 1.2797 NaN
wt -2.1883 0.2698 -8.1106 NaN
qsec 0.4290 0.1427 3.0069 NaN
vs* 1.6259 0.5522 2.9445 NaN
am* 0.8126 0.5254 1.5466 NaN
gear 1.2481 0.2464 5.0661 NaN
carb -0.6583 0.1271 -5.1802 NaN
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
cyl -0.2678 -0.0583 -0.0249 0.0639 0.1224
disp -0.0157 -0.0097 -0.0034 0.0065 0.0148
hp -0.0532 -0.0355 -0.0238 -0.0039 0.0019
drat -0.8121 0.1667 0.6667 1.2357 3.1801
wt -4.8466 -2.6870 -1.8917 -1.4949 -0.6539
qsec -0.1957 0.1062 0.3431 0.7820 1.2412
vs* 0.1074 1.0071 1.6507 2.2429 3.3543
am* -0.8303 -0.0115 0.7399 1.5441 3.1487
gear -0.4625 -0.0895 1.0600 2.1423 3.4811
carb -1.4582 -1.2111 -0.5200 -0.2011 -0.0732
(*) Reported average and percentiles of dy/dx is for discrete change of the dummy variable from min to max (usually 0 to 1)).
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
.......................-- 1. Failure: crossvalidation function works (@test_basic_usage.R#133) -------
cv$pseudoR2_oos not equal to cv_noderivs$pseudoR2_oos.
1/1 mismatches
[1] 0.919 - 0.498 == 0.421
....................
.....................
....................
....................
................
.................
................
................
....................
.....................
....................
....................
Overview of Model Performance
N: 250
Kfolds: 4
Seed: 1234
Fold 1 Fold 2 Fold 3 Fold 4
MSE (In Sample) 0.734 0.746 0.695 0.733
MSE (Out of Sample) 1.276 0.849 1.195 1.244
MSE AME (In Sample) 2.695 1.995 2.540 2.791
MSE AME (Out of Sample) 3.177 1.606 3.045 2.473
R2 (In Sample) 0.774 0.790 0.787 0.772
R2 (Out of Sample) 0.648 0.690 0.667 0.652
R2 AME (In Sample) 0.742 0.731 0.732 0.750
R2 AME (Out of Sample) 0.708 0.769 0.734 0.685
MSE denotes Mean Squared Error. AME implies calculations done with Average Marginal Effects only.
Summary of Training Model1:
MODEL SUMMARY:
lambda: 0.899
N: 187
N Effective: 156.3024
R2: 0.7736
R2AME**: 0.7417
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.9589 0.1905 5.0346 0
x2 1.4519 0.1891 7.6767 0
x3 2.0993 0.1798 11.6737 0
x4 2.7440 0.1757 15.6168 0
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.6189 -0.0087 1.0668 1.8372 3.3296
x2 -1.0785 0.6842 1.7075 2.3498 3.0562
x3 -0.3417 1.2507 2.1789 3.1527 4.4687
x4 0.7443 1.9765 2.8437 3.5838 4.4058
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Summary of Training Model2:
MODEL SUMMARY:
lambda: 0.5778
N: 188
N Effective: 151.1978
R2: 0.7902
R2AME**: 0.7308
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.7756 0.2061 3.7637 2e-04
x2 1.5853 0.2077 7.6327 0e+00
x3 2.4326 0.2053 11.8491 0e+00
x4 3.1498 0.2043 15.4139 0e+00
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.6653 -0.3349 0.9298 1.8751 3.0522
x2 -0.3976 0.8668 1.3767 2.3247 3.8451
x3 0.0959 1.5179 2.6396 3.3611 4.4275
x4 0.3016 2.0586 3.4568 4.3551 5.2430
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Summary of Training Model3:
MODEL SUMMARY:
lambda: 0.6276
N: 188
N Effective: 152.5316
R2: 0.787
R2AME**: 0.7324
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.5377 0.2009 2.6770 0.0083
x2 1.4111 0.2024 6.9715 0.0000
x3 2.3743 0.1888 12.5778 0.0000
x4 2.9933 0.1911 15.6618 0.0000
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.8595 -0.2222 0.6688 1.4931 2.4555
x2 -0.8720 0.5198 1.2902 2.5101 3.4258
x3 0.0256 1.2932 2.2869 3.3879 4.8906
x4 0.0714 2.1910 3.0890 4.0984 5.1052
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Summary of Training Model4:
MODEL SUMMARY:
lambda: 0.9487
N: 187
N Effective: 157.1787
R2: 0.7721
R2AME**: 0.7497
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.8128 0.1854 4.3831 0
x2 1.5713 0.1857 8.4612 0
x3 2.0508 0.1754 11.6948 0
x4 2.8821 0.1817 15.8639 0
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.2934 0.0750 0.9622 1.5400 2.8176
x2 -0.5135 0.7996 1.6848 2.4301 3.0530
x3 -0.3719 1.2459 2.2440 2.9152 3.8446
x4 0.4845 2.0039 2.9042 3.9110 4.8957
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
-- 2. Failure: Kfolds crossvalidation works (@test_basic_usage.R#148) ---------
kcv$folds not equal to kcvbig$folds.
2/250 mismatches (average diff: 2)
[1] 4 - 2 == 2
[204] 2 - 4 == -2
-- 3. Failure: Kfolds crossvalidation works (@test_basic_usage.R#150) ---------
sum(kcv$fold_2$tested$predicted) not equal to sum(kcvbig$fold_2$tested$predicted[]).
1/1 mismatches
[1] 297 - 296 == 1.35
-- 4. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#157
kcv$R2_is not equal to kcvbig$R2_is.
2/4 mismatches (average diff: 0.00769)
[2] 0.790 - 0.791 == -0.000627
[4] 0.772 - 0.787 == -0.014752
-- 5. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#158
kcv$R2_oos not equal to kcvbig$R2_oos.
2/4 mismatches (average diff: 0.00807)
[2] 0.690 - 0.681 == 0.00893
[4] 0.652 - 0.659 == -0.00720
-- 6. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#159
kcv$R2AME_is not equal to kcvbig$R2AME_is.
2/4 mismatches (average diff: 0.000358)
[2] 0.731 - 0.731 == -6.97e-05
[4] 0.750 - 0.750 == -6.47e-04
-- 7. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#160
kcv$MSE_is not equal to kcvbig$MSE_is.
2/4 mismatches (average diff: 0.0249)
[2] 0.746 - 0.743 == 0.00287
[4] 0.733 - 0.686 == 0.04684
-- 8. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#166
kcv$MSE_oos not equal to kcvbig$MSE_oos.
2/4 mismatches (average diff: 0.0209)
[2] 0.849 - 0.873 == -0.0238
[4] 1.244 - 1.226 == 0.0180
-- 9. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#167
kcv$MSE_AME_is not equal to kcvbig$MSE_AME_is.
2/4 mismatches (average diff: 0.297)
[2] 2.00 - 1.98 == 0.0115
[4] 2.79 - 2.21 == 0.5824
-- 10. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#16
kcv$R2AME_oos not equal to kcvbig$R2AME_oos.
2/4 mismatches (average diff: 0.00162)
[2] 0.769 - 0.769 == 0.00024
[4] 0.685 - 0.682 == 0.00300
-- 11. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#16
kcv$MSE_AME_oos not equal to kcvbig$MSE_AME_oos.
2/4 mismatches (average diff: 0.304)
[2] 1.61 - 1.68 == -0.0707
[4] 2.47 - 1.94 == 0.5368
== testthat results ===========================================================
OK: 12 SKIPPED: 0 FAILED: 11
1. Failure: crossvalidation function works (@test_basic_usage.R#133)
2. Failure: Kfolds crossvalidation works (@test_basic_usage.R#148)
3. Failure: Kfolds crossvalidation works (@test_basic_usage.R#150)
4. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#157)
5. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#158)
6. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#159)
7. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#160)
8. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#166)
9. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#167)
1. ...
Error: testthat unit tests failed
Execution halted
Flavor: r-devel-windows-ix86+x86_64
Version: 3.0.5
Check: tests
Result: ERROR
Running ‘testthat.R’ [18s/19s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(bigKRLS)
Loading required package: bigmemory
bigKRLS is authored by Pete Mohanty (Stanford University) and Robert Shaffer (University of Pennsylvania) under License GPL 3.
To get started, check out vignette("bigKRLS_basics").
If you use this package in your research, kindly cite:
Pete Mohanty and Robert Shaffer. 2018. 'Messy Data, Robust Inference? Navigating Obstacles to Inference with bigKRLS.' Political Analysis. Cambridge University Press. DOI=10.1017/pan.2018.33. pages 1-18.
>
> # have to disable R_TESTS environment variable or parallel::makePSOCKcluster hangs.
> Sys.setenv("R_TESTS" = "")
>
>
> test_check("bigKRLS")
..........................
All done. You may wish to use summary() for more detail, predict() for out-of-sample forecasts, or shiny.bigKRLS() to interact with results. For an alternative approach, see help(crossvalidate.bigKRLS). Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
MODEL SUMMARY:
lambda: 0.1306
N: 32
N Effective: 15.17858
R2: 0.9405
R2AME**: 0.8477
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
cyl -0.0733 0.0765 -0.9581 0.3806
disp -0.0037 0.0022 -1.6485 0.1582
hp -0.0145 0.0040 -3.5903 0.0148
drat 1.2786 0.6303 2.0287 0.0963
wt -1.4940 0.3115 -4.7968 0.0045
qsec 0.3678 0.1741 2.1123 0.0864
vs* 1.2772 0.6234 2.0488 0.0938
am* 0.9967 0.5954 1.6739 0.1530
gear 1.0847 0.2867 3.7828 0.0120
carb -0.5951 0.1518 -3.9200 0.0104
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
cyl -0.3219 -0.1071 -0.0182 0.0065 0.0406
disp -0.0108 -0.0080 -0.0040 -0.0011 0.0044
hp -0.0372 -0.0258 -0.0138 -0.0009 0.0038
drat -0.4892 0.6973 1.1559 2.1076 3.4894
wt -3.5756 -1.9904 -1.2308 -0.7641 -0.2853
qsec -0.1250 0.0504 0.2099 0.5900 1.2423
vs* -0.3757 0.8754 1.2843 1.8167 2.6408
am* -0.6929 0.3432 0.8767 1.8618 2.7406
gear -0.4504 -0.1866 0.8420 1.9432 3.4149
carb -1.1902 -1.0159 -0.5844 -0.2336 0.0107
(*) Reported average and percentiles of dy/dx is for discrete change of the dummy variable from min to max (usually 0 to 1)).
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Saving model estimates to:
bigKRLS_test_results
0 matrices saved as big matrices (base R save() may be used safely in this case too).
Smaller, base R elements of the outputted object saved: bigKRLS_test_results/estimates.RData
Total file size approximately 0 megabytes.
.
model estimates will be saved to:
bigKRLS_test_bigmemory_results
................
saving vcovmatyhat to /home/hornik/tmp/R.check/r-patched-gcc/Work/PKGS/bigKRLS.Rcheck/tests/testthat
vcovmatyhat successfully saved to disk (and removed from memory for speed).
..........
saving output to bigKRLS_test_bigmemory_results
writing bigKRLS_test_bigmemory_results/X.txt ...
writing bigKRLS_test_bigmemory_results/K.txt ...
writing bigKRLS_test_bigmemory_results/vcov.est.c.txt ...
writing bigKRLS_test_bigmemory_results/derivatives.txt ...
5 matrices saved as big matrices.
to reload, use syntax like:
load.bigKRLS("/home/hornik/tmp/R.check/r-patched-gcc/Work/PKGS/bigKRLS.Rcheck/tests/testthat/bigKRLS_test_bigmemory_results")
or
load.bigKRLS("/home/hornik/tmp/R.check/r-patched-gcc/Work/PKGS/bigKRLS.Rcheck/tests/testthat/bigKRLS_test_bigmemory_results", newname="my_estimates")
base R elements of the output saved to estimates.RData.
Total file size approximately 0 megabytes.
All done. You may wish to use summary() for more detail, predict() for out-of-sample forecasts, or shiny.bigKRLS() to interact with results. For an alternative approach, see help(crossvalidate.bigKRLS). Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Reading from K.txt
Reading from X.txt
Reading from derivatives.txt
Reading from vcov.est.c.txt
NOTE: vcov.est.fitted not found in .RData or in big matrix file, vcov.est.fitted.txt .
..........................
Overview of Model Performance
N: 32
Seed: 123
In Sample Out of Sample
Mean Squared Error (Full Model) 3.067 4.512
Mean Squared Error (Average Marginal Effects Only) 241.673 204.649
Pseudo-R^2 (Full Model) 0.917 0.850
Pseudo-R^2 (Average Marginal Effects Only) 0.842 0.888
N 26.000 6.000
Summary of Training Model:
MODEL SUMMARY:
lambda: 0.2203
N: 26
N Effective: 13.31767
R2: 0.917
R2AME**: 0.8424
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
cyl -0.0707 0.0728 -0.9707 0.3970
disp -0.0023 0.0022 -1.0483 0.3648
hp -0.0150 0.0038 -3.9729 0.0236
drat 0.8467 0.6580 1.2868 0.2806
wt -1.3875 0.2990 -4.6396 0.0151
qsec 0.3421 0.1685 2.0300 0.1265
vs* 1.4893 0.6266 2.3770 0.0896
am* 1.2569 0.6508 1.9311 0.1401
gear 1.0324 0.3206 3.2204 0.0421
carb -0.5432 0.1383 -3.9272 0.0244
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
cyl -0.3023 -0.0914 -0.0251 -0.0045 0.0251
disp -0.0101 -0.0065 -0.0036 0.0028 0.0062
hp -0.0310 -0.0223 -0.0151 -0.0081 0.0005
drat -0.5234 0.0534 0.5663 1.3290 3.2306
wt -3.1320 -1.8435 -1.2400 -0.7973 -0.1654
qsec -0.2059 0.1193 0.3080 0.4881 0.9398
vs* 0.2663 0.9610 1.4825 2.0927 2.6374
am* -0.5025 0.7251 1.1666 1.8925 3.0413
gear -0.4105 -0.1462 1.0044 1.9403 2.9580
carb -1.1097 -0.6895 -0.5497 -0.3194 -0.0618
(*) Reported average and percentiles of dy/dx is for discrete change of the dummy variable from min to max (usually 0 to 1)).
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
....................................
.....................
....................
....................
................
.................
................
................
....................
.....................
....................
....................
Overview of Model Performance
N: 250
Kfolds: 4
Seed: 1234
Fold 1 Fold 2 Fold 3 Fold 4
MSE (In Sample) 0.734 0.743 0.695 0.686
MSE (Out of Sample) 1.276 0.873 1.195 1.226
MSE AME (In Sample) 2.695 1.984 2.540 2.209
MSE AME (Out of Sample) 3.177 1.677 3.045 1.936
R2 (In Sample) 0.774 0.791 0.787 0.787
R2 (Out of Sample) 0.648 0.681 0.667 0.659
R2 AME (In Sample) 0.742 0.731 0.732 0.750
R2 AME (Out of Sample) 0.708 0.769 0.734 0.682
MSE denotes Mean Squared Error. AME implies calculations done with Average Marginal Effects only.
Summary of Training Model1:
MODEL SUMMARY:
lambda: 0.899
N: 187
N Effective: 156.3024
R2: 0.7736
R2AME**: 0.7417
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.9589 0.1905 5.0346 0
x2 1.4519 0.1891 7.6767 0
x3 2.0993 0.1798 11.6737 0
x4 2.7440 0.1757 15.6168 0
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.6189 -0.0087 1.0668 1.8372 3.3296
x2 -1.0785 0.6842 1.7075 2.3498 3.0562
x3 -0.3417 1.2507 2.1789 3.1527 4.4687
x4 0.7443 1.9765 2.8437 3.5838 4.4058
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Summary of Training Model2:
MODEL SUMMARY:
lambda: 0.5778
N: 188
N Effective: 151.2168
R2: 0.7908
R2AME**: 0.7309
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.8270 0.2062 4.0110 1e-04
x2 1.5459 0.2060 7.5055 0e+00
x3 2.4301 0.2042 11.9033 0e+00
x4 3.1351 0.2036 15.3949 0e+00
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.6407 -0.2399 0.9918 1.9215 3.0752
x2 -0.4728 0.7705 1.4049 2.2533 3.8459
x3 0.0212 1.5036 2.6631 3.3750 4.4612
x4 0.4207 2.1119 3.3870 4.2646 5.2462
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Summary of Training Model3:
MODEL SUMMARY:
lambda: 0.6276
N: 188
N Effective: 152.5316
R2: 0.787
R2AME**: 0.7324
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.5377 0.2009 2.6770 0.0083
x2 1.4111 0.2024 6.9715 0.0000
x3 2.3743 0.1888 12.5778 0.0000
x4 2.9933 0.1911 15.6618 0.0000
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.8595 -0.2222 0.6688 1.4931 2.4555
x2 -0.8720 0.5198 1.2902 2.5101 3.4258
x3 0.0256 1.2932 2.2869 3.3879 4.8906
x4 0.0714 2.1910 3.0890 4.0984 5.1052
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
Summary of Training Model4:
MODEL SUMMARY:
lambda: 0.6245
N: 187
N Effective: 151.8663
R2: 0.7868
R2AME**: 0.7503
Average Marginal Effects:
Estimate Std. Error t value Pr(>|t|)
x1 0.8540 0.1975 4.3235 0
x2 1.6994 0.1972 8.6181 0
x3 2.1450 0.1909 11.2334 0
x4 3.0553 0.1950 15.6664 0
Percentiles of Marginal Effects:
5% 25% 50% 75% 95%
x1 -1.1126 0.2378 0.9325 1.5662 3.0208
x2 -0.2084 1.0239 1.7979 2.5718 3.3397
x3 -0.1887 1.3657 2.3700 2.9835 3.8805
x4 0.6929 2.1339 3.1462 4.0975 5.0257
(**) Pseudo-R^2 computed using only the Average Marginal Effects.
You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
── 1. Failure: Kfolds crossvalidation works (@test_basic_usage.R#148) ─────────
kcv$folds not equal to kcvbig$folds.
2/250 mismatches (average diff: 2)
[1] 2 - 4 == -2
[204] 4 - 2 == 2
── 2. Failure: Kfolds crossvalidation works (@test_basic_usage.R#150) ─────────
sum(kcv$fold_2$tested$predicted) not equal to sum(kcvbig$fold_2$tested$predicted[]).
1/1 mismatches
[1] 296 - 297 == -1.35
── 3. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#157
kcv$R2_is not equal to kcvbig$R2_is.
2/4 mismatches (average diff: 0.00769)
[2] 0.791 - 0.790 == 0.000627
[4] 0.787 - 0.772 == 0.014752
── 4. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#158
kcv$R2_oos not equal to kcvbig$R2_oos.
2/4 mismatches (average diff: 0.00807)
[2] 0.681 - 0.690 == -0.00893
[4] 0.659 - 0.652 == 0.00720
── 5. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#159
kcv$R2AME_is not equal to kcvbig$R2AME_is.
2/4 mismatches (average diff: 0.000358)
[2] 0.731 - 0.731 == 6.97e-05
[4] 0.750 - 0.750 == 6.47e-04
── 6. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#160
kcv$MSE_is not equal to kcvbig$MSE_is.
2/4 mismatches (average diff: 0.0249)
[2] 0.743 - 0.746 == -0.00287
[4] 0.686 - 0.733 == -0.04684
── 7. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#166
kcv$MSE_oos not equal to kcvbig$MSE_oos.
2/4 mismatches (average diff: 0.0209)
[2] 0.873 - 0.849 == 0.0238
[4] 1.226 - 1.244 == -0.0180
── 8. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#167
kcv$MSE_AME_is not equal to kcvbig$MSE_AME_is.
2/4 mismatches (average diff: 0.297)
[2] 1.98 - 2.00 == -0.0115
[4] 2.21 - 2.79 == -0.5824
── 9. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#168
kcv$R2AME_oos not equal to kcvbig$R2AME_oos.
2/4 mismatches (average diff: 0.00162)
[2] 0.769 - 0.769 == -0.00024
[4] 0.682 - 0.685 == -0.00300
── 10. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#16
kcv$MSE_AME_oos not equal to kcvbig$MSE_AME_oos.
2/4 mismatches (average diff: 0.304)
[2] 1.68 - 1.61 == 0.0707
[4] 1.94 - 2.47 == -0.5368
══ testthat results ═══════════════════════════════════════════════════════════
OK: 18 SKIPPED: 0 FAILED: 10
1. Failure: Kfolds crossvalidation works (@test_basic_usage.R#148)
2. Failure: Kfolds crossvalidation works (@test_basic_usage.R#150)
3. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#157)
4. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#158)
5. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#159)
6. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#160)
7. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#166)
8. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#167)
9. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#168)
10. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#169)
Error: testthat unit tests failed
Execution halted
Flavor: r-patched-linux-x86_64
Version: 3.0.5
Check: whether package can be installed
Result: WARN
Found the following significant warnings:
Warning: S3 methods '[.fun_list', '[.grouped_df', 'all.equal.tbl_df', 'anti_join.data.frame', 'anti_join.tbl_df', 'arrange.data.frame', 'arrange.default', 'arrange.grouped_df', 'arrange.tbl_df', 'arrange_.data.frame', 'arrange_.tbl_df', 'as.data.frame.grouped_df', 'as.data.frame.rowwise_df', 'as.data.frame.tbl_cube', 'as.table.tbl_cube', 'as.tbl.data.frame', 'as.tbl.tbl', 'as.tbl_cube.array', 'as.tbl_cube.data.frame', 'as.tbl_cube.matrix', 'as.tbl_cube.table', 'as_tibble.grouped_df', 'as_tibble.tbl_cube', 'auto_copy.tbl_cube', 'auto_copy.tbl_df', 'cbind.grouped_df', 'collapse.data.frame', 'collect.data.frame', 'common_by.NULL', 'common_by.character', 'common_by.default', 'common_by.list', 'compute.data.frame', 'copy_to.DBIConnection', 'copy_to.src_local', 'default_missing.data.frame', 'default_missing.default', 'dim.tbl_cube', 'distinct.data.frame', 'distinct.default', 'distinct.grouped_df', 'distinct.tbl_df', 'distinct_.data.frame', 'distinct_.grouped_df', 'distinct_.tbl_df', 'do.NULL', 'do.da [... truncated]
Flavor: r-release-windows-ix86+x86_64