doTrials {mspath} | R Documentation |
Run a user-defined analysis repeatedly with user-defined sets of parameters. Stops when it detects a particular file's existence on the disk. Returns results and writes them to disk; also writes interim results to disk.
doTrials(generator, executor, stopFileName = "cancel", resultsFile = "trials.RData", nTrials = 10, nTime = 60 * 15, ...)
generator |
generator is a function taking the optional arguments ... and returning
a list (ordinarily a named list) of arguments for
executor . generator can return NULL to make
doTrials finish. |
executor |
executor is a function which takes arguments
produced by generator and performs the operation you are
testing. It returns an object containing the results you want to
preserve. The arguments are preserved automatically (see below). |
stopFileName |
Before every trial check if stopFileName
exists, and stop if it does. |
resultsFile |
When done, write the results of the trials to
resultsFile . Before then, this file name with .0 or .1
appended holds results so far. |
nTrials |
After every nTrials perform checkpointing. |
nTime |
After nTime seconds have passed, perform checkpointing. |
... |
Optional arguments passed to generator . |
Checkpointing writes the results so far to a file (named from
resultsFile
with, alternately, .0 and .1 appended) and prints a progress
message on the controlling terminal. Every time either nTrials
trials or nTime
seconds have passed since the last checkpoint a
new one will occur; whichever is hit first will control checkpointing.
A list
of trial
's, in the order they were
performed. trial
holds the arguments for the
trial (produced by generator
) and the results (produced by
executor
).
This function was designed to check the robustness of estimates. The generator can produce varying initial values, algorithmic choices, and tuning parameters, for example. The executor then takes those and calls to a function of interest.
The generator can produce values randomly, using a deterministic
scheme, or some mix of the two. The deterministic scheme may be
infinite. If not, generator
should return NULL
when it
is done, and the function will finish.
The executor is responsible for recording whatever information is of interest, apart from the arguments it receives. Relevant values could include convergence, parameter estimates, and run-time, among other things.
Results objects returned by estimators commonly include a lot of
information, including the arguments of the function call. To save
space (and thus time too) you may wish to record only the statistics
you want, or blank out the data elements you don't need. The
arguments returned by generator
are likely to be more compact
than those to the function being tested; for example,
generator
might return an index or name that executor
translates into a large set of parameters.
The checkpoint files are deleted on exit from this function.
Ross Boylan ross@biostat.ucsf.edu
## since the function runs forever, we don't want to really run it ## Not run: # create a set of parameters testgen <- function(...) { p <- rnorm(3, ...) list(a=p[1], b=p[2], c=p[3]) } # estimate a function using those parameters testexec <- function(a, b, c) { Sys.sleep(10) # some long computation a+b+c # result, sensitive to the parameters } ## Note that parameters, as used above, are not the same as the ## parameters of the inner function the executor runs. The parameters ## here might be the initial value for the parameters of the inner model. r <- doTrials(testgen, testexec, nTrials=20, nTime=30) ## End(Not run)