doTrials {mspath}R Documentation

Run trials until told to stop

Description

Run a user-defined analysis repeatedly with user-defined sets of parameters. Stops when it detects a particular file's existence on the disk. Returns results and writes them to disk; also writes interim results to disk.

Usage

doTrials(generator, executor, stopFileName = "cancel", resultsFile = "trials.RData", nTrials = 10, nTime = 60 * 15, ...)

Arguments

generator generator is a function taking the optional arguments ... and returning a list (ordinarily a named list) of arguments for executor. generator can return NULL to make doTrials finish.
executor executor is a function which takes arguments produced by generator and performs the operation you are testing. It returns an object containing the results you want to preserve. The arguments are preserved automatically (see below).
stopFileName Before every trial check if stopFileName exists, and stop if it does.
resultsFile When done, write the results of the trials to resultsFile. Before then, this file name with .0 or .1 appended holds results so far.
nTrials After every nTrials perform checkpointing.
nTime After nTime seconds have passed, perform checkpointing.
... Optional arguments passed to generator.

Details

Checkpointing writes the results so far to a file (named from resultsFile with, alternately, .0 and .1 appended) and prints a progress message on the controlling terminal. Every time either nTrials trials or nTime seconds have passed since the last checkpoint a new one will occur; whichever is hit first will control checkpointing.

Value

A list of trial's, in the order they were performed. trial holds the arguments for the trial (produced by generator) and the results (produced by executor).

Note

This function was designed to check the robustness of estimates. The generator can produce varying initial values, algorithmic choices, and tuning parameters, for example. The executor then takes those and calls to a function of interest.

The generator can produce values randomly, using a deterministic scheme, or some mix of the two. The deterministic scheme may be infinite. If not, generator should return NULL when it is done, and the function will finish.

The executor is responsible for recording whatever information is of interest, apart from the arguments it receives. Relevant values could include convergence, parameter estimates, and run-time, among other things.

Results objects returned by estimators commonly include a lot of information, including the arguments of the function call. To save space (and thus time too) you may wish to record only the statistics you want, or blank out the data elements you don't need. The arguments returned by generator are likely to be more compact than those to the function being tested; for example, generator might return an index or name that executor translates into a large set of parameters.

The checkpoint files are deleted on exit from this function.

Author(s)

Ross Boylan ross@biostat.ucsf.edu

See Also

trial

Examples

## since the function runs forever, we don't want to really run it
## Not run: 

# create a set of parameters
testgen <- function(...) {
  p <- rnorm(3, ...)
  list(a=p[1], b=p[2], c=p[3])
}

# estimate a function using those parameters
testexec <- function(a, b, c) {
  Sys.sleep(10)  # some long computation
  a+b+c  # result, sensitive to the parameters
}

## Note that parameters, as used above, are not the same as the
## parameters of the inner function the executor runs.  The parameters
## here might be the initial value  for the parameters of the inner model.

r <- doTrials(testgen, testexec, nTrials=20, nTime=30)
## End(Not run)

[Package mspath version 0.9-9 Index]