ffm {ff}R Documentation

flat-file database designed for large, multi-dimensional data arrays

Description

The function ffm and its methods allow for handling data using a flat file with memory mapped pages. It is a constructor function for ffm objects, which are multi-dimensional numeric arrays stored in a flat file.

Usage

  x <- ffm(file, dim = 0, pagesize = getdefaultpagesize(), readonly = FALSE)
  ## S3 method for class 'ffm':
  x[...]
  ## S3 method for class 'ffm':
  x[...] <- value    
  ## S3 method for class 'ffm':
  dim(x)
  ## S3 method for class 'ffm':
  length(x)
  ## S3 method for class 'ffm':
  print(x, ...)  

Arguments

file character string giving the name of a file to load or create.
dim dimension of the array if object should be (re-)created.
pagesize page size or 0 for default system page size (OS-specific).
readonly logical, specifying whether access to the resource is read-only.
value suitable replacement value or vector of values.
x a ffm object.
... indices specifying elements to extract or replace (indexing); further arguments passed to or from other methods (print method).

Details

On 32-bit R platforms the indexing is limited to a maximum number of 2^31-1. By using a multi-dimensional array, the data vector can be greater to overcome this limitation. The limit for ffm objects is 64 bit, even on 32-bit platforms (if supported by the virtual file system). A ffm object is stored on two files, ‘foo.ffd’ for the actual binary data and ‘foo.ffh’ for a header containing meta-information.

As ffm objects are held by external pointers, they are copied as a reference. The connection life-time of the ffm object and its implementation part (written in C++) is under control of the garbage collector gc. To explicitly close an ffm object, one should call the garbage collector after deleting the reference(!). ff depends on the OS and file-system facilities. E.g. it is not possible to create files > 4GB on FAT32 systems. The following table gives an overview of file size limits for common file systems (see http://en.wikipedia.org/wiki/Comparison_of_file_systems for further details):

File System File size limit
FAT16 2GB
FAT32 4GB
NTFS 16GB
ext2/3/4 16GB to 2TB
ReiserFS 4GB (up to version 3.4) / 8TB (from version 3.5)
XFS 8EB
JFS 4PB
HFS 2GB
HFS Plus 16GB
USF1 4GB to 256TB
USF2 512GB to 32PB
UDF 16EB

Examples

  b <- ffm("foom",c(10,20,30))   # create a multi-dimensional vector
  b[1,2,3] <- 23                 # set data cell
  b[1,2,3]                       # set data cell
  b[1:5,,] <- rnorm(20)          # complex addressing
  b[1,,]                         # show first row
  

[Package ff version 1.0-1 Index]