dataone R package overview

1. Series Identifiers

Each data, metadata, and resource map object in DataONE has a unique identifier, refered to in DataONE documentation as a Persistent Identifier (PID). A PID is associated with one object in DataONE and always refers to the same object, the same set of bytes that are stored on the DataONE network.

A data or metadata object can be updated on a DataONE Member Node by using the R method dataone::updateObject(), so that a new version of the object becomes the active version that is discoverable in searches of DataONE. The older version is still available if the PID is known, but this version will not show up in DataONE searches. In order to download the new version, the new PID must be discovered and specified when the object is downloaded.

With dataone Version 2.0, an additional, optional identifier can be associated with an object, the Series Identifier (SID). Using SIDs the most current version of an object can be obtained, without the need to determine the PID of the latest version. For example, if the SID is specified when the object is downloaded, the most recent version of that object will be downloaded.

2. New Authentication Mechanism

Uploading data to DataONE requires that a DataONE user identity be provided. In DataONE Version 1.x, the identity of a user was provided by an X.509 client certificate. DataONE Version 2.0 adds an additional method to provide identity information - an authentication token. Authentication tokens can be used with Member Nodes that have been upgraded to the DataONE Version 2.0 Member Node API.

The process of providing user identity information either via an X.509 certificate or via an authentication token is refered to as authentication.

Authentication tokens can be obtained from a user’s DataONE account settings web page. To create an authentication token:

• Navigate to https://search.dataone.org
• Click Sign in, or Sign up if necessary
• Once signed in, click on the user name and select ‘My profile’ in the drop down menu.
• Click on the “Settings” tab.
• Click the “Apps” menu item.
• Click on the “Token for DataONE R” tab.
• Click “Renew authentication token” if the token you have been using has expired.
• Click on the “Copy” button below the text window to copy the authentication string to the paste buffer.
• Note the identity string and expiration date of the token.

• In the R console, paste the string which is similiar to this example:

    options(dataone_token = "eyJhbGciOiJSUzI1NiJ9...")

  (For this example, the typically very long token value has been truncated for brevity, however the entire string must be used for the token to be valid.)

• Note that this command can be entered in a user’s ~/.Rprofile file so that this command is entered when R is started.

• Remember that the console command must be re-entered with a new token value when the token expires

If the authentication token is defined as shown above, it will automatically be used when using methods from the dataone R client.

The authentication token must be safegaurded like a password. Anyone with access to it can access content in DataONE as the user identity contained in the token. Care should be taken to not add this code to any published scripts or documents. This code will expire after a certain time period after which it will be necessary to obtain a new one.

Note that the token shown above is for use in the DataONE production environment. If it is necessary to use authentication in a DataONE test environment, then the steps shown above should be followed by navigating to the following web page to generate the token: https://search.test.dataone.org. If you generate a token from this web page, then the R option name used should be “dataone_test_token”, not “dataone_token”, for example:

options(dataone_test_token = "eyJhbGciOiJSUzI1NiJ9...")

(For this example, the typically very long token value has been truncated for brevity, however the entire string must be used for the token to be valid.)

For an explanation of DataONE environments, please see DataONE environments.

Detailed, technical information about user identities and authentication in DataONE can be viewed at

DataONE Authentication

You can check your token by entering the following R commands:

libary(dataone)
am <- AuthenticationManager()
getTokenInfo(am)

The most important column of the information returned by getTokenInfo() is the expired column. If a token is expired, then it will not provide authentication and must be re-created from the user’s profile page as shown above.

3. Ability To Update System Metadata

Metadata is maintained by DataONE for each object that has been uploaded to it. This SystemMetadata for an object contains information such as the access policy that determines the users that can read or update the data, the data’s format type, how many replicated copies of the data to create, etc.

Member Nodes that have been upgraded to DataONE Version 2.0 Member Node API now the have the ability to update the system metadata of a data object without having to update (replace) the data object itself. So for example, an object can be uploaded to DataONE without having ‘public’ read enabled (the data creator or rightsholder and possibly a specified list of users could have access however). At a later date, the system metadata. could be updated to allow public read.

See help("updateSystemMetadata") for more information.

Error Using X.509 Certificates

Using an X.509 certificate for DataONE authenticatin on certain versions of Mac OS X can cause the following error:

Error in curl::curl_fetch_memory(url, handle = handle) : 
Problem with the local SSL certificate 

Changes in the Mac OS X system libraries in OS X Mavericks have taken away support of these X.509 certificates. A workaround to make these certificates usable on Mac OS X with R is to install a version of the curl R package that supports these certificates.

On Mac OS X, the libcurl library can be installed with either Mac Ports package manager or the HomeBrew package manager. The HomeBrew package manager can be significantly faster to install but either one will work provided the directions shown below are followed.

You can check if you have MacPorts installed by entering the following command in a terminal window:

port version

sudo port install curl

Sys.setenv(LIB_DIR="/opt/local/lib")
Sys.setenv(INCLUDE_DIR="/opt/local/include")
install.packages("curl", type="source")
library(curl)
library(dataone)

# Remove the environment variables as they are no longer needed.
Sys.setenv(LIB_DIR="")
Sys.setenv(INCLUDE_DIR="")

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

brew install curl --with-openssl
brew link curl --force

Sys.setenv(LIB_DIR="/usr/local/opt/curl/lib")
Sys.setenv(INCLUDE_DIR="/usr/local/opt/curl/include")
install.packages("curl", type="source")
library(curl)
library(dataone)

# Remove the environment variables as they are no longer needed.
Sys.setenv(LIB_DIR="")
Sys.setenv(INCLUDE_DIR="")