This Data Management Plan describes the general policy for generation, description, access, redistribution, and retention of data created in the use of the NHERI Coastal, Wave/ Surge and Tsunami (NHERI-CWST) Experimental Facility (EF) for NSF funded projects.
This document describes how the data related to an experimental study is created, handled, stored and shared.
Every time a Project is executed at the EF, a project folder is created with a set of directories as follows: data, docs, images, sw, video, and wavemaker. Not all directories may be present; for example, if no video was collected as part of the project then the empty video directory will eventually be removed when the project ceases activity. Also, specific folders might be created for a given project. The project folder directories are described in the following sections.
1. End-to-end data management infrastructure and plan.
Observations are immediately recorded to disk on the DAQ and then pushed to an intermediate processing system, which records an additional copy to disk before sending it on to a private backed-up server at the EF, and also to the DesignSafe-CI Data Depot at the end of the day. Researchers can access the data via this interface and perform other tasks using the Research Workbench of DesignSafe, including publishing the data. Additionally, an entire projects worth of data is archived to disk and given to researchers prior to their departure from the facility.
2. Project Folder Directory: data
All observations from the project are stored in the data directory. Subdirectories such as raw, inter, and final are used to separate data processing stages. Each experiment for the project is a separate subdirectory with its own name, with trials contained in directories below those. Some experiment names such as Calibration, DAQTest, SelfCalibration, and Survey, will contain files or trials that are purely for testing or calibration or instrument location purposes. The raw subdirectory includes data as recorded by the DAQ without any modification, typically as raw voltage. To maintain data integrity, raw data cannot be deleted or modified. The inter subdirectory contains data that has been post-processed from its originally recorded state, typically in physical units. The originally recorded data is always retained in the raw subdirectory and never modified. The inter subdirectory will always contain a Calibration subdirectory that holds all the calibration summary sheets for the project, along with any applicable wave gauge calibration sheets.
3. Project Folder Directory: docs
All documentation and metadata from the project are stored in the docs directory. Various subdirectories may be used to separate different types of documentation, when appropriate. The setup subdirectory will always be present, with two subdirectories (instm_locations and wiring_details) always used to store spreadsheets that log instrument locations and the setup of instruments and their connectivity to the data acquisition system (DAQ).
4. Project Folder Directory: images
All images from the project are stored in the images directory. Various subdirectories may be used to separate different types of images, when appropriate.
5. Project Folder Directory: sw
All source code used during the project is stored in the sw directory. Various subdirectories may be used to separate different types of source code, when appropriate. The data_processing subdirectory holds project-specific MATLAB routines for post-processing of data. The hwrl_daq subdirectory holds the LabVIEW source code used to run the data acquisition system (DAQ). The wavemaker subdirectory holds project-specific MATLAB routines for generating waves; the output from these routines is stored in the wavemaker project folder directory. Finally, a toolbox subdirectory may be present which holds a reference copy of the HWRL MATLAB toolboxes used for the project.
6. Project Folder Directory: video
All videos from the project are stored in the video directory. Various subdirectories may be used to separate different types of videos, when appropriate.
7. Project Folder Directory: wavemaker
All wavemaker input files from the project are stored in the wavemaker directory. Various subdirectories may be used to separate different types of wavemaker inputs, when appropriate. The disp_bin subdirectory holds the converted binary inputs used to run the wavemaker in displacement control mode; these are the actual files loaded by the wavemaker control computer after conversion from ASCII text. The disp_txt subdirectory holds ASCII text files specifying the board displacement; these are the files taken to the wavemaker control computer and subsequently converted to binary files for running waves. Finally, the waves_txt subdirectory holds ASCII text files specifying the free surface height at the face of the wavemaker piston. In some cases, wavemaker input files will not be used, and instead built-in commands for the wavemaker control computer will be used. The distinction is typically noted in metadata headers of the data file. MATLAB routines for wave generation are used to create all ASCII text files, whether displacement or free surface.
More information about the Project Folder structure and the Data recorded can be found in the Guide to the Project Folder.