The DataPool class gathers data files across directories that span a parameter space. More...

#include <DataPool.hpp>

Inheritance diagram for tinc::DataPool:

Public Member Functions
	DataPool (std::string id, ParameterSpace &ps, std::string sliceCacheDir=std::string())

	DataPool (ParameterSpace &ps, std::string sliceCacheDir=std::string())

void	registerDataFile (std::string filename, std::string dimensionInFile)
	registerDataFile More...

std::map< std::string, std::string >	getRegisteredDataFiles ()
	get registered data files More...

void	clearRegisteredFiles ()
	Clear list of registered data files. More...

ParameterSpace &	getParameterSpace ()
	Get parameter space that controls this data pool. More...

std::string	createDataSlice (std::string field, std::string sliceDimension)
	Extract a slice of data. More...

std::string	createDataSlice (std::string field, std::vector< std::string > sliceDimensions)
	Extract a slice of data. More...

std::vector< std::string >	getCurrentFiles ()
	get list of full path to current files in datapool More...

size_t	readDataSlice (std::string field, std::string sliceDimension, void *data, size_t maxLen)
	readDataSlice More...

std::string	getCacheDirectory ()
	Get directory for data slices. More...

void	setCacheDirectory (std::string cacheDirectory, al::Socket *src=nullptr)
	set directory to be used to store slices More...

std::vector< std::string >	listFields (bool verifyConsistency=false)
	list fields in datapool More...

const DataPoolType	getType ()
	type of the data pool. Set on constructor, can't change. More...

std::string	getId ()
	get this object's id More...

void	setId (std::string id)
	set this object's id More...

std::string	getDocumentation () const

void	setDocumentation (const std::string &documentation)

Public Attributes
std::function< std::vector< std::string >std::vector< std::string >)>	getAllPaths

std::function< void(al::Socket *src)>	modified = [](al::Socket * ) {}

Protected Member Functions
virtual std::vector< std::string >	listFieldInFile (std::string file)=0

virtual bool	getFieldFromFile (std::string field, std::string file, size_t dimensionInFileIndex, void *data)=0

virtual bool	getFieldFromFile (std::string field, std::string file, void *data, size_t length)=0

std::string	getFileType (std::string file)

Protected Attributes
DataPoolType	mType {DataPoolType::DATAPOOL_USER}

std::string	mId

std::string	mDocumentation

Detailed Description

The DataPool class gathers data files across directories that span a parameter space.

The parameter space and its current values determine where the files in the data pool are found. This class is useful to manage data files that are the result of parameter sweeps, generating the same type of file in different directories, where each directory represents a sample of the parameter space.

Definition at line 55 of file DataPool.hpp.

Constructor & Destructor Documentation

◆ DataPool() [1/2]

tinc::DataPool::DataPool	(	std::string	id,
		ParameterSpace &	ps,
		std::string	sliceCacheDir = `std::string()`
	)

◆ DataPool() [2/2]

tinc::DataPool::DataPool	(	ParameterSpace &	ps,
		std::string	sliceCacheDir = `std::string()`
	)

Member Function Documentation

◆ clearRegisteredFiles()

void tinc::DataPool::clearRegisteredFiles ( )

Clear list of registered data files.

◆ createDataSlice() [1/2]

std::string tinc::DataPool::createDataSlice	(	std::string	field,
		std::string	sliceDimension
	)

Extract a slice of data.

Parameters

field	name of the field to extract
sliceDimension	dimension that can change across the slice

Returns: filename of the extracted slice

The slice will be created as a NetCDF4 file with a single variable called "data" that spans a dimension "values". The result will be a one dimensional slice containing the values of the "field" across all values for "sliceDimension" that must be registered in the parameter space.

◆ createDataSlice() [2/2]

std::string tinc::DataPool::createDataSlice	(	std::string	field,
		std::vector< std::string >	sliceDimensions
	)

Extract a slice of data.

Parameters

field	name of the field to extract
sliceDimensions	dimensions that can change across the slice

Returns: filename of the extracted slice

The output is a multidimensional slice of the data for the "field" values. The number of dimensions of the result is the size of sliceDimensions

◆ getCacheDirectory()

std::string tinc::DataPool::getCacheDirectory ( )

inline

Get directory for data slices.

Returns: cache directory

Definition at line 142 of file DataPool.hpp.

◆ getCurrentFiles()

std::vector<std::string> tinc::DataPool::getCurrentFiles ( )

get list of full path to current files in datapool

Returns: list of files

Both root path and current run path are prepended to the file names.

◆ getDocumentation()

std::string tinc::IdObject::getDocumentation ( ) const

inherited

◆ getFieldFromFile() [1/2]

virtual bool tinc::DataPool::getFieldFromFile	(	std::string	field,
		std::string	file,
		size_t	dimensionInFileIndex,
		void *	data
	)

protectedpure virtual

Implemented in tinc::DataPoolNetCDF, and tinc::DataPoolJson.

◆ getFieldFromFile() [2/2]

virtual bool tinc::DataPool::getFieldFromFile	(	std::string	field,
		std::string	file,
		void *	data,
		size_t	length
	)

protectedpure virtual

Implemented in tinc::DataPoolNetCDF, and tinc::DataPoolJson.

◆ getFileType()

std::string tinc::DataPool::getFileType ( std::string file )

protected

◆ getId()

std::string tinc::IdObject::getId ( )

inherited

get this object's id

◆ getParameterSpace()

ParameterSpace& tinc::DataPool::getParameterSpace ( )

inline

Get parameter space that controls this data pool.

Returns: the parameter space

Definition at line 91 of file DataPool.hpp.

◆ getRegisteredDataFiles()

std::map<std::string, std::string> tinc::DataPool::getRegisteredDataFiles ( )

get registered data files

Returns: key is the filename and vlaue is the name of the dimension that file contains

The dimension name in the output names the dimension whose changes have produced the values in the fields in the file

◆ getType()

const DataPoolType tinc::DataPool::getType ( )

inline

type of the data pool. Set on constructor, can't change.

Definition at line 171 of file DataPool.hpp.

◆ listFieldInFile()

virtual std::vector<std::string> tinc::DataPool::listFieldInFile ( std::string file )

protectedpure virtual

Implemented in tinc::DataPoolNetCDF, and tinc::DataPoolJson.

◆ listFields()

std::vector<std::string> tinc::DataPool::listFields ( bool verifyConsistency = false )

list fields in datapool

Parameters

verifyConsistency verify that all files in datapool are consistent

Returns: lis of field names

◆ readDataSlice()

size_t tinc::DataPool::readDataSlice	(	std::string	field,
		std::string	sliceDimension,
		void *	data,
		size_t	maxLen
	)

readDataSlice

Parameters

field
sliceDimension	dimension that can vary in the slice
data
maxLen

Returns: number of elements written to data pointer

◆ registerDataFile()

void tinc::DataPool::registerDataFile	(	std::string	filename,
		std::string	dimensionInFile
	)

registerDataFile

Parameters

filename
dimensionInFile	This filename must be relative to mParameterSpace->generateRelativeRunPath. It should be present in all data paths for the parameter space.

◆ setCacheDirectory()

void tinc::DataPool::setCacheDirectory	(	std::string	cacheDirectory,
		al::Socket *	src = `nullptr`
	)

set directory to be used to store slices

Parameters

cacheDirectory
src	socket that originated this request (nullptr if local)

◆ setDocumentation()

void tinc::IdObject::setDocumentation ( const std::string & documentation )

inherited

◆ setId()

void tinc::IdObject::setId ( std::string id )

inherited

set this object's id

Member Data Documentation

◆ getAllPaths

std::function<std::vector<std::string>std::vector<std::string>)> tinc::DataPool::getAllPaths

Initial value:

= [&](std::vector<std::string> fixedDimensions =
                            std::vector<std::string>()) {
        return mParameterSpace->runningPaths(fixedDimensions);
      }

Replace this function when the parameter space runningPaths() function is not adequate.

Definition at line 163 of file DataPool.hpp.

◆ mDocumentation

std::string tinc::IdObject::mDocumentation

protectedinherited

Definition at line 69 of file IdObject.hpp.

◆ mId

std::string tinc::IdObject::mId

protectedinherited

Definition at line 68 of file IdObject.hpp.

◆ modified

std::function<void(al::Socket *src)> tinc::IdObject::modified = [](al::Socket * ) {}

inherited

Definition at line 65 of file IdObject.hpp.

◆ mType

DataPoolType tinc::DataPool::mType {DataPoolType::DATAPOOL_USER}

protected

Definition at line 181 of file DataPool.hpp.

The documentation for this class was generated from the following file:

C:/Users/Andres/source/repos/casm_viewer/external/tinc/include/tinc/DataPool.hpp

Public Member Functions

Public Attributes

Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ DataPool() [1/2]

◆ DataPool() [2/2]

Member Function Documentation

◆ clearRegisteredFiles()

◆ createDataSlice() [1/2]

◆ createDataSlice() [2/2]

◆ getCacheDirectory()

◆ getCurrentFiles()

◆ getDocumentation()

◆ getFieldFromFile() [1/2]

◆ getFieldFromFile() [2/2]

◆ getFileType()

◆ getId()

◆ getParameterSpace()

◆ getRegisteredDataFiles()

◆ getType()

◆ listFieldInFile()

◆ listFields()

◆ readDataSlice()

◆ registerDataFile()

◆ setCacheDirectory()

◆ setDocumentation()

◆ setId()

Member Data Documentation

◆ getAllPaths

◆ mDocumentation

◆ mId

◆ modified

◆ mType