CollectorClass.h File Reference

UDP readout collector class. More...

#include <filesystem>
#include <cstdlib>
#include <sstream>
#include <string>
#include <iomanip>
#include <utility>
#include <optional>
#include <random>
#include <map>
#include <set>
#include "Structs.h"
#include "Readout.h"
#include "enums.h"
#include "hdf_interface.h"
#include "version.hpp"
#include "efu_time.h"
#include "TypeDescriptionParser.h"

Include dependency graph for CollectorClass.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
class	CollectorSink
class	Collector

Functions
RL_API int	validate_collector_file (const std::string &filename)
	Validate that a given file is a valid collector file, and return the number of scan points it contains (0 if it's not a point-based collector)
RL_API int	validate_collector_file_impl (const HighFive::File &file, const std::string &filename)
RL_API bool	warn_about_another_build (const HighFive::File &file, const std::string &filename)
	Warn on stderr if a collector file was written by another libreadout build.
RL_API void	ensure_file_attributes (HighFive::File &file)
RL_API void	ensure_collector_group_attributes (HighFive::Group &group)
RL_API void	ensure_parameter_group_attributes (HighFive::Group &group)
RL_API std::string	validate_file_attributes (const HighFive::File &file)
RL_API std::string	validate_collector_group (const HighFive::Group &group)
RL_API std::string	validate_collector_root (const HighFive::Group &group)
RL_API bool	validate_collector_files (const std::vector< std::string > &in_filenames, std::set< std::string > &datasets, std::map< std::string, size_t > &sizes, std::set< std::string > &valid_files, std::string &collector_name, std::set< std::string > &parameters)
RL_API bool	append_collector_files (const std::string &out_filename, const std::vector< std::string > &in_filenames, bool reset_datasets)
	Append readouts from multiple same-point collector files into a single output file.
RL_API bool	concatenate_collector_files (const std::string &out_filename, const std::vector< std::string > &in_filenames)
	Concatenate readouts from multiple different-point collector files into a single multi-point output file.
RL_API bool	merge_collector_files (const std::string &out_filename, const std::vector< std::string > &in_filenames, bool remove_after_merge)
	Merge (append) readouts from multiple same-point collector files, optionally removing the inputs afterwards.
RL_API bool	combine_collector_files (const std::string &out_filename, const std::vector< std::string > &in_filenames)
	Combine multiple collector files automatically choosing append vs concatenate based on parameter identity.
RL_API void	merge_collector_datasets (const std::string &out_filename, const std::vector< std::string > &in_filenames, const std::string &which_dataset, bool remove_after_merge)
	Merge dataset(s) from multiple collector files into a single output file, for a specified scan point if applicable.
RL_API void	copy_collector_parameters (const std::string &out_filename, const std::vector< std::string > &in_filenames)
	Copy parameters from multiple collector files into a single output file, for a specified scan point if applicable.
std::string	filename_for_collector (const std::string &basepath, const std::string &basename, const std::string &extension="h5")
std::string	filename_for_collector_node (const std::string &basepath, const std::string &basename, int node, int nodes)

Detailed Description

UDP readout collector class.

Definition in file CollectorClass.h.

Macro Definition Documentation

RL_API

#define RL_API

Definition at line 44 of file CollectorClass.h.

Function Documentation

append_collector_files()

RL_API bool append_collector_files	(	const std::string &	out_filename,
		const std::vector< std::string > &	in_filenames,
		bool	reset_datasets
	)

Append readouts from multiple same-point collector files into a single output file.

Parameters

out_filename	the output file to create (must not already exist — uses HighFive::File::Create)
in_filenames	the input files to combine; all must have identical parameters
reset_datasets	if true, zero out the readout/cue/weight/normalisation datasets in the input files after combining

Returns

true on success, false on any validation or write error (details printed to std::cerr)

combine_collector_files()

RL_API bool combine_collector_files	(	const std::string &	out_filename,
		const std::vector< std::string > &	in_filenames
	)

Combine multiple collector files automatically choosing append vs concatenate based on parameter identity.

Parameters

out_filename	the output file to create (must not already exist — uses HighFive::File::Create)
in_filenames	the input files to combine

Returns

true on success, false on any validation or write error (details printed to std::cerr)

concatenate_collector_files()

RL_API bool concatenate_collector_files	(	const std::string &	out_filename,
		const std::vector< std::string > &	in_filenames
	)

Concatenate readouts from multiple different-point collector files into a single multi-point output file.

Parameters

out_filename	the output file to create (must not already exist — uses HighFive::File::Create)
in_filenames	the input files to concatenate; parameters must be consistent (same names/types) but not identical (different values)

Returns

true on success, false on any validation or write error (details printed to std::cerr)

copy_collector_parameters()

RL_API void copy_collector_parameters	(	const std::string &	out_filename,
		const std::vector< std::string > &	in_filenames
	)

Copy parameters from multiple collector files into a single output file, for a specified scan point if applicable.

Parameters

out_filename	the output file to create or open for writing
in_filenames	the input files to merge

A parameter group, named "parameters", which is located in the file root group or a (point, points)-based collector group, will be copied from one of the input files to the output file.

merge_collector_datasets()

RL_API void merge_collector_datasets	(	const std::string &	out_filename,
		const std::vector< std::string > &	in_filenames,
		const std::string &	which_dataset,
		bool	remove_after_merge
	)

Merge dataset(s) from multiple collector files into a single output file, for a specified scan point if applicable.

Parameters

out_filename	the output file to create or open for writing
in_filenames	the input files to merge
which_dataset	if non-empty, only merge the dataset with this name (which must be present in all input files), otherwise merge all datasets which are present in all input files
remove_after_merge	if true, remove the input dataset(s) after merging

merge_collector_files()

RL_API bool merge_collector_files	(	const std::string &	out_filename,
		const std::vector< std::string > &	in_filenames,
		bool	remove_after_merge
	)

Merge (append) readouts from multiple same-point collector files, optionally removing the inputs afterwards.

Parameters

out_filename	the output file to create (must not already exist — uses HighFive::File::Create)
in_filenames	the input files to merge; all must have identical parameters
remove_after_merge	if true, remove each input file after a successful merge

Returns

true on success, false on any validation or write error (details printed to std::cerr)

validate_collector_file()

RL_API int validate_collector_file

(

const std::string &

filename

)

Validate that a given file is a valid collector file, and return the number of scan points it contains (0 if it's not a point-based collector)

Returns

the number of scan points in the file, or -1 if the file is not a valid collector file

warn_about_another_build()

RL_API bool warn_about_another_build	(	const HighFive::File &	file,
		const std::string &	filename
	)

Warn on stderr if a collector file was written by another libreadout build.

Compares the file's version and revision stamps with this build's. Another build's file is still a valid file, so this only warns: combining it with this build's files is refused, and replaying it relies on record layouts and weight conventions not having changed in between. The file must carry both stamps (validation checks that first).

Returns

true if it warned

Referenced by ReaderSource::ReaderSource().