Version of masterUncollatedFileOperation that collates regIOobjects into a container in the processors/ subdirectory. More...

Inheritance diagram for collatedFileOperation:

[legend]

Collaboration diagram for collatedFileOperation:

[legend]

Public Member Functions
	TypeName ("collated")
	Runtime type information. More...

	collatedFileOperation (bool verbose=false)
	Default construct. More...

	collatedFileOperation (const Tuple2< label, labelList > &commAndIORanks, const bool distributedRoots, bool verbose=false)
	Construct from communicator with specified io-ranks. More...

virtual	~collatedFileOperation ()
	Destructor. More...

virtual void	storeComm () const
	Transfer ownership of communicator to this fileOperation. Use with caution. More...

virtual fileName	objectPath (const IOobject &io, const word &typeName) const
	Generate disk file name for object. Opposite of filePath. More...

virtual bool	writeObject (const regIOobject &, IOstreamOption streamOpt=IOstreamOption(), const bool writeOnProc=true) const
	Writes a regIOobject (so header, contents and divider). More...

virtual void	flush () const
	Forcibly wait until all output done. Flush any cached data. More...

virtual word	processorsDir (const IOobject &) const
	Actual name of processors dir. More...

virtual word	processorsDir (const fileName &) const
	Actual name of processors dir. More...

Public Member Functions inherited from masterUncollatedFileOperation
	TypeName ("masterUncollated")
	Runtime type information. More...

	masterUncollatedFileOperation (bool verbose=false)
	Default construct. More...

	masterUncollatedFileOperation (const Tuple2< label, labelList > &commAndIORanks, const bool distributedRoots, bool verbose=false)
	Construct from communicator with specified io-ranks. More...

virtual	~masterUncollatedFileOperation ()
	Destructor. More...

virtual bool	mkDir (const fileName &, mode_t=0777) const
	Make directory. More...

virtual bool	chMod (const fileName &, const mode_t) const
	Set the file mode. More...

virtual mode_t	mode (const fileName &, const bool followLink=true) const
	Return the file mode. More...

virtual fileName::Type	type (const fileName &, const bool followLink=true) const
	Return the file type: DIRECTORY, FILE or SYMLINK. More...

virtual bool	exists (const fileName &, const bool checkGzip=true, const bool followLink=true) const
	Does the name exist (as DIRECTORY or FILE) in the file system? More...

virtual bool	isDir (const fileName &, const bool followLink=true) const
	Does the name exist as a DIRECTORY in the file system? More...

virtual bool	isFile (const fileName &, const bool checkGzip=true, const bool followLink=true) const
	Does the name exist as a FILE in the file system? More...

virtual off_t	fileSize (const fileName &, const bool followLink=true) const
	Return size of file. More...

virtual time_t	lastModified (const fileName &, const bool followLink=true) const
	Return time of last file modification. More...

virtual double	highResLastModified (const fileName &, const bool followLink=true) const
	Return time of last file modification. More...

virtual fileNameList	readDir (const fileName &, const fileName::Type=fileName::FILE, const bool filtergz=true, const bool followLink=true) const
	Read a directory and return the entries as a string list. More...

virtual bool	cp (const fileName &src, const fileName &dst, const bool followLink=true) const
	Copy, recursively if necessary, the source to the destination. More...

virtual bool	ln (const fileName &src, const fileName &dst) const
	Create a softlink. dst should not exist. Returns true if. More...

virtual bool	mv (const fileName &src, const fileName &dst, const bool followLink=false) const
	Rename src to dst. More...

virtual bool	mvBak (const fileName &, const std::string &ext="bak") const
	Rename to a corresponding backup file. More...

virtual bool	rm (const fileName &) const
	Remove a file, returning true if successful otherwise false. More...

virtual bool	rmDir (const fileName &dir, const bool silent=false, const bool emptyOnly=false) const
	Remove a directory and its contents. More...

virtual fileName	filePath (const bool checkGlobal, const IOobject &io, const word &typeName, const bool search) const
	Search for an object. checkGlobal : also check undecomposed case. More...

virtual fileName	dirPath (const bool checkGlobal, const IOobject &io, const bool search) const
	Search for a directory. checkGlobal : also check undecomposed. More...

virtual fileNameList	readObjects (const objectRegistry &db, const fileName &instance, const fileName &local, word &newInstance) const
	Search directory for objects. Used in IOobjectList. More...

virtual bool	readHeader (IOobject &, const fileName &, const word &typeName) const
	Read object header from supplied file. More...

virtual autoPtr< ISstream >	readStream (regIOobject &, const fileName &, const word &typeName, const bool readOnProc=true) const
	Reads header for regIOobject and returns an ISstream to read the contents. More...

virtual bool	read (regIOobject &, const bool masterOnly, const IOstreamOption::streamFormat format, const word &typeName) const
	Top-level read. More...

virtual autoPtr< ISstream >	NewIFstream (const fileName &) const
	Generate an ISstream that reads a file. More...

virtual autoPtr< OSstream >	NewOFstream (const fileName &pathname, IOstreamOption streamOpt=IOstreamOption(), const bool writeOnProc=true) const
	Generate an OSstream that writes a file. More...

virtual autoPtr< OSstream >	NewOFstream (IOstreamOption::atomicType atomic, const fileName &pathname, IOstreamOption streamOpt=IOstreamOption(), const bool writeOnProc=true) const
	Generate an OSstream that writes a file. More...

virtual label	addWatch (const fileName &) const
	Add watching of a file. Returns handle. More...

virtual bool	removeWatch (const label) const
	Remove watch on a file (using handle) More...

virtual label	findWatch (const labelList &watchIndices, const fileName &) const
	Find index (or -1) of file in list of handles. More...

virtual void	addWatches (regIOobject &, const fileNameList &) const
	Helper: add watches for list of regIOobjects. More...

virtual fileName	getFile (const label) const
	Get name of file being watched (using handle) More...

virtual void	updateStates (const bool masterOnly, const bool syncPar) const
	Update state of all files. More...

virtual fileMonitor::fileState	getState (const label) const
	Get current state of file (using handle) More...

virtual void	setUnmodified (const label) const
	Set current state of file (using handle) to unmodified. More...

virtual instantList	findTimes (const fileName &, const word &) const
	Get sorted list of times. More...

virtual IOobject	findInstance (const IOobject &io, const scalar startValue, const word &stopInstance, const bool constant_fallback=true) const
	Find time instance where IOobject is located. The name of the IOobject can be empty, in which case only the IOobject::local() is checked. Does not search beyond `stopInstance` (if set) or `constant`. More...

virtual void	setTime (const Time &) const
	Callback for time change. More...

virtual void	sync ()
	Forcibly parallel sync. More...

const HashPtrTable< DynamicList< instant > > &	times () const noexcept
	Return cached times. More...

Public Member Functions inherited from fileOperation
	TypeName ("fileOperation")
	Runtime type information. More...

	fileOperation (const label comm, const labelUList &ioRanks=labelUList::null(), const bool distributedRoots=false)
	Construct from communicator, optionally with specified io-ranks and/or distributed roots. More...

	fileOperation (const Tuple2< label, labelList > &commAndIORanks, const bool distributedRoots=false)
	Construct from communicator with specified io-ranks. More...

	declareRunTimeSelectionTable (autoPtr, fileOperation, word,(bool verbose),(verbose))

	declareRunTimeSelectionTable (autoPtr, fileOperation, comm,(const Tuple2< label, labelList > &commAndIORanks, const bool distributedRoots, bool verbose),(commAndIORanks, distributedRoots, verbose))

virtual	~fileOperation ()=default
	Destructor. More...

label	comm () const noexcept
	Communicator to use. More...

label	comm (label communicator) const noexcept
	Set communicator to use [mutable]. Negative values are a no-op. More...

bool	distributed () const noexcept
	Distributed roots (parallel run) More...

bool	distributed (bool on) const noexcept
	Set distributed roots on/off [mutable]. More...

const labelList &	ioRanks () const noexcept
	The list of IO ranks (global ranks) More...

InfoProxy< fileOperation >	info () const noexcept
	Return info proxy, used to print information to a stream. More...

virtual bool	good () const
	True if the fileOperation can be considered valid. At the moment, primarily used to detect the dummy fileOperation. More...

virtual bool	broadcastCopy (const label comm, const bool writeOnProc, const fileName &src, const fileName &dst) const
	Read dir/file (recursively if necessary) on master of the communicator, send and write contents to all 'writeOnProc' processors with local file name. More...

virtual fileName	filePath (const fileName &, const bool checkGzip=true, const bool followLink=true) const
	Search for a file or directory. Use IOobject version in preference. More...

label	nProcs () const noexcept
	Overall number of processors, from UPstream::nProcs() or detected from directories/results. More...

label	nProcs (const label numProcs) noexcept
	Set number of processor directories/results. More...

virtual label	nProcs (const fileName &dir, const fileName &local="", const label wantedNProcs=0) const
	Get number of processor directories/results. More...

fileName	processorsCasePath (const IOobject &io, const word &procDir) const
	Generate path (like io.path) from root+casename with any. More...

fileName	processorsPath (const IOobject &io, const word &instance, const word &procDir) const
	Generate path (like io.path) with provided instance and any. More...

fileName	processorsPath (const fileName &, const word &procDir) const
	Operating on fileName: replace processorXXX with procDir. More...

label	setNProcs (label numProcs)
	Same as nProcs. More...

Static Public Attributes
static float	maxThreadFileBufferSize
	Max size of thread buffer size. This is the overall size of. More...

Static Public Attributes inherited from masterUncollatedFileOperation
static float	maxMasterFileBufferSize
	Max size of parallel communications. Switches from non-blocking. More...

Static Public Attributes inherited from fileOperation
static const Enum< pathType >	pathTypeNames_

static word	processorsBaseDir = "processors"
	Return the processors directory name (usually "processors") More...

static word	defaultFileHandler
	Name of the default fileHandler. More...

static refPtr< fileOperation >	fileHandlerPtr_
	The currently active file handler. Avoid accessing directly. More...

Protected Member Functions
void	printBanner (const bool withRanks=false) const
	Print banner information, optionally with io ranks. More...

bool	appendObject (const regIOobject &io, const fileName &pathName, IOstreamOption streamOpt) const
	Append to processorsNN/ file. More...

Protected Member Functions inherited from masterUncollatedFileOperation
template<class Type , class FileOp >
Type	masterOp (const fileName &fName, const FileOp &fop, const int tag, const label comm) const

template<class Type , class FileOp >
Type	masterOp (const fileName &src, const fileName &dest, const FileOp &fop, const int tag, const label comm) const

virtual fileName	filePathInfo (const bool checkGlobal, const bool isFile, const IOobject &io, const dirIndexList &pDirs, const bool search, pathType &searchType, word &processorsDir, word &instance) const
	Search (locally!) for object; return info on how it was found. More...

fileName	localObjectPath (const IOobject &, const pathType &searchType, const word &processorsDir, const word &instancePath) const
	Construct filePath. More...

bool	exists (const dirIndexList &, IOobject &io) const
	Helper: check IO for local existence. Like filePathInfo but. More...

Protected Member Functions inherited from fileOperation
fileMonitor &	monitor () const
	Get or create fileMonitor singleton. More...

refPtr< dirIndexList >	lookupAndCacheProcessorsPath (const fileName &objectPath, const bool syncPar) const
	Lookup name of processorsDDD using cache. More...

virtual refPtr< dirIndexList >	lookupProcessorsPath (const fileName &objectPath) const
	Lookup name of processorsDDD using cache. More...

bool	exists (IOobject &io) const
	Does IOobject exist? Is either a directory (empty name()) or a file. More...

bool	isIOrank (const label proci) const
	Is proci a master rank in the communicator (in parallel) or a master rank in the IO ranks (non-parallel) More...

void	printRanks () const
	Helper: output which ranks are IO. More...

Protected Attributes
OFstreamCollator	writer_
	Threaded writer. More...

Protected Attributes inherited from masterUncollatedFileOperation
HashPtrTable< DynamicList< instant > >	times_
	Cached times for a given directory. More...

Protected Attributes inherited from fileOperation
label	comm_
	Communicator to use. More...

label	nProcs_
	Overall number of processors. More...

bool	distributed_
	Distributed roots (parallel run) More...

const labelList	ioRanks_
	The list of IO ranks (global ranks) More...

HashTable< dirIndexList >	procsDirs_
	Detected processors directories. More...

std::unique_ptr< fileMonitor >	monitorPtr_
	File-change monitor for all registered files. More...

Additional Inherited Members
Public Types inherited from fileOperation
enum	pathType : int { NOTFOUND = 0, ABSOLUTE, OBJECT, WRITEOBJECT, PROCUNCOLLATED, PROCBASEOBJECT = PROCUNCOLLATED + 1, PROCOBJECT = PROCBASEOBJECT + 1, PARENTOBJECT, FINDINSTANCE, PROCUNCOLLATEDINSTANCE, PROCBASEINSTANCE, PROCINSTANCE }
	Enumeration for the location of an IOobject. More...

typedef Tuple2< fileName, Tuple2< pathType, int > >	dirIndex
	Augment fileName with pathType and local offset. More...

typedef List< dirIndex >	dirIndexList

typedef IntRange< int >	procRangeType
	For addressing a range of processors, identical to UPstream::rangeType. More...

Static Public Member Functions inherited from fileOperation
static refPtr< fileOperation >	null ()
	Reference to a dummy file handler. More...

static autoPtr< fileOperation >	New (const word &handlerType, bool verbose=false)
	Select fileHandler-type. Uses defaultFileHandler if the handlerType is empty. More...

static autoPtr< fileOperation >	New (const word &handlerType, const Tuple2< label, labelList > &commAndIORanks, const bool distributedRoots, bool verbose=false)
	Select fileHandler-type. Uses defaultFileHandler if the handlerType is empty. More...

static autoPtr< fileOperation >	New (const fileOperation &origHandler, const boolUList &useProc, bool verbose=false)
	Replicate the given fileHandler properties with a subset of (global) ranks. More...

static autoPtr< fileOperation >	New (const fileOperation &origHandler, const bitSet &useProc, bool verbose=false)
	Replicate the given fileHandler properties with a subset of (global) ranks. More...

static autoPtr< fileOperation >	NewUncollated ()
	The commonly used uncollatedFileOperation. More...

static const fileOperation &	fileHandler ()
	Return the current file handler. Will create the default file handler if necessary. More...

static refPtr< fileOperation >	fileHandler (std::nullptr_t)
	Delete current file handler. More...

static refPtr< fileOperation >	fileHandler (refPtr< fileOperation > &newHandler)
	Replace the current file handler. More...

static refPtr< fileOperation >	fileHandler (refPtr< fileOperation > &&newHandler)
	Replace the current file handler. More...

static refPtr< fileOperation >	fileHandler (autoPtr< fileOperation > &&newHandler)
	Replace the current file handler. More...

static int	cacheLevel () noexcept
	Return cache level. More...

static int	cacheLevel (int level) noexcept
	Set cache level (0 = off). More...

static int	nProcsFilter () noexcept
	Return collated 'processorsDDD' filtering. More...

static int	nProcsFilter (int level) noexcept
	Set collated 'processorsDDD' filtering (0 = off). More...

static instantList	sortTimes (const fileNameList &dirEntries, const word &constantName="constant")
	Sort directory entries according to time value,. More...

static bool	uniformFile (const fileNameList &names)
	True if the file names are identical. False on an empty list. More...

static bool	uniformFile (const label comm, const fileName &name)
	True if the file name is identical on all ranks. More...

static label	splitProcessorPath (const fileName &objectPath, fileName &path, fileName &procDir, fileName &local, procRangeType &group, label &nProcs)
	Split objectPath into part before 'processor' and part after. More...

static label	detectProcessorPath (const fileName &objPath)
	Detect processor number from '/aa/bb/processorDDD/cc'. More...

static labelRange	subRanks (const labelUList &mainIOranks)
	Get (contiguous) range/bounds of ranks addressed within the given main io-ranks. More...

static labelList	getGlobalHostIORanks ()
	Get list of global IO master ranks based on the hostname. It is assumed that each host range is contiguous. More...

static labelList	getGlobalIORanks ()
	Get list of global IO ranks from FOAM_IORANKS env variable. If set, these correspond to the IO master ranks. More...

Static Protected Member Functions inherited from masterUncollatedFileOperation
static void	readAndSend (const fileName &filePath, const labelUList &recvProcs, PstreamBuffers &pBufs)
	Read file contents and send to processors. More...

static autoPtr< ISstream >	read (IOobject &io, const label comm, const bool uniform, const fileNameList &filePaths, const boolUList &readOnProcs)
	Read files on comms master. More...

Static Protected Member Functions inherited from fileOperation
static void	mergeTimes (const instantList &extraTimes, const word &constantName, instantList &times)
	Merge two times. More...

static bool	isFileOrDir (const bool isFile, const fileName &)
	Helper: check for file (isFile) or directory (!isFile) More...

static label	getManagedComm (const label communicator)
	Construction helper: check for locally allocated communicator. More...

Static Protected Attributes inherited from fileOperation
static int	cacheLevel_
	Cache level (eg, for caching time directories). Default: 1. More...

Detailed Description

Version of masterUncollatedFileOperation that collates regIOobjects into a container in the processors/ subdirectory.

Uses threading if maxThreadFileBufferSize != 0. > 0 : Can use mpi inside thread to collect data if buffer is not large enough. Does need full thread support inside MPI.

< 0 : special : -maxThreadFileBufferSize is guaranteed large enough for all writing. Initialises MPI without thread support.

See also: masterUncollatedFileOperation

Source files

Definition at line 62 of file collatedFileOperation.H.

Constructor & Destructor Documentation

◆ collatedFileOperation() [1/2]

collatedFileOperation ( bool verbose = false )

explicit

Default construct.

Definition at line 262 of file collatedFileOperation.C.

◆ collatedFileOperation() [2/2]

collatedFileOperation	(	const Tuple2< label, labelList > &	commAndIORanks,
		const bool	distributedRoots,
		bool	verbose = `false`
	)

explicit

Construct from communicator with specified io-ranks.

Definition at line 280 of file collatedFileOperation.C.

◆ ~collatedFileOperation()

~collatedFileOperation ( )

virtual

Destructor.

Definition at line 308 of file collatedFileOperation.C.

References Foam::flush(), and UPstream::freeCommunicator().

Here is the call graph for this function:

Member Function Documentation

◆ printBanner()

void printBanner ( const bool withRanks = false ) const

protected

Print banner information, optionally with io ranks.

fileModificationChecking already set by base class (masterUncollated)

if (IOobject::fileModificationChecking == IOobject::timeStampMaster) { WarningInFunction << "Resetting fileModificationChecking to timeStamp" << endl; } else if (IOobject::fileModificationChecking == IOobject::inotifyMaster) { WarningInFunction << "Resetting fileModificationChecking to inotify" << endl; }

Definition at line 79 of file collatedFileOperation.C.

References DetailInfo, Foam::endl(), Foam::mag(), and Foam::nl.

Here is the call graph for this function:

◆ appendObject()

bool appendObject	(	const regIOobject &	io,
		const fileName &	pathName,
		IOstreamOption	streamOpt
	)		const

protected

Append to processorsNN/ file.

Definition at line 145 of file collatedFileOperation.C.

References Foam::ensightOutput::debug, Foam::endl(), Foam::exit(), Foam::FatalError, FatalErrorInFunction, Foam::FatalIOError, FatalIOErrorInFunction, io(), os(), Foam::Pout, IOstreamOption::version(), and Foam::writeHeader().

Here is the call graph for this function:

◆ TypeName()

TypeName ( "collated" )

Runtime type information.

◆ storeComm()

void storeComm ( ) const

virtual

Transfer ownership of communicator to this fileOperation. Use with caution.

Reimplemented from masterUncollatedFileOperation.

Reimplemented in hostCollatedFileOperation.

Definition at line 299 of file collatedFileOperation.C.

◆ objectPath()

Foam::fileName objectPath	(	const IOobject &	io,
		const word &	typeName
	)		const

virtual

Generate disk file name for object. Opposite of filePath.

Reimplemented from fileOperation.

Definition at line 320 of file collatedFileOperation.C.

References io(), masterUncollatedFileOperation::localObjectPath(), word::null, fileOperation::OBJECT, and fileOperation::PROCOBJECT.

Here is the call graph for this function:

◆ writeObject()

bool writeObject	(	const regIOobject &	io,
		IOstreamOption	streamOpt = `IOstreamOption()`,
		const bool	writeOnProc = `true`
	)		const

virtual

Writes a regIOobject (so header, contents and divider).

Returns success state.

Reimplemented from masterUncollatedFileOperation.

Definition at line 350 of file collatedFileOperation.C.

References IOobject::bannerEnabled(), Foam::ensightOutput::debug, dict, Foam::endl(), io(), fileName::isAbsolute(), Foam::mag(), UPstream::master(), Foam::mkDir(), IOstreamOption::NO_APPEND, os(), UPstream::parRun(), path(), Foam::Pout, TimePaths::processorCase(), IOobject::writeEndDivider(), and decomposedBlockData::writeExtraHeaderContent().

Here is the call graph for this function:

◆ flush()

void flush ( ) const

virtual

Forcibly wait until all output done. Flush any cached data.

Reimplemented from masterUncollatedFileOperation.

Definition at line 523 of file collatedFileOperation.C.

References Foam::ensightOutput::debug, Foam::endl(), masterUncollatedFileOperation::flush(), and Foam::Pout.

Here is the call graph for this function:

◆ processorsDir() [1/2]

Foam::word processorsDir ( const IOobject & io ) const

virtual

Actual name of processors dir.

Reimplemented from fileOperation.

Definition at line 606 of file collatedFileOperation.C.

References io().

Here is the call graph for this function:

◆ processorsDir() [2/2]

Foam::word processorsDir ( const fileName & fName ) const

virtual

Actual name of processors dir.

Reimplemented from fileOperation.

Definition at line 537 of file collatedFileOperation.C.

References UList< T >::first(), UList< T >::last(), Foam::name(), UPstream::parRun(), UPstream::procID(), and UList< T >::size().

Here is the call graph for this function:

Member Data Documentation

◆ writer_

OFstreamCollator writer_

mutableprotected

Threaded writer.

Definition at line 89 of file collatedFileOperation.H.

◆ maxThreadFileBufferSize

float maxThreadFileBufferSize

static

Max size of thread buffer size. This is the overall size of.

all files to be written. Starts blocking if not enough size. Read as float to enable easy specification of large sizes.

Definition at line 126 of file collatedFileOperation.H.

Referenced by fileOperationInitialise_collated::needsThreading().

The documentation for this class was generated from the following files:

src/OpenFOAM/global/fileOperations/collatedFileOperation/collatedFileOperation.H
src/OpenFOAM/global/fileOperations/collatedFileOperation/collatedFileOperation.C

Public Member Functions

Static Public Attributes

Protected Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ collatedFileOperation() [1/2]

◆ collatedFileOperation() [2/2]

◆ ~collatedFileOperation()

Member Function Documentation

◆ printBanner()

◆ appendObject()

◆ TypeName()

◆ storeComm()

◆ objectPath()

◆ writeObject()

◆ flush()

◆ processorsDir() [1/2]

◆ processorsDir() [2/2]

Member Data Documentation

◆ writer_

◆ maxThreadFileBufferSize