Inter-processor communications stream. More...
Public Member Functions | |
ClassName ("Pstream") | |
Declare name of the class and its debug switch. More... | |
Pstream (const UPstream::commsTypes commsType, const label bufSize=0) | |
Construct for given communication type, with optional buffer size. More... | |
template<class Container , class Type > | |
Foam::Map< Container > | exchangeConsensus (const Map< Container > &sendBufs, const int tag, const label comm, const bool) |
template<class T > | |
Foam::List< T > | listGatherValues (const T &localValue, const label comm, const int tag) |
Public Member Functions inherited from UPstream | |
ClassName ("UPstream") | |
Declare name of the class and its debug switch. More... | |
UPstream (const commsTypes commsType) noexcept | |
Construct for given communication type. More... | |
commsTypes | commsType () const noexcept |
Get the communications type of the stream. More... | |
commsTypes | commsType (const commsTypes ct) noexcept |
Set the communications type of the stream. More... | |
template<class T > | |
Foam::List< T > | allGatherValues (const T &localValue, const label comm) |
template<class T > | |
Foam::List< T > | listGatherValues (const T &localValue, const label comm) |
Static Public Member Functions | |
template<class Type > | |
static void | broadcast (Type &value, const label comm=UPstream::worldComm) |
Broadcast content (contiguous or non-contiguous) to all communicator ranks. Does nothing in non-parallel. More... | |
template<class Type , class... Args> | |
static void | broadcasts (const label comm, Type &arg1, Args &&... args) |
Broadcast multiple items to all communicator ranks. Does nothing in non-parallel. More... | |
template<class ListType > | |
static void | broadcastList (ListType &list, const label comm=UPstream::worldComm) |
Broadcast list content (contiguous or non-contiguous) to all communicator ranks. Does nothing in non-parallel. More... | |
template<class T , class BinaryOp > | |
static void | gather (T &value, const BinaryOp &bop, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Gather (reduce) data, applying bop to combine value from different processors. The basis for Foam::reduce(). More... | |
template<class T > | |
static List< T > | listGatherValues (const T &localValue, const label comm=UPstream::worldComm, const int tag=UPstream::msgType()) |
Gather individual values into list locations. More... | |
template<class T > | |
static T | listScatterValues (const UList< T > &allValues, const label comm=UPstream::worldComm, const int tag=UPstream::msgType()) |
Scatter individual values from list locations. More... | |
template<class T , class CombineOp > | |
static void | combineGather (T &value, const CombineOp &cop, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Gather data, applying cop to inplace combine value from different processors. More... | |
template<class T , class CombineOp > | |
static void | combineReduce (T &value, const CombineOp &cop, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Reduce inplace (cf. MPI Allreduce) applying cop to inplace combine value from different processors. After completion all processors have the same data. More... | |
template<class T , class CombineOp > | |
static void | combineAllGather (T &value, const CombineOp &cop, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Same as Pstream::combineReduce. More... | |
template<class T , class CombineOp > | |
static void | listCombineGather (UList< T > &values, const CombineOp &cop, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Combines List elements. More... | |
template<class T , class CombineOp > | |
static void | listCombineReduce (List< T > &values, const CombineOp &cop, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Combines List elements. After completion all processors have the same data. More... | |
template<class T , class CombineOp > | |
static void | listCombineAllGather (List< T > &values, const CombineOp &cop, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Same as Pstream::listCombineReduce. More... | |
template<class Container , class CombineOp > | |
static void | mapCombineGather (Container &values, const CombineOp &cop, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Combine Map elements. More... | |
template<class Container , class CombineOp > | |
static void | mapCombineReduce (Container &values, const CombineOp &cop, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Reduce inplace (cf. MPI Allreduce) applying cop to inplace combine map values from different processors. After completion all processors have the same data. More... | |
template<class Container , class CombineOp > | |
static void | mapCombineAllGather (Container &values, const CombineOp &cop, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Same as Pstream::mapCombineReduce. More... | |
template<class T > | |
static void | gatherList (const UList< commsStruct > &comms, UList< T > &values, const int tag, const label comm) |
Gather data, but keep individual values separate. Uses the specified communication schedule. More... | |
template<class T > | |
static void | gatherList (UList< T > &values, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Gather data, but keep individual values separate. Uses linear/tree communication. More... | |
template<class T > | |
static void | allGatherList (UList< T > &values, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Gather data, but keep individual values separate. Uses MPI_Allgather or manual linear/tree communication. More... | |
template<class T > | |
static void | scatterList (const UList< commsStruct > &comms, UList< T > &values, const int tag, const label comm) |
Inverse of gatherList. Uses the specified communication schedule. More... | |
template<class T > | |
static void | scatterList (UList< T > &values, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Inverse of gatherList. Uses linear/tree communication. More... | |
template<class Container > | |
static void | exchangeSizes (const labelUList &sendProcs, const labelUList &recvProcs, const Container &sendBufs, labelList &sizes, const label tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Helper: exchange sizes of sendBufs for specified send/recv ranks. More... | |
template<class Container > | |
static void | exchangeSizes (const labelUList &neighProcs, const Container &sendBufs, labelList &sizes, const label tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Helper: exchange sizes of sendBufs for specified neighbour ranks. More... | |
template<class Container > | |
static void | exchangeSizes (const Container &sendBufs, labelList &recvSizes, const label comm=UPstream::worldComm) |
Helper: exchange sizes of sendBufs. The sendBufs is the data per processor (in the communicator). More... | |
template<class Container > | |
static void | exchangeSizes (const Map< Container > &sendBufs, Map< label > &recvSizes, const label tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
Exchange the non-zero sizes of sendBufs entries (sparse map) with other ranks in the communicator using non-blocking consensus exchange. More... | |
template<class Container , class Type > | |
static void | exchange (const UList< Container > &sendBufs, const labelUList &recvSizes, List< Container > &recvBufs, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm, const bool wait=true) |
Helper: exchange contiguous data. Sends sendBufs, receives into recvBufs using predetermined receive sizing. More... | |
template<class Container , class Type > | |
static void | exchange (const Map< Container > &sendBufs, const Map< label > &recvSizes, Map< Container > &recvBufs, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm, const bool wait=true) |
Exchange contiguous data. Sends sendBufs, receives into recvBufs. More... | |
template<class Container , class Type > | |
static void | exchange (const UList< Container > &sendBufs, List< Container > &recvBufs, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm, const bool wait=true) |
Exchange contiguous data. Sends sendBufs, receives into recvBufs. Determines sizes to receive. More... | |
template<class Container , class Type > | |
static void | exchange (const Map< Container > &sendBufs, Map< Container > &recvBufs, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm, const bool wait=true) |
Exchange contiguous data. Sends sendBufs, receives into recvBufs. Determines sizes to receive. More... | |
template<class Container , class Type > | |
static void | exchangeConsensus (const UList< Container > &sendBufs, List< Container > &recvBufs, const int tag, const label comm, const bool wait=true) |
Exchange contiguous data using non-blocking consensus (NBX) Sends sendData, receives into recvData. More... | |
template<class Container , class Type > | |
static void | exchangeConsensus (const Map< Container > &sendBufs, Map< Container > &recvBufs, const int tag, const label comm, const bool wait=true) |
Exchange contiguous data using non-blocking consensus (NBX) Sends sendData, receives into recvData. More... | |
template<class Container , class Type > | |
static Map< Container > | exchangeConsensus (const Map< Container > &sendBufs, const int tag, const label comm, const bool wait=true) |
Exchange contiguous data using non-blocking consensus (NBX) Sends sendData returns receive information. More... | |
template<class T > | |
static void | scatter (T &value, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
template<class T > | |
static void | combineScatter (T &value, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
template<class T > | |
static void | listCombineScatter (List< T > &value, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
template<class Container > | |
static void | mapCombineScatter (Container &values, const int tag=UPstream::msgType(), const label comm=UPstream::worldComm) |
static bool | broadcast (char *buf, const std::streamsize bufSize, const label communicator, const int rootProcNo=masterNo()) |
Broadcast buffer content to all processes in communicator. More... | |
Static Public Member Functions inherited from UPstream | |
static constexpr label | commGlobal () noexcept |
Communicator for all ranks, irrespective of any local worlds. More... | |
static constexpr label | commSelf () noexcept |
Communicator within the current rank only. More... | |
static label | commWorld () noexcept |
Communicator for all ranks (respecting any local worlds) More... | |
static label | commWorld (const label communicator) noexcept |
Set world communicator. Negative values are a no-op. More... | |
static label | commWarn (const label communicator) noexcept |
Alter communicator debugging setting. Warns for use of any communicator differing from specified. More... | |
static label | nComms () noexcept |
Number of currently defined communicators. More... | |
static void | printCommTree (const label communicator) |
Debugging: print the communication tree. More... | |
static label | commIntraHost () |
Demand-driven: Intra-host communicator (respects any local worlds) More... | |
static label | commInterHost () |
Demand-driven: Inter-host communicator (respects any local worlds) More... | |
static bool | hasHostComms () |
Test for presence of any intra or inter host communicators. More... | |
static void | clearHostComms () |
Remove any existing intra and inter host communicators. More... | |
static label | allocateCommunicator (const label parent, const labelRange &subRanks, const bool withComponents=true) |
Allocate new communicator with contiguous sub-ranks on the parent communicator. More... | |
static label | allocateCommunicator (const label parent, const labelUList &subRanks, const bool withComponents=true) |
Allocate new communicator with sub-ranks on the parent communicator. More... | |
static void | freeCommunicator (const label communicator, const bool withComponents=true) |
Free a previously allocated communicator. More... | |
static label | allocateInterHostCommunicator (const label parentCommunicator=worldComm) |
Allocate an inter-host communicator. More... | |
static label | allocateIntraHostCommunicator (const label parentCommunicator=worldComm) |
Allocate an intra-host communicator. More... | |
static int | baseProcNo (label comm, int procID) |
Return physical processor number (i.e. processor number in worldComm) given communicator and processor. More... | |
static label | procNo (const label comm, const int baseProcID) |
Return processor number in communicator (given physical processor number) (= reverse of baseProcNo) More... | |
static label | procNo (const label comm, const label currentComm, const int currentProcID) |
Return processor number in communicator (given processor number and communicator) More... | |
static void | addValidParOptions (HashTable< string > &validParOptions) |
Add the valid option this type of communications library adds/requires on the command line. More... | |
static bool | init (int &argc, char **&argv, const bool needsThread) |
Initialisation function called from main. More... | |
static bool | initNull () |
Special purpose initialisation function. More... | |
static void | barrier (const label communicator, UPstream::Request *req=nullptr) |
Impose a synchronisation barrier (optionally non-blocking) More... | |
static std::pair< int, int64_t > | probeMessage (const UPstream::commsTypes commsType, const int fromProcNo, const int tag=UPstream::msgType(), const label communicator=worldComm) |
Probe for an incoming message. More... | |
static label | nRequests () noexcept |
Number of outstanding requests (on the internal list of requests) More... | |
static void | resetRequests (const label n) |
Truncate outstanding requests to given length, which is expected to be in the range [0 to nRequests()]. More... | |
static void | addRequest (UPstream::Request &req) |
Transfer the (wrapped) MPI request to the internal global list. More... | |
static void | cancelRequest (const label i) |
Non-blocking comms: cancel and free outstanding request. Corresponds to MPI_Cancel() + MPI_Request_free() More... | |
static void | cancelRequest (UPstream::Request &req) |
Non-blocking comms: cancel and free outstanding request. Corresponds to MPI_Cancel() + MPI_Request_free() More... | |
static void | cancelRequests (UList< UPstream::Request > &requests) |
Non-blocking comms: cancel and free outstanding requests. Corresponds to MPI_Cancel() + MPI_Request_free() More... | |
static void | removeRequests (const label pos, label len=-1) |
Non-blocking comms: cancel/free outstanding requests (from position onwards) and remove from internal list of requests. Corresponds to MPI_Cancel() + MPI_Request_free() More... | |
static void | freeRequest (UPstream::Request &req) |
Non-blocking comms: free outstanding request. Corresponds to MPI_Request_free() More... | |
static void | freeRequests (UList< UPstream::Request > &requests) |
Non-blocking comms: free outstanding requests. Corresponds to MPI_Request_free() More... | |
static void | waitRequests (const label pos, label len=-1) |
Wait until all requests (from position onwards) have finished. Corresponds to MPI_Waitall() More... | |
static void | waitRequests (UList< UPstream::Request > &requests) |
Wait until all requests have finished. Corresponds to MPI_Waitall() More... | |
static bool | waitAnyRequest (const label pos, label len=-1) |
Wait until any request (from position onwards) has finished. Corresponds to MPI_Waitany() More... | |
static bool | waitSomeRequests (const label pos, label len=-1, DynamicList< int > *indices=nullptr) |
Wait until some requests (from position onwards) have finished. Corresponds to MPI_Waitsome() More... | |
static bool | waitSomeRequests (UList< UPstream::Request > &requests, DynamicList< int > *indices=nullptr) |
Wait until some requests have finished. Corresponds to MPI_Waitsome() More... | |
static label | waitAnyRequest (UList< UPstream::Request > &requests) |
Wait until any request has finished and return its index. Corresponds to MPI_Waitany() More... | |
static void | waitRequest (const label i) |
Wait until request i has finished. Corresponds to MPI_Wait() More... | |
static void | waitRequest (UPstream::Request &req) |
Wait until specified request has finished. Corresponds to MPI_Wait() More... | |
static bool | finishedRequest (const label i) |
Non-blocking comms: has request i finished? Corresponds to MPI_Test() More... | |
static bool | finishedRequest (UPstream::Request &req) |
Non-blocking comms: has request finished? Corresponds to MPI_Test() More... | |
static bool | finishedRequests (const label pos, label len=-1) |
Non-blocking comms: have all requests (from position onwards) finished? Corresponds to MPI_Testall() More... | |
static bool | finishedRequests (UList< UPstream::Request > &requests) |
Non-blocking comms: have all requests finished? Corresponds to MPI_Testall() More... | |
static bool | finishedRequestPair (label &req0, label &req1) |
Non-blocking comms: have both requests finished? Corresponds to pair of MPI_Test() More... | |
static void | waitRequestPair (label &req0, label &req1) |
Non-blocking comms: wait for both requests to finish. Corresponds to pair of MPI_Wait() More... | |
static bool | parRun (const bool on) noexcept |
Set as parallel run on/off. More... | |
static bool & | parRun () noexcept |
Test if this a parallel run. More... | |
static bool | haveThreads () noexcept |
Have support for threads. More... | |
static constexpr int | masterNo () noexcept |
Relative rank for the master process - is always 0. More... | |
static label | nProcs (const label communicator=worldComm) |
Number of ranks in parallel run (for given communicator). It is 1 for serial run. More... | |
static int | myProcNo (const label communicator=worldComm) |
Rank of this process in the communicator (starting from masterNo()). Can be negative if the process is not a rank in the communicator. More... | |
static bool | master (const label communicator=worldComm) |
True if process corresponds to the master rank in the communicator. More... | |
static bool | is_rank (const label communicator=worldComm) |
True if process corresponds to any rank (master or sub-rank) in the given communicator. More... | |
static bool | is_subrank (const label communicator=worldComm) |
True if process corresponds to a sub-rank in the given communicator. More... | |
static bool | is_parallel (const label communicator=worldComm) |
True if parallel algorithm or exchange is required. More... | |
static label | parent (const label communicator) |
The parent communicator. More... | |
static List< int > & | procID (const label communicator) |
The list of ranks within a given communicator. More... | |
static const wordList & | allWorlds () noexcept |
All worlds. More... | |
static const labelList & | worldIDs () noexcept |
The indices into allWorlds for all processes. More... | |
static label | myWorldID () |
My worldID. More... | |
static const word & | myWorld () |
My world. More... | |
static rangeType | allProcs (const label communicator=worldComm) |
Range of process indices for all processes. More... | |
static rangeType | subProcs (const label communicator=worldComm) |
Range of process indices for sub-processes. More... | |
static const List< commsStruct > & | linearCommunication (const label communicator=worldComm) |
Communication schedule for linear all-to-master (proc 0) More... | |
static const List< commsStruct > & | treeCommunication (const label communicator=worldComm) |
Communication schedule for tree all-to-master (proc 0) More... | |
static const List< commsStruct > & | whichCommunication (const label communicator=worldComm) |
Communication schedule for all-to-master (proc 0) as linear/tree/none with switching based on UPstream::nProcsSimpleSum and the is_parallel() state. More... | |
static int & | msgType () noexcept |
Message tag of standard messages. More... | |
static int | msgType (int val) noexcept |
Set the message tag for standard messages. More... | |
static int | incrMsgType (int val=1) noexcept |
Increment the message tag for standard messages. More... | |
static void | shutdown (int errNo=0) |
Shutdown (finalize) MPI as required. More... | |
static void | abort () |
Call MPI_Abort with no other checks or cleanup. More... | |
static void | exit (int errNo=1) |
Shutdown (finalize) MPI as required and exit program with errNo. More... | |
static void | allToAll (const UList< int32_t > &sendData, UList< int32_t > &recvData, const label communicator=worldComm) |
Exchange int32_t data with all ranks in communicator. More... | |
static void | allToAllConsensus (const UList< int32_t > &sendData, UList< int32_t > &recvData, const int tag, const label communicator=worldComm) |
Exchange non-zero int32_t data between ranks [NBX]. More... | |
static void | allToAllConsensus (const Map< int32_t > &sendData, Map< int32_t > &recvData, const int tag, const label communicator=worldComm) |
Exchange int32_t data between ranks [NBX]. More... | |
static Map< int32_t > | allToAllConsensus (const Map< int32_t > &sendData, const int tag, const label communicator=worldComm) |
Exchange int32_t data between ranks [NBX]. More... | |
static void | allToAll (const UList< int64_t > &sendData, UList< int64_t > &recvData, const label communicator=worldComm) |
Exchange int64_t data with all ranks in communicator. More... | |
static void | allToAllConsensus (const UList< int64_t > &sendData, UList< int64_t > &recvData, const int tag, const label communicator=worldComm) |
Exchange non-zero int64_t data between ranks [NBX]. More... | |
static void | allToAllConsensus (const Map< int64_t > &sendData, Map< int64_t > &recvData, const int tag, const label communicator=worldComm) |
Exchange int64_t data between ranks [NBX]. More... | |
static Map< int64_t > | allToAllConsensus (const Map< int64_t > &sendData, const int tag, const label communicator=worldComm) |
Exchange int64_t data between ranks [NBX]. More... | |
static void | mpiGather (const char *sendData, char *recvData, int count, const label communicator=worldComm) |
Receive identically-sized char data from all ranks. More... | |
static void | mpiScatter (const char *sendData, char *recvData, int count, const label communicator=worldComm) |
Send identically-sized char data to all ranks. More... | |
static void | mpiAllGather (char *allData, int count, const label communicator=worldComm) |
Gather/scatter identically-sized char data. More... | |
static void | gather (const char *sendData, int sendCount, char *recvData, const UList< int > &recvCounts, const UList< int > &recvOffsets, const label communicator=worldComm) |
Receive variable length char data from all ranks. More... | |
static void | scatter (const char *sendData, const UList< int > &sendCounts, const UList< int > &sendOffsets, char *recvData, int recvCount, const label communicator=worldComm) |
Send variable length char data to all ranks. More... | |
static void | mpiGather (const int32_t *sendData, int32_t *recvData, int count, const label communicator=worldComm) |
Receive identically-sized int32_t data from all ranks. More... | |
static void | mpiScatter (const int32_t *sendData, int32_t *recvData, int count, const label communicator=worldComm) |
Send identically-sized int32_t data to all ranks. More... | |
static void | mpiAllGather (int32_t *allData, int count, const label communicator=worldComm) |
Gather/scatter identically-sized int32_t data. More... | |
static void | gather (const int32_t *sendData, int sendCount, int32_t *recvData, const UList< int > &recvCounts, const UList< int > &recvOffsets, const label communicator=worldComm) |
Receive variable length int32_t data from all ranks. More... | |
static void | scatter (const int32_t *sendData, const UList< int > &sendCounts, const UList< int > &sendOffsets, int32_t *recvData, int recvCount, const label communicator=worldComm) |
Send variable length int32_t data to all ranks. More... | |
static void | mpiGather (const int64_t *sendData, int64_t *recvData, int count, const label communicator=worldComm) |
Receive identically-sized int64_t data from all ranks. More... | |
static void | mpiScatter (const int64_t *sendData, int64_t *recvData, int count, const label communicator=worldComm) |
Send identically-sized int64_t data to all ranks. More... | |
static void | mpiAllGather (int64_t *allData, int count, const label communicator=worldComm) |
Gather/scatter identically-sized int64_t data. More... | |
static void | gather (const int64_t *sendData, int sendCount, int64_t *recvData, const UList< int > &recvCounts, const UList< int > &recvOffsets, const label communicator=worldComm) |
Receive variable length int64_t data from all ranks. More... | |
static void | scatter (const int64_t *sendData, const UList< int > &sendCounts, const UList< int > &sendOffsets, int64_t *recvData, int recvCount, const label communicator=worldComm) |
Send variable length int64_t data to all ranks. More... | |
static void | mpiGather (const uint32_t *sendData, uint32_t *recvData, int count, const label communicator=worldComm) |
Receive identically-sized uint32_t data from all ranks. More... | |
static void | mpiScatter (const uint32_t *sendData, uint32_t *recvData, int count, const label communicator=worldComm) |
Send identically-sized uint32_t data to all ranks. More... | |
static void | mpiAllGather (uint32_t *allData, int count, const label communicator=worldComm) |
Gather/scatter identically-sized uint32_t data. More... | |
static void | gather (const uint32_t *sendData, int sendCount, uint32_t *recvData, const UList< int > &recvCounts, const UList< int > &recvOffsets, const label communicator=worldComm) |
Receive variable length uint32_t data from all ranks. More... | |
static void | scatter (const uint32_t *sendData, const UList< int > &sendCounts, const UList< int > &sendOffsets, uint32_t *recvData, int recvCount, const label communicator=worldComm) |
Send variable length uint32_t data to all ranks. More... | |
static void | mpiGather (const uint64_t *sendData, uint64_t *recvData, int count, const label communicator=worldComm) |
Receive identically-sized uint64_t data from all ranks. More... | |
static void | mpiScatter (const uint64_t *sendData, uint64_t *recvData, int count, const label communicator=worldComm) |
Send identically-sized uint64_t data to all ranks. More... | |
static void | mpiAllGather (uint64_t *allData, int count, const label communicator=worldComm) |
Gather/scatter identically-sized uint64_t data. More... | |
static void | gather (const uint64_t *sendData, int sendCount, uint64_t *recvData, const UList< int > &recvCounts, const UList< int > &recvOffsets, const label communicator=worldComm) |
Receive variable length uint64_t data from all ranks. More... | |
static void | scatter (const uint64_t *sendData, const UList< int > &sendCounts, const UList< int > &sendOffsets, uint64_t *recvData, int recvCount, const label communicator=worldComm) |
Send variable length uint64_t data to all ranks. More... | |
static void | mpiGather (const float *sendData, float *recvData, int count, const label communicator=worldComm) |
Receive identically-sized float data from all ranks. More... | |
static void | mpiScatter (const float *sendData, float *recvData, int count, const label communicator=worldComm) |
Send identically-sized float data to all ranks. More... | |
static void | mpiAllGather (float *allData, int count, const label communicator=worldComm) |
Gather/scatter identically-sized float data. More... | |
static void | gather (const float *sendData, int sendCount, float *recvData, const UList< int > &recvCounts, const UList< int > &recvOffsets, const label communicator=worldComm) |
Receive variable length float data from all ranks. More... | |
static void | scatter (const float *sendData, const UList< int > &sendCounts, const UList< int > &sendOffsets, float *recvData, int recvCount, const label communicator=worldComm) |
Send variable length float data to all ranks. More... | |
static void | mpiGather (const double *sendData, double *recvData, int count, const label communicator=worldComm) |
Receive identically-sized double data from all ranks. More... | |
static void | mpiScatter (const double *sendData, double *recvData, int count, const label communicator=worldComm) |
Send identically-sized double data to all ranks. More... | |
static void | mpiAllGather (double *allData, int count, const label communicator=worldComm) |
Gather/scatter identically-sized double data. More... | |
static void | gather (const double *sendData, int sendCount, double *recvData, const UList< int > &recvCounts, const UList< int > &recvOffsets, const label communicator=worldComm) |
Receive variable length double data from all ranks. More... | |
static void | scatter (const double *sendData, const UList< int > &sendCounts, const UList< int > &sendOffsets, double *recvData, int recvCount, const label communicator=worldComm) |
Send variable length double data to all ranks. More... | |
template<class T > | |
static List< T > | allGatherValues (const T &localValue, const label communicator=worldComm) |
Allgather individual values into list locations. More... | |
template<class T > | |
static List< T > | listGatherValues (const T &localValue, const label communicator=worldComm) |
Gather individual values into list locations. More... | |
template<class T > | |
static T | listScatterValues (const UList< T > &allValues, const label communicator=worldComm) |
Scatter individual values from list locations. More... | |
static bool | broadcast (char *buf, const std::streamsize bufSize, const label communicator, const int rootProcNo=masterNo()) |
Broadcast buffer contents to all processes in given communicator. The sizes must match on all processes. More... | |
static void | reduceAnd (bool &value, const label communicator=worldComm) |
Logical (and) reduction (MPI_AllReduce) More... | |
static void | reduceOr (bool &value, const label communicator=worldComm) |
Logical (or) reduction (MPI_AllReduce) More... | |
static void | waitRequests () |
Wait for all requests to finish. More... | |
static constexpr int | firstSlave () noexcept |
Process index of first sub-process. More... | |
Protected Attributes | |
DynamicList< char > | transferBuf_ |
Allocated transfer buffer (can be used for send or receive) More... | |
Additional Inherited Members | |
Public Types inherited from UPstream | |
enum | commsTypes : char { buffered, scheduled, nonBlocking, blocking = buffered } |
Communications types. More... | |
enum | sendModes : char { normal, sync } |
Different MPI-send modes (ignored for commsTypes::buffered) More... | |
typedef IntRange< int > | rangeType |
Int ranges are used for MPI ranks (processes) More... | |
Static Public Attributes inherited from UPstream | |
static const Enum< commsTypes > | commsTypeNames |
Enumerated names for the communication types. More... | |
static bool | floatTransfer |
Should compact transfer be used in which floats replace doubles reducing the bandwidth requirement at the expense of some loss in accuracy. More... | |
static int | nProcsSimpleSum |
Number of processors to change from linear to tree communication. More... | |
static int | nProcsNonblockingExchange |
Number of processors to change to nonBlocking consensual exchange (NBX). Ignored for zero or negative values. More... | |
static int | nPollProcInterfaces |
Number of polling cycles in processor updates. More... | |
static commsTypes | defaultCommsType |
Default commsType. More... | |
static int | maxCommsSize |
Optional maximum message size (bytes) More... | |
static int | tuning_NBX_ |
Tuning parameters for non-blocking exchange (NBX) More... | |
static const int | mpiBufferSize |
MPI buffer-size (bytes) More... | |
static label | worldComm |
Communicator for all ranks. May differ from commGlobal() if local worlds are in use. More... | |
static label | warnComm |
Debugging: warn for use of any communicator differing from warnComm. More... | |
Inter-processor communications stream.
|
inlineexplicit |
Construct for given communication type, with optional buffer size.
Definition at line 82 of file Pstream.H.
References DynamicList< T, SizeMin >::setCapacity(), and Pstream::transferBuf_.
ClassName | ( | "Pstream" | ) |
Declare name of the class and its debug switch.
|
static |
Broadcast content (contiguous or non-contiguous) to all communicator ranks. Does nothing in non-parallel.
Definition at line 28 of file PstreamBroadcast.C.
References UPstream::broadcast(), UPstream::is_parallel(), UPstream::master(), and os().
Referenced by abaqusMeshSet::abaqusMeshSet(), unwatchedIOdictionary::addWatch(), masterUncollatedFileOperation::addWatch(), masterUncollatedFileOperation::addWatches(), Foam::broadcastFile_recursive(), decomposedBlockData::calcNumProcs(), viewFactor::calculate(), Pstream::combineScatter(), masterUncollatedFileOperation::dirPath(), systemCall::dispatch(), abort::execute(), masterUncollatedFileOperation::filePath(), masterUncollatedFileOperation::findInstance(), masterUncollatedFileOperation::findTimes(), masterUncollatedFileOperation::findWatch(), ensightSurfaceReader::geometry(), masterUncollatedFileOperation::getFile(), fileOperation::getGlobalHostIORanks(), masterUncollatedFileOperation::getState(), Random::globalGaussNormal(), Random::globalPosition(), Random::globalRandomise01(), Random::globalSample01(), Pstream::listCombineScatter(), fileOperation::lookupAndCacheProcessorsPath(), Pstream::mapCombineScatter(), fileOperation::nProcs(), InflationInjection< CloudType >::parcelsToInject(), argList::parse(), triangulatedPatch::randomGlobalPoint(), decomposedBlockData::readBlocks(), masterUncollatedFileOperation::readHeader(), masterUncollatedFileOperation::readStream(), surfaceNoise::readSurfaceData(), Foam::reduce(), masterUncollatedFileOperation::removeWatch(), Foam::returnBroadcast(), Pstream::scatter(), mapDistributeBase::schedule(), Time::setMonitoring(), globalMeshData::sharedPoints(), rigidBodyMotion::solve(), ParSortableList< Type >::sort(), surfaceNoise::surfaceAverage(), hexRef8Data::sync(), syncTools::syncEdgeMap(), syncTools::syncPointMap(), triSurfaceMesh::triSurfaceMesh(), fileOperation::uniformFile(), sixDoFRigidBodyMotion::update(), fileMonitor::updateStates(), dynamicCode::waitForFile(), externalFileCoupler::waitForMaster(), externalFileCoupler::waitForSlave(), decomposedBlockData::writeBlocks(), surfaceNoise::writeSurfaceData(), and streamLineBase::writeToFile().
|
static |
Broadcast multiple items to all communicator ranks. Does nothing in non-parallel.
Definition at line 57 of file PstreamBroadcast.C.
References args, Foam::Detail::inputLoop(), UPstream::is_parallel(), UPstream::master(), os(), and Foam::Detail::outputLoop().
Referenced by regIOobject::addWatch(), surfaceNoise::calculate(), masterUncollatedFileOperation::dirPath(), ensightSurfaceReader::ensightSurfaceReader(), masterUncollatedFileOperation::filePath(), surfaceNoise::initialise(), faMeshTools::newMesh(), fvMeshTools::newMesh(), decomposedBlockData::readBlocks(), lumpedPointState::readData(), masterUncollatedFileOperation::readHeader(), masterUncollatedFileOperation::readObjects(), masterUncollatedFileOperation::sync(), fileOperation::sync(), OFstreamCollator::write(), and decomposedBlockData::writeData().
|
static |
Broadcast list content (contiguous or non-contiguous) to all communicator ranks. Does nothing in non-parallel.
For contiguous list data, this avoids serialization overhead, but at the expense of an additional broadcast call.
Definition at line 76 of file PstreamBroadcast.C.
References UPstream::broadcast(), UPstream::is_parallel(), UPstream::is_subrank(), UPstream::master(), and os().
|
static |
Gather (reduce) data, applying bop
to combine value
from different processors. The basis for Foam::reduce().
Uses linear/tree communication (with parallel guard).
Definition at line 37 of file PstreamGather.C.
References Foam::blockMeshTools::read(), T, and Foam::vtk::write().
Referenced by Foam::reduce().
|
static |
Gather individual values into list locations.
On master list length == nProcs, otherwise zero length.
For non-parallel : the returned list length is 1 with localValue.
tag | Only used for non-contiguous types |
|
static |
Scatter individual values from list locations.
On master input list length == nProcs, ignored on other procs.
For non-parallel : returns the first list element (or default initialized).
tag | Only used for non-contiguous types |
Definition at line 170 of file PstreamGather.C.
References Foam::abort(), UList< T >::cdata_bytes(), UList< T >::empty(), Foam::endl(), Foam::FatalError, FatalErrorInFunction, UList< T >::size(), and T.
|
static |
Gather data, applying cop
to inplace combine value
from different processors.
Uses linear/tree communication (with parallel guard).
[in,out] | value |
Definition at line 42 of file PstreamCombineGather.C.
References Foam::ensightOutput::debug, Foam::endl(), Foam::Perr, Foam::blockMeshTools::read(), T, and Foam::vtk::write().
|
static |
Reduce inplace (cf. MPI Allreduce) applying cop
to inplace combine value
from different processors. After completion all processors have the same data.
Uses linear/tree communication. Wraps combineGather/broadcast (may change in the future).
[in,out] | value |
Definition at line 135 of file PstreamCombineGather.C.
Referenced by Pstream::combineAllGather(), Foam::combineReduce(), multiWorldConnections::createComms(), globalMeshData::geometricSharedPoints(), and faGlobalMeshData::updateMesh().
|
inlinestatic |
Same as Pstream::combineReduce.
Definition at line 234 of file Pstream.H.
References Pstream::combineReduce().
|
static |
Combines List elements.
Uses linear/tree communication (with parallel guard).
[in,out] | values |
Definition at line 154 of file PstreamCombineGather.C.
References Foam::ensightOutput::debug, Foam::endl(), forAll, Foam::Perr, Foam::blockMeshTools::read(), Foam::HashTableOps::values(), and Foam::vtk::write().
Referenced by patchInjection::correct(), cellCellStencil::count(), injectionModelList::info(), transferModelList::info(), patchInjection::patchInjectedMassTotals(), meshRefinement::printMeshInfo(), fileMonitor::updateStates(), unequalBinWidth::write(), and equalBinWidth::write().
|
static |
Combines List elements. After completion all processors have the same data.
Uses linear/tree communication (with parallel guard).
[in,out] | values | - List (not UList) due to broadcast() |
Definition at line 253 of file PstreamCombineGather.C.
References Foam::HashTableOps::values().
Referenced by lumpedPointMovement::areas(), meshRefinement::balance(), viewFactor::calculate(), extractEulerianParticles::calculateAddressing(), NURBS3DVolume::computeControlPointSensitivities(), meshRefinement::createZoneBaffles(), Curle::execute(), patchProbes::findElements(), mappedPatchBase::findSamples(), lumpedPointMovement::forcesAndMoments(), propellerInfo::interpolate(), Pstream::listCombineAllGather(), fvMeshSubset::reset(), patchProbes::sample(), probes::sample(), cellVolumeWeight::update(), inverseDistance::update(), and propellerInfo::updateSampleDiskCells().
|
inlinestatic |
Same as Pstream::listCombineReduce.
[in,out] | values | - List (not UList) due to broadcast() |
Definition at line 283 of file Pstream.H.
References Pstream::listCombineReduce(), and Foam::HashTableOps::values().
Referenced by MMA::computeNewtonDirection().
|
static |
Combine Map elements.
Uses linear/tree communication (with parallel guard).
Definition at line 272 of file PstreamCombineGather.C.
References Foam::ensightOutput::debug, Foam::endl(), Foam::Perr, and Foam::HashTableOps::values().
|
static |
Reduce inplace (cf. MPI Allreduce) applying cop
to inplace combine map values
from different processors. After completion all processors have the same data.
Uses the specified communication schedule. Wraps mapCombineGather/broadcast (may change in the future). After completion all processors have the same data.
Definition at line 346 of file PstreamCombineGather.C.
References Foam::HashTableOps::values().
Referenced by Pstream::mapCombineAllGather(), Foam::regionSum(), areaWrite::write(), and meshRefinement::zonify().
|
inlinestatic |
Same as Pstream::mapCombineReduce.
Definition at line 336 of file Pstream.H.
References Pstream::mapCombineReduce(), and Foam::HashTableOps::values().
|
static |
Gather data, but keep individual values separate. Uses the specified communication schedule.
[in,out] | values |
Definition at line 41 of file PstreamGatherList.C.
References Foam::abort(), UList< T >::cdata_bytes(), Foam::ensightOutput::debug, UList< T >::empty(), Foam::endl(), Foam::FatalError, FatalErrorInFunction, forAll, Foam::nl, Foam::Perr, Foam::blockMeshTools::read(), UList< T >::size(), UList< T >::size_bytes(), Foam::HashTableOps::values(), and Foam::vtk::write().
Referenced by sizeDistribution::combineFields(), Foam::createReconstructMap(), viewFactor::initialise(), masterUncollatedFileOperation::NewIFstream(), InflationInjection< CloudType >::parcelsToInject(), argList::parse(), fileOperation::printRanks(), masterUncollatedFileOperation::readHeader(), masterUncollatedFileOperation::readStream(), ParSortableList< Type >::sort(), externalCoupled::writeGeometry(), and isoAdvection::writeIsoFaces().
|
static |
Gather data, but keep individual values separate. Uses linear/tree communication.
[in,out] | values |
Definition at line 320 of file PstreamGatherList.C.
References Foam::HashTableOps::values().
|
static |
Gather data, but keep individual values separate. Uses MPI_Allgather or manual linear/tree communication.
After completion all processors have the same data. Wraps gatherList/scatterList (may change in the future).
[in,out] | values |
Definition at line 357 of file PstreamGatherList.C.
References Foam::abort(), Foam::FatalError, FatalErrorInFunction, Foam::nl, T, and Foam::HashTableOps::values().
Referenced by surfaceZonesInfo::addCellZonesToMesh(), surfaceZonesInfo::addFaceZonesToMesh(), meshRefinement::checkCoupledFaceZones(), mappedPatchBase::collectSamples(), fieldValue::combineFields(), distributedTriSurfaceMesh::distribute(), mappedPatchBase::findSamples(), InjectedParticleInjection< CloudType >::initialise(), InjectedParticleDistributionInjection< CloudType >::initialise(), pointHistory::pointHistory(), processorTopology::procAdjacency(), fvMeshSubset::reset(), trackingInverseDistance::update(), inverseDistance::update(), and patchInjectionBase::updateMesh().
|
static |
Inverse of gatherList. Uses the specified communication schedule.
Definition at line 188 of file PstreamGatherList.C.
References Foam::abort(), Foam::ensightOutput::debug, UList< T >::empty(), Foam::endl(), Foam::FatalError, FatalErrorInFunction, forAll, forAllReverse, Foam::nl, Foam::Perr, Foam::blockMeshTools::read(), UList< T >::size(), Foam::HashTableOps::values(), and Foam::vtk::write().
|
static |
Inverse of gatherList. Uses linear/tree communication.
Definition at line 339 of file PstreamGatherList.C.
References Foam::HashTableOps::values().
|
static |
Helper: exchange sizes of sendBufs for specified send/recv ranks.
Definition at line 774 of file PstreamExchange.C.
References Foam::abort(), List< T >::clear(), Foam::FatalError, FatalErrorInFunction, Foam::nl, Foam::blockMeshTools::read(), List< T >::resize_nocopy(), UList< T >::size(), and Foam::vtk::write().
Referenced by mapDistributeBase::mapDistributeBase().
|
static |
Helper: exchange sizes of sendBufs for specified neighbour ranks.
Definition at line 852 of file PstreamExchange.C.
References List< T >::clear().
|
static |
Helper: exchange sizes of sendBufs. The sendBufs is the data per processor (in the communicator).
Returns sizes of sendBufs on the sending processor.
For non-parallel : copy sizes from sendBufs directly.
Definition at line 921 of file PstreamExchange.C.
References Foam::abort(), Foam::PstreamDetail::allToAll(), Foam::PstreamDetail::allToAllConsensus(), List< T >::clear(), Foam::FatalError, FatalErrorInFunction, forAll, Foam::nl, List< T >::resize_nocopy(), and UList< T >::size().
|
static |
Exchange the non-zero sizes of sendBufs entries (sparse map) with other ranks in the communicator using non-blocking consensus exchange.
Since the recvData map always cleared before receipt and sizes of zero are never transmitted, a simple check of its keys is sufficient to determine connectivity.
For non-parallel : copy size of rank (if it exists and non-empty) from sendBufs to recvSizes.
Definition at line 881 of file PstreamExchange.C.
References Foam::PstreamDetail::allToAllConsensus(), HashTable< T, Key, Hash >::clear(), Foam::BitOps::count(), HashTable< T, Key, Hash >::emplace(), forAllConstIters(), HashTable< T, Key, Hash >::reserve(), and HashTable< T, label, Hash< label > >::size().
|
static |
Helper: exchange contiguous data. Sends sendBufs, receives into recvBufs using predetermined receive sizing.
If wait=true will wait for all transfers to finish.
wait | Wait for requests to complete |
Definition at line 633 of file PstreamExchange.C.
References Foam::abort(), List< T >::clear(), Foam::BitOps::count(), Foam::FatalError, FatalErrorInFunction, forAll, Foam::nl, List< T >::resize_nocopy(), and UList< T >::size().
|
static |
Exchange contiguous data. Sends sendBufs, receives into recvBufs.
Data provided and received as container.
No internal guards or resizing.
recvSizes | Num of recv elements (not bytes) |
wait | Wait for requests to complete |
Definition at line 697 of file PstreamExchange.C.
References Foam::BitOps::count(), HashTable< T, label, Hash< label > >::erase(), HashTable< T, label, Hash< label > >::find(), and forAllIters.
|
static |
Exchange contiguous data. Sends sendBufs, receives into recvBufs. Determines sizes to receive.
If wait=true will wait for all transfers to finish.
wait | Wait for requests to complete |
Definition at line 974 of file PstreamExchange.C.
|
static |
Exchange contiguous data. Sends sendBufs, receives into recvBufs. Determines sizes to receive.
If wait=true will wait for all transfers to finish.
wait | Wait for requests to complete |
Definition at line 1005 of file PstreamExchange.C.
|
static |
Exchange contiguous data using non-blocking consensus (NBX) Sends sendData, receives into recvData.
Each entry of the recvBufs list is cleared before receipt. For non-parallel : copy own rank from sendBufs to recvBufs.
wait | (ignored) |
Definition at line 413 of file PstreamExchangeConsensus.C.
References Foam::abort(), Foam::FatalError, FatalErrorInFunction, List< T >::resize_nocopy(), and UList< T >::size().
|
static |
Exchange contiguous data using non-blocking consensus (NBX) Sends sendData, receives into recvData.
Each entry of the recvBufs map is cleared before receipt, but the map itself if not cleared. This allows the map to preserve allocated space (eg DynamicList entries) between calls.
For non-parallel : copy own rank (if it exists and non-empty) from sendBufs to recvBufs.
wait | (ignored) |
Definition at line 448 of file PstreamExchangeConsensus.C.
|
static |
Exchange contiguous data using non-blocking consensus (NBX) Sends sendData returns receive information.
For non-parallel : copy own rank (if it exists and non-empty)
wait | (ignored) |
|
inlinestatic |
tag | ignored |
Definition at line 640 of file Pstream.H.
References Pstream::broadcast().
|
inlinestatic |
tag | ignored |
Definition at line 655 of file Pstream.H.
References Pstream::broadcast().
|
inlinestatic |
tag | ignored |
Definition at line 670 of file Pstream.H.
References Pstream::broadcast().
|
inlinestatic |
tag | ignored |
Definition at line 685 of file Pstream.H.
References Pstream::broadcast(), and Foam::HashTableOps::values().
Foam::Map<Container> exchangeConsensus | ( | const Map< Container > & | sendBufs, |
const int | tag, | ||
const label | comm, | ||
const bool | |||
) |
Definition at line 471 of file PstreamExchangeConsensus.C.
Foam::List<T> listGatherValues | ( | const T & | localValue, |
const label | comm, | ||
const int | tag | ||
) |
Definition at line 102 of file PstreamGather.C.
References UList< T >::data_bytes(), List< T >::resize(), and T.
|
static |
Broadcast buffer content to all processes in communicator.
Definition at line 26 of file UPstreamBroadcast.C.
|
protected |
Allocated transfer buffer (can be used for send or receive)
Definition at line 65 of file Pstream.H.
Referenced by Pstream::Pstream().