Scotch domain decomposition. More...
Public Member Functions | |
scotchDecomp (const scotchDecomp &)=delete | |
No copy construct. More... | |
void | operator= (const scotchDecomp &)=delete |
No copy assignment. More... | |
TypeName ("scotch") | |
Runtime type information. More... | |
scotchDecomp (const label numDomains) | |
Construct with number of domains (no coefficients or constraints) More... | |
scotchDecomp (const dictionary &decompDict, const word ®ionName="") | |
Construct given decomposition dictionary and optional region name. More... | |
virtual | ~scotchDecomp ()=default |
Destructor. More... | |
virtual bool | parallelAware () const |
Knows about coupled boundaries. More... | |
virtual labelList | decompose (const polyMesh &mesh, const pointField &points=pointField::null(), const scalarField &pointWeights=scalarField::null()) const |
Return for every coordinate the wanted processor number. More... | |
virtual labelList | decompose (const polyMesh &mesh, const labelList &agglom, const pointField &agglomPoints, const scalarField &agglomWeights=scalarField::null()) const |
Return for every coordinate the wanted processor number. More... | |
virtual labelList | decompose (const CompactListList< label > &globalCellCells, const pointField &cc=pointField::null(), const scalarField &cWeights=scalarField::null()) const |
Return the wanted processor number for every cell. More... | |
virtual labelList | decompose (const labelListList &globalCellCells, const pointField &cc=pointField::null(), const scalarField &cWeights=scalarField::null()) const |
Return the wanted processor number for every cell. More... | |
virtual labelList | decompose (const pointField &points, const scalarField &pointWeights=scalarField::null()) const |
Inherit all decompose methods. More... | |
virtual labelList | decompose (const polyMesh &mesh, const pointField &points, const scalarField &pointWeights=scalarField::null()) const=0 |
Inherit all decompose methods. More... | |
virtual labelList | decompose (const polyMesh &mesh, const labelList &cellToRegion, const pointField ®ionPoints, const scalarField ®ionWeights=scalarField::null()) const |
Inherit all decompose methods. More... | |
virtual labelList | decompose (const CompactListList< label > &globalCellCells, const pointField &cc, const scalarField &cWeights=scalarField::null()) const=0 |
Inherit all decompose methods. More... | |
virtual labelList | decompose (const labelListList &globalCellCells, const pointField &cc, const scalarField &cWeights=scalarField::null()) const=0 |
Inherit all decompose methods. More... | |
virtual labelList | decompose (const polyMesh &mesh, const scalarField &cellWeights, const boolList &blockedFace, const PtrList< labelList > &specifiedProcessorFaces, const labelList &specifiedProcessor, const List< labelPair > &explicitConnections) const |
Inherit all decompose methods. More... | |
labelList | decompose (const polyMesh &mesh, const scalarField &cWeights) const |
Inherit all decompose methods. More... | |
Public Member Functions inherited from metisLikeDecomp | |
virtual | ~metisLikeDecomp ()=default |
Destructor. More... | |
virtual labelList | decompose (const pointField &points, const scalarField &pointWeights=scalarField::null()) const |
Inherit decompose from decompositionMethod. More... | |
virtual labelList | decompose (const polyMesh &mesh, const pointField &points, const scalarField &pointWeights=scalarField::null()) const=0 |
Inherit decompose from decompositionMethod. More... | |
virtual labelList | decompose (const polyMesh &mesh, const labelList &cellToRegion, const pointField ®ionPoints, const scalarField ®ionWeights=scalarField::null()) const |
Inherit decompose from decompositionMethod. More... | |
virtual labelList | decompose (const CompactListList< label > &globalCellCells, const pointField &cc, const scalarField &cWeights=scalarField::null()) const=0 |
Inherit decompose from decompositionMethod. More... | |
virtual labelList | decompose (const labelListList &globalCellCells, const pointField &cc, const scalarField &cWeights=scalarField::null()) const=0 |
Inherit decompose from decompositionMethod. More... | |
virtual labelList | decompose (const polyMesh &mesh, const scalarField &cellWeights, const boolList &blockedFace, const PtrList< labelList > &specifiedProcessorFaces, const labelList &specifiedProcessor, const List< labelPair > &explicitConnections) const |
Inherit decompose from decompositionMethod. More... | |
labelList | decompose (const polyMesh &mesh, const scalarField &cWeights) const |
Inherit decompose from decompositionMethod. More... | |
Public Member Functions inherited from decompositionMethod | |
decompositionMethod (const decompositionMethod &)=delete | |
No copy construct. More... | |
void | operator= (const decompositionMethod &)=delete |
No copy assignment. More... | |
TypeName ("decompositionMethod") | |
Runtime type information. More... | |
declareRunTimeSelectionTable (autoPtr, decompositionMethod, dictionary,(const dictionary &decompDict, const word ®ionName),(decompDict, regionName)) | |
decompositionMethod (const dictionary &decompDict, const word ®ionName="") | |
Construct given the decomposition dictionary, optionally region-specific. More... | |
virtual | ~decompositionMethod ()=default |
Destructor. More... | |
label | nDomains () const noexcept |
Number of domains. More... | |
virtual bool | geometric () const |
True if the method is purely geometric, often using cell centre points. More... | |
virtual labelList | decompose (const pointField &points, const scalarField &pointWeights=scalarField::null()) const |
Return the wanted processor number for every coordinate, using uniform or specified point weights. More... | |
void | setConstraints (const polyMesh &mesh, boolList &blockedFace, PtrList< labelList > &specifiedProcessorFaces, labelList &specifiedProcessor, List< labelPair > &explicitConnections) const |
Helper: extract constraints: More... | |
void | applyConstraints (const polyMesh &mesh, const boolList &blockedFace, const PtrList< labelList > &specifiedProcessorFaces, const labelList &specifiedProcessor, const List< labelPair > &explicitConnections, labelList &finalDecomp) const |
Helper: apply constraints to a decomposition. More... | |
virtual labelList | decompose (const polyMesh &mesh, const scalarField &cellWeights, const boolList &blockedFace, const PtrList< labelList > &specifiedProcessorFaces, const labelList &specifiedProcessor, const List< labelPair > &explicitConnections) const |
labelList | decompose (const polyMesh &mesh, const scalarField &cWeights) const |
Decompose a mesh. More... | |
Protected Member Functions | |
virtual label | decomposeSerial (const labelList &adjncy, const labelList &xadj, const List< scalar > &cWeights, labelList &decomp) const |
Decompose non-parallel. More... | |
Protected Member Functions inherited from metisLikeDecomp | |
virtual label | decomposeGeneral (const labelList &adjncy, const labelList &xadj, const List< scalar > &cellWeights, labelList &decomp) const |
Serial and/or collect/distribute for parallel operation. More... | |
metisLikeDecomp (const metisLikeDecomp &)=delete | |
No copy construct. More... | |
void | operator= (const metisLikeDecomp &)=delete |
No copy assignment. More... | |
metisLikeDecomp (const label numDomains) | |
Construct with number of domains (no coefficients or constraints) More... | |
metisLikeDecomp (const word &derivedType, const dictionary &decompDict, int select=selectionType::NULL_DICT) | |
Construct for derived type name and decomposition dictionary. More... | |
metisLikeDecomp (const word &derivedType, const dictionary &decompDict, const word ®ionName, int select=selectionType::NULL_DICT) | |
Construct for derived type name, decomposition dictionary and (optional) region name. More... | |
Protected Member Functions inherited from decompositionMethod | |
const dictionary & | findCoeffsDict (const word &coeffsName, int select=selectionType::DEFAULT) const |
Locate coeffsName dictionary or the fallback "coeffs" dictionary. More... | |
decompositionMethod (const label numDomains) | |
Construct with specified number of domains, no coefficients or constraints. More... | |
Additional Inherited Members | |
Static Public Member Functions inherited from decompositionMethod | |
static label | nDomains (const dictionary &decompDict, const word ®ionName="") |
Return region-specific or top-level numberOfSubdomains entry. More... | |
static const dictionary & | optionalRegionDict (const dictionary &decompDict, const word ®ionName) |
Return an optional region-specific dictionary from "regions" sub-dictionary, or dictionary::null on failure. More... | |
static autoPtr< decompositionMethod > | New (const dictionary &decompDict, const word ®ionName="") |
Return a reference to the selected decomposition method, optionally region-specific. More... | |
static void | calcCellCells (const polyMesh &mesh, const labelList &agglom, const label nLocalCoarse, const bool parallel, CompactListList< label > &cellCells) |
Determine (local or global) cellCells from mesh agglomeration. More... | |
static void | calcCellCells (const polyMesh &mesh, const labelList &agglom, const label nLocalCoarse, const bool parallel, CompactListList< label > &cellCells, CompactListList< scalar > &cellCellWeights) |
Determine (local or global) cellCells and face weights from mesh agglomeration. More... | |
Protected Types inherited from decompositionMethod | |
enum | selectionType { DEFAULT = 0, EXACT = 1, MANDATORY = 2, NULL_DICT = 4 } |
Selection type when handling the coefficients dictionary. More... | |
Static Protected Member Functions inherited from decompositionMethod | |
static const dictionary & | findCoeffsDict (const dictionary &dict, const word &coeffsName, int select=selectionType::DEFAULT) |
Locate coeffsName dictionary or the fallback "coeffs" dictionary within an enclosing dictionary. More... | |
Protected Attributes inherited from metisLikeDecomp | |
const dictionary & | coeffsDict_ |
Coefficients for all derived methods. More... | |
Protected Attributes inherited from decompositionMethod | |
const dictionary & | decompDict_ |
Top-level decomposition dictionary (eg, decomposeParDict) More... | |
const dictionary & | decompRegionDict_ |
Region-specific decomposition dictionary information. More... | |
label | nDomains_ |
Number of domains for the decomposition. More... | |
PtrList< decompositionConstraint > | constraints_ |
Optional constraints. More... | |
Scotch domain decomposition.
When run in parallel will collect the whole graph on to the master, decompose and send back. Use ptscotchDecomp for proper distributed decomposition.
Coefficients dictionary: scotchCoeffs, coeffs.
Quoting from the Scotch forum, on the 2008-08-22 10:09, Francois PELLEGRINI posted the following details:
RE: Graph mapping 'strategy' string Strategy handling in Scotch is a bit tricky. In order not to be confused, you must have a clear view of how they are built. Here are some rules: 1- Strategies are made up of "methods" which are combined by means of "operators". 2- A method is of the form "m{param=value,param=value,...}", where "m" is a single character (this is your first error: "f" is a method name, not a parameter name). 3- There exist different sort of strategies : bipartitioning strategies, mapping strategies, ordering strategies, which cannot be mixed. For instance, you cannot build a bipartitioning strategy and feed it to a mapping method (this is your second error). To use the "mapCompute" routine, you must create a mapping strategy, not a bipartitioning one, and so use stratGraphMap() and not stratGraphBipart(). Your mapping strategy should however be based on the "recursive bipartitioning" method ("b"). For instance, a simple (and hence not very efficient) mapping strategy can be : "b{sep=f}" which computes mappings with the recursive bipartitioning method "b", this latter using the Fiduccia-Mattheyses method "f" to compute its separators. If you want an exact partition (see your previous post), try "b{sep=fx}". However, these strategies are not the most efficient, as they do not make use of the multi-level framework. To use the multi-level framework, try for instance: "b{sep=m{vert=100,low=h,asc=f}x}" The current default mapping strategy in Scotch can be seen by using the "-vs" option of program gmap. It is, to date: r { job=t, map=t, poli=S, sep= ( m { asc=b { bnd= ( d{pass=40,dif=1,rem=1} | ) f{move=80,pass=-1,bal=0.002491}, org=f{move=80,pass=-1,bal=0.002491}, width=3 }, low=h{pass=10} f{move=80,pass=-1,bal=0.002491}, type=h, vert=80, rat=0.8 } | m { asc=b { bnd= ( d{pass=40,dif=1,rem=1} | ) f{move=80,pass=-1,bal=0.002491}, org=f{move=80,pass=-1,bal=0.002491}, width=3 }, low=h{pass=10} f{move=80,pass=-1,bal=0.002491}, type=h, vert=80, rat=0.8 } ) }
Given that this information was written in 2008, this example strategy will unlikely work as-is with the more recent Scotch versions. Therefore, the steps for getting the current default strategy from within Scotch, is to do the following steps:
Edit the file system/decomposeParDict
and use the following settings:
method scotch; scotchCoeffs { writeGraph true; }
Run decomposePar
. For example, it will write a file named region0.grf
.
Now, instead of using gmap
, run gpart
with the following command structure to get the default strategy:
gpart \<nProcs\> -vs \<grfFile\>
where:
writeGraph=true
, namely region0.grf
. numberOfSubdomains
defined in the dictionary file. At the end of the execution will be shown a long string, similar to the following example (complete line was cropped at [...]
):
S Strat=m{asc=b{width=3,bnd=d{pass=40,dif=1,rem=0}[...],type=h}
Edit the file system/decomposeParDict
once again and add the strategy
entry as exemplified:
method scotch; scotchCoeffs { //writeGraph true; strategy "m{asc=b{width=3,bnd=d{pass=40,dif=1,rem=0}[...],type=h}"; }
Finally, run decomposePar
once again, to at least test if it works as intended.
Also support for multi-level decomposition by specifying inter-level communication weights:
numberOfSubdomains 2048; coeffs { // Divide into 64 clusters, each of 32 cores domains (64 32); // Inside a cluster the communication weight is 1% of that inbetween // clusters domainWeights (1 0.01); }
Alternatively the first-level decomposition can be left out by assuming weights are 1 for that level:
numberOfSubdomains 2048; coeffs { // Divide into 2048/32 clusters domains (32); // Inside a cluster the communication weight is 1% of that inbetween // clusters domainWeights (0.01); }
gpart
can be found in the current search path by adding the respective bin
folder from the Scotch installation, namely by running the following commands:source $(foamEtcFile config.sh/scotch) export PATH=$PATH:$SCOTCH_ARCH_PATH/bin
Definition at line 259 of file scotchDecomp.H.
|
delete |
No copy construct.
|
explicit |
Construct with number of domains (no coefficients or constraints)
Definition at line 520 of file scotchDecomp.C.
|
explicit |
Construct given decomposition dictionary and optional region name.
Definition at line 527 of file scotchDecomp.C.
|
virtualdefault |
Destructor.
|
protectedvirtual |
Decompose non-parallel.
Hack to test clustering. Note that decomp is non-compact
numbers!
//- Set up variable sizes architecture check ( SCOTCH_archVcmplt(&archdat), "SCOTCH_archVcmplt" );
//- Stategy flags: go for quality or load balance (or leave default) SCOTCH_Num straval = 0; //straval |= SCOTCH_STRATQUALITY; //straval |= SCOTCH_STRATQUALITY;
//- Number of cells per agglomeration //SCOTCH_Num agglomSize = SCOTCH_archSize(&archdat); SCOTCH_Num agglomSize = 3;
//- Build strategy for agglomeration check ( SCOTCH_stratGraphClusterBuild ( &stradat, // strategy to build straval, // strategy flags agglomSize, // cells per cluster 1.0, // weight? 0.01 // max load imbalance ), "SCOTCH_stratGraphClusterBuild" );
Implements metisLikeDecomp.
Definition at line 102 of file scotchDecomp.C.
References UList< T >::cdata(), Foam::check(), refPtr< Container< Type > >::clear(), DebugInfo, UList< T >::empty(), stdFoam::end(), Foam::endl(), Foam::exit(), Foam::FatalError, FatalErrorInFunction, Foam::FatalIOError, FatalIOErrorInFunction, forAll, Foam::Info, Foam::max(), Foam::min(), n, OFstream::name(), Foam::nl, refPtr< Container< Type > >::ref(), List< T >::resize(), List< T >::resize_nocopy(), ConstPrecisionAdaptor< Type, InputType, Container >::set(), PrecisionAdaptor< Type, InputType, Container >::set(), List< T >::setSize(), UList< T >::size(), Foam::sum(), and WarningInFunction.
|
delete |
No copy assignment.
TypeName | ( | "scotch" | ) |
Runtime type information.
|
inlinevirtual |
Knows about coupled boundaries.
Implements decompositionMethod.
Definition at line 335 of file scotchDecomp.H.
|
virtual |
Return for every coordinate the wanted processor number.
Reimplemented from metisLikeDecomp.
Definition at line 539 of file scotchDecomp.C.
References metisLikeDecomp::decompose(), Foam::getGraphPathBase(), mesh, and points.
|
virtual |
Return for every coordinate the wanted processor number.
agglom | agglomeration: fine-to-coarse |
Reimplemented from metisLikeDecomp.
Definition at line 558 of file scotchDecomp.C.
References metisLikeDecomp::decompose(), Foam::getGraphPathBase(), and mesh.
|
virtual |
Return the wanted processor number for every cell.
Reimplemented from metisLikeDecomp.
Definition at line 579 of file scotchDecomp.C.
References metisLikeDecomp::decompose().
|
virtual |
Return the wanted processor number for every cell.
Reimplemented from metisLikeDecomp.
Definition at line 598 of file scotchDecomp.C.
References metisLikeDecomp::decompose().
Foam::labelList decompose |
Inherit all decompose methods.
Definition at line 941 of file decompositionMethod.C.
virtual labelList decompose |
Inherit all decompose methods.
Foam::labelList decompose |
Inherit all decompose methods.
Definition at line 396 of file decompositionMethod.C.
virtual labelList decompose |
Inherit all decompose methods.
virtual labelList decompose |
Inherit all decompose methods.
Foam::labelList decompose |
Inherit all decompose methods.
Definition at line 476 of file decompositionMethod.C.
Foam::labelList decompose |
Inherit all decompose methods.
Definition at line 887 of file decompositionMethod.C.