In most cases, it is sufficient to use the automatic domain decompostion which is provided for within the parallel PHOENICS solver. In some cases though, the user may prefer to choose how the domain is partitioned, ie apply a manual decompostion. To do this the user should supply a file PARDAT in the working directory which describes how the domain is to be split between the processes.
Description of file PARDAT
File PARDAT serves the manual decomposition in parallel run of PHOENICS. If it is absent then the automatic decomposition is made.
The format of file PARDAT depends on kind of decomposition:
First type of decomposition has very simple format of file PARDAT. User should set three lines,
for example:
NXSD = 1 NYSD = 1 NZSD = 2
The first line shows that number of sub-domains in X-direction is one, the domain is not split in the X-direction. The second line shows that number of sub-domains in the Y-direction is one too. The third line shows that number of sub-domains in the Z-direction is two, the domain is split in the Z direction in two equal (or almost equal) parts.
This simple type of manual decomposition may alternatively be achieved by adding settings to the Q1 file.
For example, to split the domain into 8 sub-domains (2 in each direction), the following statements must be set in the Q1 file:
LG(2)=T IG(1)=2 IG(2)=2 IG(3)=2
The logical LG(2) will instruct the splitter to by-pass the automatic domain decomposition, and split the domain according to the settings defined in the IG array as follows.
IG(1) specifies the number of sub-domains in the x-direction;
IG(2) specifies the number of sub-domains in the y-direction;
IG(3) specifies the number of sub-domains in the z-direction.
Second type of decomposition allows dividing of domain on non-equal parts. It is helpful
sometimes when user has PCs with non-equal parameters connected by network. Therefore for better
performance user wants to set more cells to more powerful PC. In this case user should set three
lines, for example:
NXSD = 1 38 NYSD = 2 36 16 NZSD = 1 28
The first numbers after sign ‘=’ are numbers of sub-domains in current direction. Next number (or numbers) is number of cells in current direction for each sub-domain. Sum of cells for all sub-domains should be equal the total number of cells in each direction. The sample demonstrates case when there is not splitting in X and Z direction (number of cells in X-direction is 38, number of cells in Z-direction is 28) and domain is divided in Y direction on two sub-domain, first has 36 cells and second has 16 cells in Y direction.
Third type of decomposition deals with Multi-Block cases. In this case the first line should be
MULTIBLOCK = T
The comment line should be started from sign’*’. Next lines demonstrates the sample of well-commented PARDAT file:
NUMBLOCKS = 5 NUMPROCS = 3 * description of first block * firstly we set current block * CUR_BLOCK = <number of current block> <amount of processes> CUR_BLOCK = 1 2 * secondly we enumerate process for current block * PROC = <local number of process in structure TGlobBlock for current block> <global processor ID> PROC = 1 0 * for each process we should set global BOUND BOX (for current block) * BOUND_BOX = <IXF> <IXL> <IYF> <IYL> <IZF> <IZL> BOUND_BOX = 1 16 1 16 1 5 PROC = 2 1 BOUND_BOX = 1 16 1 16 6 9 * * next blocks here * block 2 in process 2 CUR_BLOCK = 2 2 PROC = 1 1 BOUND_BOX = 1 16 1 16 11 14 PROC = 2 2 BOUND_BOX = 1 16 1 16 15 19 * block 3 in process 0 and 1 CUR_BLOCK = 3 1 PROC = 1 1 BOUND_BOX = 1 4 1 4 21 30 * block 4 in processes 0, 1 and 2 CUR_BLOCK = 4 1 PROC = 1 1 BOUND_BOX = 1 4 1 4 32 61 * block 5 in processes 1 and 2 CUR_BLOCK = 5 1 PROC = 1 1 BOUND_BOX = 1 4 1 4 63 72