| internetscooter |
February 12, 2014 00:41 |
Different value of Cd for different number of processors
Hi,
I am what can best be described as an OpenFOAM hobbyist with access to a HPC facility.
I am tweaking the following to run my own Vespa model /usr/local/openfoam/2.2.2/tutorials/incompressible/pisoFoam/les/motorBike/
I am playing with scaling and am noticing that the Cd value changes based on number of cores - is this expected? I would have thought it would be somewhat deterministic - is it or should I be looking for an error in the way I am approaching?
Here are some results - the last Cd value in log.simpleFoam followed by the last in log.pisoFoam, both differ from run to run.
Code:
grep Cd 20140212_*/*.o*
20140212_0843/nodes=1:ppn=12.o1132447: Cd = 0.650766
20140212_0843/nodes=1:ppn=12.o1132447: Cd = 0.628932
20140212_084738/nodes=1:ppn=10.o1132450: Cd = 0.650916
20140212_084738/nodes=1:ppn=10.o1132450: Cd = 0.628049
20140212_090106/nodes=2:ppn=10.o1132451: Cd = 0.651073
20140212_090106/nodes=2:ppn=10.o1132451: Cd = 0.628595
20140212_090150/nodes=3:ppn=10.o1132452: Cd = 0.650694
20140212_090150/nodes=3:ppn=10.o1132452: Cd = 0.626113
20140212_090216/nodes=4:ppn=10.o1132453: Cd = 0.650944
20140212_090216/nodes=4:ppn=10.o1132453: Cd = 0.638436
20140212_122959/nodes=5:ppn=10.o1132454: Cd = 0.650798
20140212_122959/nodes=5:ppn=10.o1132454: Cd = 0.640691
20140212_150541/nodes=6:ppn=10.o1132455: Cd = 0.650733
20140212_150541/nodes=6:ppn=10.o1132455: Cd = 0.631151
20140212_151006/nodes=7:ppn=10.o1132456: Cd = 0.651895
20140212_151006/nodes=7:ppn=10.o1132456: Cd = 0.625597
As it might help someone - here's how I do it...
In my PBS script...
Code:
echo Running with $PBS_NP processors across $PBS_NUM_NODES nodes. $PBS_NUM_PPN procs per node
rm ./vespa_env
echo "vespa_env_procs $PBS_NP;" >> ./vespa_env
echo "vespa_env_num_nodes $PBS_NUM_NODES;" >> ./vespa_env
echo "vespa_env_num_ppn $PBS_NUM_PPN;" >> ./vespa_env
cat ./vespa_env
In bash scripts I use the PBS variables directly. in dictionaries I...
e.g. motorBike/system/decomposeParDict.hierarchical
Code:
#include "../../vespa_env"
numberOfSubdomains $vespa_env_procs;
method hierarchical;
// paulmc - we will split the problem across cores in X and nodes in y
hierarchicalCoeffs
{
n ( $vespa_env_num_ppn $vespa_env_num_nodes 1 );
delta 0.001;
order xyz;
}
Cheers,
Paul |
