CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > ANSYS > CFX

User fortran to input/output file in a parallel run

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree2Likes
  • 1 Post By doublestrong
  • 1 Post By juliom

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   March 30, 2017, 06:18
Default User fortran to input/output file in a parallel run
  #1
New Member
 
Tony
Join Date: Mar 2016
Posts: 24
Rep Power: 10
doublestrong is on a distinguished road
I managed to use user fortran to input/output file in a parallel run, but I got confused for the role of master processor.

I am using user fortran to assign value of the pressure at the outlet of my computational domain. The user fortran code is a one-dimensional unsteady simulation, whose time step is the same as three-dimensional simulation. Now you see that I need to save transient results of user fortran and, sometimes, read them as the intial solution for user fortran.

My simulation is in a parallel run. I found the help document already mentioned input/output file in a parallel run. Here it is:

The CFX-Solver is designed so that all of the numerically intensive tasks can be performed in parallel. Administrative tasks, such as simulation control and user interaction, as well as the input/output phases of a parallel run, are performed serially by the master process. This approach guarantees good parallel performance and scalability of the parallel code, as well as ensuring that the input/output files are like those of a sequential run.


Therefore I only use master processor to output file in my user fortran code, here is the code block about output:

if ((PAR_MODE .EQ. 1).OR.(RUN_MODE .EQ. 0)) then
write(ctstep,*) currenttimestep
open(12,FILE=('pipeSolutionFile'//trim(adjustl(ctstep))))
write(12,'(<xnodes>F12.4)') u
write(12,'(<xnodes>F12.4)') a
write(12,'(<xnodes>F12.4)') aA
write(12,'(<xnodes>F12.4)') lamda
write(12,'(<xnodes>F12.4)') beta
close(12)
endif


However, when my code read file at the very beginning to get initial solution, the solver crashed in a paraller run but worked well in a serial run if I only user master processor to read. After I deleted code about only using master processor to read, the input phrase got to work well in a parallel run. Here is the code block about input:

! call GET_PARALLEL_INFO ('PARMOD',PAR_MODE,CNAME,CRESLT)
! call GET_PARALLEL_INFO ('RUNMOD',RUN_MODE,CNAME,CRESLT)
! if ((PAR_MODE .EQ. 1).OR.(RUN_MODE .EQ. 0)) then
open(11,FILE='../pipeSolutionFile')
read(11,'(<xnodes>F12.4)') u
read(11,'(<xnodes>F12.4)') a
read(11,'(<xnodes>F12.4)') aA
read(11,'(<xnodes>F12.4)') lamda
read(11,'(<xnodes>F12.4)') beta
close(11)
! endif


All the u, a, aA, lamda and beta are stated by SAVE function which I think would help to share data between master and slave processors.

I have two questions:
1. when a simulation runs at the very begining, is master processor the first one to call user routine? If it is not, then the slave provessor which is called firstly may need to do the input phase in my case.
2. are slave processors able to excute READ but not able to excute WRITE?

Welcome to discuss,
Thanks.
doublestrong is offline   Reply With Quote

Old   March 30, 2017, 09:33
Default
  #2
Senior Member
 
Julio Mendez
Join Date: Apr 2009
Location: Fairburn, GA. USA
Posts: 290
Rep Power: 18
juliom is on a distinguished road
Send a message via Skype™ to juliom
Are you using Distribuitrd or shared memory approach ? I also use parallel computing for my codes but I always run away of parallel writing and reading. There are two reasons: race conditions and overhead. I'm sure there are libraries and good practices for reading and writing in parallel computing but I'd rather to pay a price in the last stage of my computations with writing or when I'm reading.
juliom is offline   Reply With Quote

Old   March 30, 2017, 09:43
Default
  #3
Senior Member
 
Julio Mendez
Join Date: Apr 2009
Location: Fairburn, GA. USA
Posts: 290
Rep Power: 18
juliom is on a distinguished road
Send a message via Skype™ to juliom
I forgot to answer your questions. The first questions depends if you are using distribuitrd or shared memory. The second, slave can do both. The problem is: are you reading and writing in the proper manner ? Is your final file contiguos as you expected ?
juliom is offline   Reply With Quote

Old   March 30, 2017, 22:41
Default
  #4
New Member
 
Tony
Join Date: Mar 2016
Posts: 24
Rep Power: 10
doublestrong is on a distinguished road
Hi Julio, I have no idea about distributed and shared memory approach. I only know that I select Distributed Parallel or Local Parallel or Serial while defining run. Based on you answer, I think I may be using distributed memory approach. The output file is what I expected and the data read from input file is also good.

Could you tell me a little bit more about distributed memory approach, shared memory approach, race conditions and overhead? It is also great that if I could know where to find them in the help document or other materials.
doublestrong is offline   Reply With Quote

Old   March 30, 2017, 22:49
Default
  #5
New Member
 
Tony
Join Date: Mar 2016
Posts: 24
Rep Power: 10
doublestrong is on a distinguished road
By the way, I have a flag variable in my user fortran code. The flag is used to monitor whether this is a new time step for 3D simulation. If it is, then the main body of the user fortran code would be run and, in the case of the first time step, the fortran code will read from input file to get an initial solution; if it is not a new time step, then the main body will be skipped and results computed and stored in the call of former time step would be returned directly. I do this to avoid the main body being excuted frequently and unnecessarily by all master and slave processors, since I just want it to be run once during one time step of 3D simulation. Although the code is called a lot of times, the time cost could be reduced.
amd18 likes this.
doublestrong is offline   Reply With Quote

Old   March 31, 2017, 09:15
Default
  #6
Senior Member
 
Julio Mendez
Join Date: Apr 2009
Location: Fairburn, GA. USA
Posts: 290
Rep Power: 18
juliom is on a distinguished road
Send a message via Skype™ to juliom
Are you using a commercial code ? what you said is a flag I think is an instruction from the main code. Nevertheless, if someone else wrote the code for you, you are ver ucky because writing parallel coding is not easy and it is not a straightforward task, since you need to consider what I said before.
Distributed memory is basically OpenMP, in this approach your spawn the instruction among the threads (multi-cores processors). Off course this is more complex but in this way you can easily get the gist of it. In this approach all the variables are shared, in the memory. Imagine that yours threads access the samve variable on the ram and they can overwrite what other thread did. Now you need to structure your code in such a way that this problem does not happen. Usually you declare private vaiables in this approach.

The distributed memory is MPI (Message PAssing Interface). This approach is more complex because now you have a section of the memory private for each task, now you do not talk about threads now are tasks. This apporach is more complex.

You can visualize OpenMP as multihreads computation. All the computations are performed in the same node (or Motherboard). MPI you send information using a network (a very fast network usually infinityband is a especification) to different nodes or computer.

In both approaches you need to deal with race conditions (one task finish before other), overhead (the price you pay for communication), complexity (this is not a serial approach) and writing and reading using parallel environment is very complex and is not efficient, because you need to make sure that you are reading the information in the same way you have that information in memory on in a file. Usually this information is contiguous in the memory, it means 1...2 ..... 3..... 4..... 5.... If you use different threads or tasks there is no guarantee that your solution will be contiguous too.

This is a VERY fast explanation and maybe a computer scientist will disagree with me, but as engineer that uses supercomputer to have my work done this is the easiest way I understood this.
amd18 likes this.
juliom is offline   Reply With Quote

Reply

Tags
parallel, read, user fortran, user routine, write

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
polynomial BC srv537 OpenFOAM Pre-Processing 4 December 3, 2016 10:07
[Other] Adding solvers from DensityBasedTurbo to foam-extend 3.0 Seroga OpenFOAM Community Contributions 9 June 12, 2015 18:18
[swak4Foam] build problem swak4Foam OF 2.2.0 mcathela OpenFOAM Community Contributions 14 April 23, 2013 14:59
[swak4Foam] funkySetFields compilation error tayo OpenFOAM Community Contributions 39 December 3, 2012 06:18
Version 15 on Mac OS X gschaider OpenFOAM Installation 113 December 2, 2009 11:23


All times are GMT -4. The time now is 13:08.