CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Main CFD Forum

Hard Disk Access Time

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   July 13, 2000, 16:10
Default Hard Disk Access Time
  #1
steve podleski
Guest
 
Posts: n/a
I am running a Fortran CFD application code PC with a 550mhz PIII and Windows NT. I want to know how much total time is used to i/o the hard drive during a typical run of the CFD code. Is there any way I can do this without modifying the code?

My aim is to determine if I can save significant time, in obtaining a solution, by getting a faster hard drive. BTW, I have the whole model in RAM during computations to minimize any hard disk io; any i/o is for interblock interpolation.
  Reply With Quote

Old   July 14, 2000, 01:03
Default Re: Hard Disk Access Time
  #2
John C. Chien
Guest
 
Posts: n/a
(1). I am not a system person, so when running UNIX job on workstation, I normally open another console window and type in "top" to display the CPU and the job(process) activities. (2). You can then monitor the process status every few seconds, and be able to see the CPU time, % usage, memory usage etc... (3). So, on Window/NT, if you can find the similar utility program, then you can do the same thing. In other words, try to find something which will display the real time process activities, including the CPU and memory usage. (4). Using the hard disk is not a good idea at all. It always creat emotional problem for the ANSYS program users in the office. It is always between the huge model size, finite computer memory size, slow computer models, project deadline, trying to put their job on someone's larger and faster computers, etc... (5). It is perfectly all right to present a coarse mesh model solution based on the available computer resources. If you model the problem properly, it still possible to use the coarse mesh solution (properly distributed mesh). (6). The large mesh size will create convergence problem, that is, the convergence will have much larger effect on the solution accuracy than the effect caused by the coarse converged solution. So, the properly distributed mesh and converged solution are two important factors.
  Reply With Quote

Old   July 14, 2000, 06:26
Default Re: Hard Disk Access Time
  #3
Adam Wu
Guest
 
Posts: n/a
In Windows NT, there is also a command called "top" to display the run job status. You can type the command in the dos prompt
  Reply With Quote

Old   July 14, 2000, 07:56
Default Re: Hard Disk Access Time
  #4
JD
Guest
 
Posts: n/a
In Windows NT if you right click on your task bar and choose task manager..... a gui will pop up that helps you to monitor your system resources

or

there may be an option for you to instruct your program to write this information in its output. In StarCD the program will record the cpu time and i/o time.
  Reply With Quote

Old   July 14, 2000, 09:01
Default Re: Hard Disk Access Time
  #5
steve podleski
Guest
 
Posts: n/a
JD,the Task Manager gives NO info on hard drive activity regarding I/O time; and, the CFD program has no option to give that info.
  Reply With Quote

Old   July 14, 2000, 09:03
Default Re: Hard Disk Access Time
  #6
steve podleski
Guest
 
Posts: n/a
Adam, the 'top' command does not work (for me) when invoked from the command window.
  Reply With Quote

Old   July 14, 2000, 09:09
Default Re: Hard Disk Access Time
  #7
steve podleski
Guest
 
Posts: n/a
John, I have no choice in the matter of the use of the hard drive by the CFD code. It generates large temporary files that contain flow field, turbulent, and interpolation coefficients info for the block interface boundaries. Also Windows NT uses about 1GB of virtual memory for these jobs even when there is plenty of RAM not being used; I am baffled by this; why is this virtual memory being used? I'm sure that there is plenty of hard drive I/O when virtual memory is in use. I think that the main limitation in the use of PCs for CFD is the slow I/O of the hard drives.
  Reply With Quote

Old   July 14, 2000, 09:39
Default Re: Hard Disk Access Time
  #8
John C. Chien
Guest
 
Posts: n/a
(1). I think it is the programming philosophy behind the code you are using. (2). The I/O time for hard disk is always slow regardless of the machine or CPU. So, in old days, they have to charge the I/O time in addition to the CPU time in using the super-computer. (3). Well, if you can see the CPU running all the time ,say 98% , then it is not doing the I/O. (if you have two jobs on the same machine, then it will be 50-50 split, and the CPU time for one process will be say 48%.)
  Reply With Quote

Old   July 14, 2000, 09:59
Default Re: Hard Disk Access Time
  #9
Ed Blosch
Guest
 
Posts: n/a
If you are running over your physical memory limit into virtual memory, then the OS is swapping your process into and out of the real memory from disk. Yes, swapping will kill you. I almost always stop swapping processes because they run, say, 20 times slower than those that stay in memory. So I'd guess you need more memory, not more disk.

Sometimes CFD codes will store certain bits and pieces of info on disk, e.g. inter-zone interpolation coefficents. But the designer of the CFD code will only store information that is relatively infrequently used....thus any i/o vs. memory-speed penalties will not be significant. I doubt your CFD code author would use disk improperly....most people writing programs are well aware that disk is much much slower than memory.
  Reply With Quote

Old   July 14, 2000, 10:30
Default Re: Hard Disk Access Time
  #10
Joern Beilke
Guest
 
Posts: n/a
A real OS just caches all the temporary and scatch files as long as there is enough main memory. So even a poorly designed code might run fairly fast. If you insist on using NT you have to accept the limitations.
  Reply With Quote

Old   July 14, 2000, 10:53
Default Re: Hard Disk Access Time
  #11
steve podleski
Guest
 
Posts: n/a
Joern, what is a 'real' OS?
  Reply With Quote

Old   July 14, 2000, 10:57
Default Re: Hard Disk Access Time
  #12
steve podleski
Guest
 
Posts: n/a
Ed, you haven't read my post carefully. I said that NT uses virtual memory even when when much RAM (~400MB)is NOT being used.
  Reply With Quote

Old   July 14, 2000, 11:27
Default Re: Hard Disk Access Time
  #13
Kai Kang
Guest
 
Posts: n/a
Steve,

As JC Chien said, you need to monitor your CPU usage, I have been running Fluent on NT and if the CPU usage close to 100% (single-cpu) then it means all the CFD are stored in memory. If it is fully-coupled solver, then I think everything should be in core. But for segregated solver, each variables are solved iteratively/sequentially, I noticed after each iteration, there is a noticeable disk i/o for a very short period (several seconds). Then for a number of iterations, there is a relatively long disk-swap(tens of seconds). This happens no matter how big your memory is, that's just the way the software was designed. This disk i/o should be minimal, otherwise the code is really bad.

  Reply With Quote

Old   July 14, 2000, 11:49
Default Re: Hard Disk Access Time
  #14
andy
Guest
 
Posts: n/a
A complete guess. Using files is the default communication mechanism that always work (albeit slowly). The author of your code (you have not said who) is using separate streams for each block interface (or group of interfaces). The mechanism used to communicate sensibly exists on the platform the code was developed (e.g. unix) and does not exist on NT or has not been purchased or does not quite conform to specification or you have not configured the software to use it. Check for things like MPI, PVM, signals/sockets, pipes, TCP, etc... in the supporting documentation.

If that gets you know where, it should be easy enough for someone in your computer support group to show you how to connect up the streams in the environment of the program or how to use a chunk of memory as a temporary file system rather than the disk. This is straightforward in unix and is probably doable in NT if you can find someone who knows how.

  Reply With Quote

Old   July 14, 2000, 12:31
Default Re: Hard Disk Access Time
  #15
steve podleski
Guest
 
Posts: n/a
Kai, are you saying that the CPU is not involved in transferring data from RAM to the hard drive?

  Reply With Quote

Old   July 14, 2000, 12:56
Default Re: Hard Disk Access Time
  #16
Kai Kang
Guest
 
Posts: n/a
yes, it surely need cpu, but because the slow disk i/o, the cpu nned not be dedicated for that process, meaning only upto, say 20% of cpu time will be used depending on the disk access time.

  Reply With Quote

Old   July 14, 2000, 13:56
Default Re: Hard Disk Access Time
  #17
Ed Blosch
Guest
 
Posts: n/a
I did read it.

I don't know all the ins and outs of NT, but I assume that it may allocate VM to a process but doesn't actually USE it unless needed. Ideally (and practically), you don't want to use more physical memory than you really have. I merely meant to offer the advice that you check first, before anything else, that you are not swapping.

Try turning down virtual memory (set to 50Mb or something under Control Panel-System-Performance) and rebooting and run again. If you ARE swapping (NT kills your process with an "out-of-virtual-memory" message), then it's time to buy more physical memory, not a faster disk.

Under the NT Task Manager, go to View->Options->Select Columns, and pick Page Faults and Virtual memory --- this will enable you to get some idea of i/o activity. Also, total time spent in i/o will be essentially the "System" time if you time your process. Get the free CYGWIN tools for NT and run your process from the command line as"time executeable-name". Observe system time and compare to total time.

To address your first question, a "typical" CFD code spends about 0.1% of the total run time doing i/o. "Typical" CFD codes do not run in this "out-of-core" mode that you seem to be running in, and only write to disk periodically to save results.

  Reply With Quote

Old   July 14, 2000, 15:18
Default Re: Hard Disk Access Time
  #18
steve podleski
Guest
 
Posts: n/a
Ed, it seems you are assuming parallel operation. This code has that capability in its unix version but not in the NT version. If I had a computer support group, I would not be asking questions on this discussion board . I know no one who have the combined competent knowledge of Windows NT and CFD.
  Reply With Quote

Old   July 14, 2000, 15:46
Default Re: Hard Disk Access Time
  #19
Joern Beilke
Guest
 
Posts: n/a
For the pc there are Linux, Solaris and Unixware. Personally I prefer to use SGI/Irix.

The problem with the "very fast" disks is, that they are noisy and tend to crash faster than the slow ones.
  Reply With Quote

Old   July 14, 2000, 18:49
Default Re: Hard Disk Access Time
  #20
steve podleski
Guest
 
Posts: n/a
Joern, if you are willing to donate $10,000-20,000, I'll buy an SGI or other Unix workstation . Unfortunately, our budget only allows PCs.
  Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Multiple floating objects CKH OpenFOAM Running, Solving & CFD 12 March 21, 2016 14:05
Moving mesh Niklas Wikstrom (Wikstrom) OpenFOAM Running, Solving & CFD 122 June 15, 2014 06:20
Upgraded from Karmic Koala 9.10 to Lucid Lynx10.04.3 bookie56 OpenFOAM Installation 8 August 13, 2011 04:03
Orifice Plate with a fully developed flow - Problems with convergence jonmec OpenFOAM Running, Solving & CFD 3 July 28, 2011 05:24
calculation diverge after continue to run zhajingjing OpenFOAM 0 April 28, 2010 04:35


All times are GMT -4. The time now is 09:06.