Using database files in OpenFOAM
Hello all,
I am trying to use sql database files (.db) with OpenFOAM for my application. I am able to run codes that successfully read from the database files during the simulation. It also works with parallel processors. However, I suspect that the simulation slows down since all processors are accessing the database file from the same location (in my case folder). Is there perhaps a way to distribute the database file in each processor folder so as to speed up operation. I tried copying the database file to each processor folder but it is not being read. I use the following code in "main" to open the database file: Code:
if (sqlite3_open("Database.db", &db) != SQLITE_OK) Thanks! Shyam |
Hi Shyam,
I've had a somewhat similar problem and switched from a SQLite database to a mySQL database, for that very reason. This database is not hosted on the machines I use to run the simulations on, which distributes the IO. Further on I created a data structure that takes care of the data being only read once *and* only by the master process (Pstream::master()). I then use Pstream::scatterList, to scatter the data, which is stored in a List<T>, to all processes. Maybe this helps. Jens |
Using database files in OpenFOAM
Hello Jens,
Thank you for your reply. It seems that mySQL is installed and available on the processors i use to run my simulations. I'll have to see how much of the SQL programming code i need to change to go from SQLite to mySQL! However, in my code i go through cell and patch arrays and query the table in a sequential manner every time step. Since the queries themselves change every time step, I do not think i can query just once like in your simulation. Else, I would have to carry around a lot of data which is something i want to avoid. I am still hoping that there is a way to allow each processor to access their own copy of the database. There appears to be a distributed data option in OF. Not sure if that can accomplish what i need. Thanks! Shyam |
Hi,
code change: I would strongly suggest, that you create a class hierarchy, that wraps the database communication in a separate layer. Doing so enables you to change the type of database later on, without affecting the rest of the code. mySQL vs SQLite: I doubt that you would have to change a lot of the SQL syntax. Querying: I do the same stuff, basically. That's why I cache the data on the master node. Otherwise the database communication - in the case of my application - way to dominant. The entire process would be slow as hell. Access: If you can really separate the data on down to each processor, you should be fairly safe. mySQL can handle quite a lot of queries. |
All times are GMT -4. The time now is 20:02. |