CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Main CFD Forum

PETSc Library

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   July 1, 2010, 06:24
Default PETSc Library
  #1
New Member
 
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 8
kostaspan is on a distinguished road
I have installed PETSc libraries on Linux platform for solution of large sparse linear systems which derived by an implicit scheme at finite element code.
The installation integrated successful but i can use it.

I put as includes in C language #include"petscksp.h" but when i try to compile the program, i take errors. If someone know to tell me what flags must i write for compilation and what else muct be specified for right reading of all petsc libraries.
kostaspan is offline   Reply With Quote

Old   July 2, 2010, 22:11
Default
  #2
jed
Member
 
Jed Brown
Join Date: Mar 2009
Posts: 56
Rep Power: 11
jed is on a distinguished road
If the errors are about the missing header, you probably need -I$PETSC_DIR/include -I$PETSC_DIR/$PETSC_ARCH/include. One way to do this is to use PETSc makefiles (which can also handle linking). If it's something else, please send mail to petsc-maint@mcs.anl.gov (you will get a prompt reply).
jed is offline   Reply With Quote

Old   July 3, 2010, 10:48
Default
  #3
New Member
 
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 8
kostaspan is on a distinguished road
I use c of gcc. At ./configuration i put and --with-c-support. Its ok this for c of gcc?
kostaspan is offline   Reply With Quote

Old   July 5, 2010, 02:30
Default
  #4
jed
Member
 
Jed Brown
Join Date: Mar 2009
Posts: 56
Rep Power: 11
jed is on a distinguished road
You can use --with-cc=gcc, but it will usually pick up this compiler anyway. You don't need --with-c-support, that is only if you are compiling PETSc with a C++ compiler, but want to be able to call PETSc from C.
jed is offline   Reply With Quote

Old   July 6, 2010, 03:21
Default
  #5
New Member
 
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 8
kostaspan is on a distinguished road
Yes i put --with-c-support. Do you thing that i have problem with commands in code or compiling?
Now i reinstall --with-cc=gcc.


I want to ask you about functions for matrix construction. When i use MatSetValues function with very large matrix it take me so long time for finish. And is supposed that these constructions must be work quickly that to use in finite element code with very large systems. Have you any ideas for this? What i do?
kostaspan is offline   Reply With Quote

Old   July 6, 2010, 11:19
Default
  #6
jed
Member
 
Jed Brown
Join Date: Mar 2009
Posts: 56
Rep Power: 11
jed is on a distinguished road
MatSetValues is slow because you didn't preallocate correctly. http://www.mcs.anl.gov/petsc/petsc-a...cient-assembly
jed is offline   Reply With Quote

Old   July 6, 2010, 13:24
Default
  #7
New Member
 
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 8
kostaspan is on a distinguished road
Ok. I do some things which i read. But with 5000x5000 elements in matrix its take again much time. Below i give sou the construct of this matrix and please tell me what i do wrong. I have already lost a lot of days for this problem .
Below A: matrix , col3 : pointer to int for number of nonzeros at each row.


MatCreateSeqAIJ(PETSC_COMM_SELF,4000,4000,4000,col 3,&A);

for(n=0;n<4000;n++){
for(m=0;m<4000;m++){
acol1=n;
acol2=m;

if(run.STFMline[n*4000+m]!=0.0){
MatSetValue(A,acol1,acol2,run.STFMline[n*4000+m],INSERT_VALUES);
}
}
}

MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);
MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);



More time take first function.
kostaspan is offline   Reply With Quote

Old   July 6, 2010, 14:05
Default
  #8
jed
Member
 
Jed Brown
Join Date: Mar 2009
Posts: 56
Rep Power: 11
jed is on a distinguished road
Looks like you have a dense matrix containing lots of explicit zeros, is possible that they are just very close to zero (but not exactly zero so they still get inserted)? The dimensions 4000x4000 are not at all large. How many nonzeros do you expect (what is in col3)? Run with -info | grep malloc, the allocation is almost certainly insufficient. To generate an error when inserting a value that has not been allocated, call
Code:
MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE);
after creating the matrix, but before inserting values.
jed is offline   Reply With Quote

Old   July 6, 2010, 16:53
Default
  #9
New Member
 
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 8
kostaspan is on a distinguished road
Is there any function which convert a allocated pointer to Mat type matrix? Why to allocate memory for a matrix which i have already made in pointer (ex. double * or float *) ?

I did all above which you tell me but it takes same time...
kostaspan is offline   Reply With Quote

Old   July 6, 2010, 16:59
Default
  #10
jed
Member
 
Jed Brown
Join Date: Mar 2009
Posts: 56
Rep Power: 11
jed is on a distinguished road
Were there any mallocs during insertion? How long does it take for a particular size? If you have a dense matrix, you can use MatCreateSeqDense() and pass in your array. But it doesn't make sense to hold a sparse matrix with dense storage, usually people create the matrix using MatSetValues rather than assembling into their own structure and copying over.
jed is offline   Reply With Quote

Old   July 6, 2010, 17:06
Default
  #11
New Member
 
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 8
kostaspan is on a distinguished road
How many elements in matrix have you solved with petsc? I will try again other functions and i send you if i have problem.

Thanks for help
kostaspan is offline   Reply With Quote

Old   July 6, 2010, 17:10
Default
  #12
jed
Member
 
Jed Brown
Join Date: Mar 2009
Posts: 56
Rep Power: 11
jed is on a distinguished road
Matrix dimensions of a few hundred million, others have done 500 billion, these matrices usually have from a few 10s to 100s of nonzeros per row. Usually memory limits single-process matrix dimensions to a couple million for 3D problems, a bit more in 2D (many more nonzeros per row in 3D). It would be helpful if you describe what you are trying to do (e.g. where the matrix comes from and what you want to do with it).
jed is offline   Reply With Quote

Old   July 6, 2010, 17:37
Default
  #13
New Member
 
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 8
kostaspan is on a distinguished road
I have made code for spectral finite element analysis (DG method) in 3D for diffusion-convection equation. Until now i used only LU Dec direct method for steady or unsteady (explicit ) schemes because the DG method has an advantage of iterative procedure (i have many small systems with same left side at each element).
Before five days i make the implicit for above equation which requires the creation of stiffness matrix which include all DOFs of all elements. This matrix(by which we discuss) remain const for all time steps but change RHS where you know.
Therefore, at each time step i can solute with GMRES the system for to take the solution at next time. This matrix for 40000 elements and second order approximation (p2) become ~4000000 X4000000. But i here there is the problem. It take much time for allocate memory the petsc lib. I dont have such problems with malloc of c. It work quickly.
And documentation of Petsc dosnt give understandable things... If you can tell me which functions and which way you use...


Kostas
kostaspan is offline   Reply With Quote

Old   July 7, 2010, 07:47
Default
  #14
New Member
 
proximity 6
Join Date: Jul 2010
Posts: 1
Rep Power: 0
proximityinfotech6 is on a distinguished road
I put as includes in C language #include"petscksp.h" but when i try to compile the program, i take errors. If someone know to tell me what flags must i write for compilation and what else muct be specified for right reading of all petsc libraries. E20 fuel, which blends 20 percent ethanol with gasoline, reduces the tail pipe emissions of hydrocarbons and carbon monoxide, compared with traditional gasoline or E10 blends, according to a new study in the Journal of Automobile Engineering. In addition, the research found no measurable impact to vehicle drivability or maintenance in conventional internal combustion engines.
_______________________________________________
Used Auto Parts | Used Car Parts
proximityinfotech6 is offline   Reply With Quote

Old   July 7, 2010, 08:08
Default
  #15
jed
Member
 
Jed Brown
Join Date: Mar 2009
Posts: 56
Rep Power: 11
jed is on a distinguished road
You still haven't answered my question about number of mallocs during insertion, nor told me how slow is "slow". Your matrix dimension of 4M doesn't make sense, 40k Q_2 hex elements produces 27*40k=1.08M, 40k P_2 tet elements produces 10*40k=400k, both of which are significantly smaller than 4M. Additionally, your quoted code is clearly extracting from a dense matrix, which you certainly can't have for large sizes (and doesn't make sense for this problem). Of course creating a dense matrix bigger than available memory will cause lots of swapping and be tremendously slow. You should MatCreateSeqAIJ (or more generically, see the user's manual) with correct precallocation (the number of nonzeros per row depends on which elliptic flux you use) or at least an upper bound (e.g. 27*7 for structured hexes with a local flux), then loop over elements and faces, computing local matrices and inserting them.
jed is offline   Reply With Quote

Old   July 7, 2010, 12:40
Default
  #16
New Member
 
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 8
kostaspan is on a distinguished road
I did relatively many mallocs because i havent completed the optimization. I usually do this in order to have easy procedures and structures as far as possible. Now i correct this and the only values which saved for procedure are these in Sparse Mat . Of course, only non-zeros it must be saved. Finally, i found the petsc functions for efficient creation Mat. I give these lines below:
Therefore all values as blocks of size (sizeb) saved in A which have preallocation memory for only non-zeros blocks.

MatCreateSeqBAIJ(PETSC_COMM_SELF,sizeb,rows,cols,8 ,0,&A);

MatSetValuesBlocked(A,1,&cola,1,&colb,ImpEline,INS ERT_VALUES);

for(f=0;f<run.el[i].tfcnume;f++){
MatSetValuesBlocked(A,1,&cola,1,&colb,ImpFline[f],INSERT_VALUES);

}
}

MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);
MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);



With this i run 100.000 x100000 matrices in a few sec...
kostaspan is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
problem loading UDF library in parallel cluster Veera Gutti FLUENT 8 July 26, 2016 07:24
Huge library of books on FD! More than 680 books!!! Eugeen1948 Main CFD Forum 42 November 12, 2010 17:39
OpenFOAM141dev linking error on IBM AIX 52 matthias OpenFOAM Installation 24 April 28, 2008 15:49
Help! I cann't make library Bowling FLUENT 5 May 12, 2004 04:56
Good library to solve huge linear system Ricardo Bonon Main CFD Forum 3 May 19, 2000 06:24


All times are GMT -4. The time now is 02:50.