
[Sponsors] 
July 1, 2010, 06:24 
PETSc Library

#1 
New Member
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 7 
I have installed PETSc libraries on Linux platform for solution of large sparse linear systems which derived by an implicit scheme at finite element code.
The installation integrated successful but i can use it. I put as includes in C language #include"petscksp.h" but when i try to compile the program, i take errors. If someone know to tell me what flags must i write for compilation and what else muct be specified for right reading of all petsc libraries. 

July 2, 2010, 22:11 

#2 
Member
Jed Brown
Join Date: Mar 2009
Posts: 56
Rep Power: 10 
If the errors are about the missing header, you probably need I$PETSC_DIR/include I$PETSC_DIR/$PETSC_ARCH/include. One way to do this is to use PETSc makefiles (which can also handle linking). If it's something else, please send mail to petscmaint@mcs.anl.gov (you will get a prompt reply).


July 3, 2010, 10:48 

#3 
New Member
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 7 
I use c of gcc. At ./configuration i put and withcsupport. Its ok this for c of gcc?


July 5, 2010, 02:30 

#4 
Member
Jed Brown
Join Date: Mar 2009
Posts: 56
Rep Power: 10 
You can use withcc=gcc, but it will usually pick up this compiler anyway. You don't need withcsupport, that is only if you are compiling PETSc with a C++ compiler, but want to be able to call PETSc from C.


July 6, 2010, 03:21 

#5 
New Member
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 7 
Yes i put withcsupport. Do you thing that i have problem with commands in code or compiling?
Now i reinstall withcc=gcc. I want to ask you about functions for matrix construction. When i use MatSetValues function with very large matrix it take me so long time for finish. And is supposed that these constructions must be work quickly that to use in finite element code with very large systems. Have you any ideas for this? What i do? 

July 6, 2010, 11:19 

#6 
Member
Jed Brown
Join Date: Mar 2009
Posts: 56
Rep Power: 10 
MatSetValues is slow because you didn't preallocate correctly. http://www.mcs.anl.gov/petsc/petsca...cientassembly


July 6, 2010, 13:24 

#7 
New Member
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 7 
Ok. I do some things which i read. But with 5000x5000 elements in matrix its take again much time. Below i give sou the construct of this matrix and please tell me what i do wrong. I have already lost a lot of days for this problem .
Below A: matrix , col3 : pointer to int for number of nonzeros at each row. MatCreateSeqAIJ(PETSC_COMM_SELF,4000,4000,4000,col 3,&A); for(n=0;n<4000;n++){ for(m=0;m<4000;m++){ acol1=n; acol2=m; if(run.STFMline[n*4000+m]!=0.0){ MatSetValue(A,acol1,acol2,run.STFMline[n*4000+m],INSERT_VALUES); } } } MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY); MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY); More time take first function. 

July 6, 2010, 14:05 

#8 
Member
Jed Brown
Join Date: Mar 2009
Posts: 56
Rep Power: 10 
Looks like you have a dense matrix containing lots of explicit zeros, is possible that they are just very close to zero (but not exactly zero so they still get inserted)? The dimensions 4000x4000 are not at all large. How many nonzeros do you expect (what is in col3)? Run with info  grep malloc, the allocation is almost certainly insufficient. To generate an error when inserting a value that has not been allocated, call
Code:
MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE); 

July 6, 2010, 16:53 

#9 
New Member
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 7 
Is there any function which convert a allocated pointer to Mat type matrix? Why to allocate memory for a matrix which i have already made in pointer (ex. double * or float *) ?
I did all above which you tell me but it takes same time... 

July 6, 2010, 16:59 

#10 
Member
Jed Brown
Join Date: Mar 2009
Posts: 56
Rep Power: 10 
Were there any mallocs during insertion? How long does it take for a particular size? If you have a dense matrix, you can use MatCreateSeqDense() and pass in your array. But it doesn't make sense to hold a sparse matrix with dense storage, usually people create the matrix using MatSetValues rather than assembling into their own structure and copying over.


July 6, 2010, 17:06 

#11 
New Member
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 7 
How many elements in matrix have you solved with petsc? I will try again other functions and i send you if i have problem.
Thanks for help 

July 6, 2010, 17:10 

#12 
Member
Jed Brown
Join Date: Mar 2009
Posts: 56
Rep Power: 10 
Matrix dimensions of a few hundred million, others have done 500 billion, these matrices usually have from a few 10s to 100s of nonzeros per row. Usually memory limits singleprocess matrix dimensions to a couple million for 3D problems, a bit more in 2D (many more nonzeros per row in 3D). It would be helpful if you describe what you are trying to do (e.g. where the matrix comes from and what you want to do with it).


July 6, 2010, 17:37 

#13 
New Member
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 7 
I have made code for spectral finite element analysis (DG method) in 3D for diffusionconvection equation. Until now i used only LU Dec direct method for steady or unsteady (explicit ) schemes because the DG method has an advantage of iterative procedure (i have many small systems with same left side at each element).
Before five days i make the implicit for above equation which requires the creation of stiffness matrix which include all DOFs of all elements. This matrix(by which we discuss) remain const for all time steps but change RHS where you know. Therefore, at each time step i can solute with GMRES the system for to take the solution at next time. This matrix for 40000 elements and second order approximation (p2) become ~4000000 X4000000. But i here there is the problem. It take much time for allocate memory the petsc lib. I dont have such problems with malloc of c. It work quickly. And documentation of Petsc dosnt give understandable things... If you can tell me which functions and which way you use... Kostas 

July 7, 2010, 07:47 

#14 
New Member
proximity 6
Join Date: Jul 2010
Posts: 1
Rep Power: 0 
I put as includes in C language #include"petscksp.h" but when i try to compile the program, i take errors. If someone know to tell me what flags must i write for compilation and what else muct be specified for right reading of all petsc libraries. E20 fuel, which blends 20 percent ethanol with gasoline, reduces the tail pipe emissions of hydrocarbons and carbon monoxide, compared with traditional gasoline or E10 blends, according to a new study in the Journal of Automobile Engineering. In addition, the research found no measurable impact to vehicle drivability or maintenance in conventional internal combustion engines.
_______________________________________________ Used Auto Parts  Used Car Parts 

July 7, 2010, 08:08 

#15 
Member
Jed Brown
Join Date: Mar 2009
Posts: 56
Rep Power: 10 
You still haven't answered my question about number of mallocs during insertion, nor told me how slow is "slow". Your matrix dimension of 4M doesn't make sense, 40k Q_2 hex elements produces 27*40k=1.08M, 40k P_2 tet elements produces 10*40k=400k, both of which are significantly smaller than 4M. Additionally, your quoted code is clearly extracting from a dense matrix, which you certainly can't have for large sizes (and doesn't make sense for this problem). Of course creating a dense matrix bigger than available memory will cause lots of swapping and be tremendously slow. You should MatCreateSeqAIJ (or more generically, see the user's manual) with correct precallocation (the number of nonzeros per row depends on which elliptic flux you use) or at least an upper bound (e.g. 27*7 for structured hexes with a local flux), then loop over elements and faces, computing local matrices and inserting them.


July 7, 2010, 12:40 

#16 
New Member
kostas
Join Date: Mar 2010
Posts: 12
Rep Power: 7 
I did relatively many mallocs because i havent completed the optimization. I usually do this in order to have easy procedures and structures as far as possible. Now i correct this and the only values which saved for procedure are these in Sparse Mat . Of course, only nonzeros it must be saved. Finally, i found the petsc functions for efficient creation Mat. I give these lines below:
Therefore all values as blocks of size (sizeb) saved in A which have preallocation memory for only nonzeros blocks. MatCreateSeqBAIJ(PETSC_COMM_SELF,sizeb,rows,cols,8 ,0,&A); MatSetValuesBlocked(A,1,&cola,1,&colb,ImpEline,INS ERT_VALUES); for(f=0;f<run.el[i].tfcnume;f++){ MatSetValuesBlocked(A,1,&cola,1,&colb,ImpFline[f],INSERT_VALUES); } } MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY); MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY); With this i run 100.000 x100000 matrices in a few sec... 

Thread Tools  
Display Modes  


Similar Threads  
Thread  Thread Starter  Forum  Replies  Last Post 
problem loading UDF library in parallel cluster  Veera Gutti  FLUENT  7  July 3, 2011 09:15 
Huge library of books on FD! More than 680 books!!!  Eugeen1948  Main CFD Forum  42  November 12, 2010 17:39 
OpenFOAM141dev linking error on IBM AIX 52  matthias  OpenFOAM Installation  24  April 28, 2008 15:49 
Help! I cann't make library  Bowling  FLUENT  5  May 12, 2004 04:56 
Good library to solve huge linear system  Ricardo Bonon  Main CFD Forum  3  May 19, 2000 06:24 