[aerosayan]

Many developers seem to accept using third-party libraries like BLAS, LAPACK, METIS, ParMETIS, Eigen 3 (C++ vector algebra library) etc. It makes sense for some teams, because it helps them reduce development time and cost.

I understand there's possibly no right or wrong answer, as every dev's requirement and philosophy will be different. What's your take on the issue?

Have you observed better results, by writing your own code? I remember once paolo said that one of his procedures was performing and scaling better than a commercial CFD code. Have you experienced such improvements in your own code?

edit : removed rant about intel mkl

[naffrancois]

It is always good whenever you can to develop your own routines, reinvent the wheel at first and eventually maybe do better. This may be good on the long run and helps you understand deeper what is behind a package.

But time is the ennemy and you usually need to prioritize things. I'd like to write my own thermo-chemical solver to fit my equations of state and not rely on one I don't control all the details. Same happens with linear solver, efficient parallel I/O, efficient domain decomposition load-balancing, dynamic meshing etc etc. But in the end, my client need my model to match some experimental results on a difficult flow configuration. Whether the code is fast or slow comes always second. Then I focus my work on the model.

[sbaffini]

Originally Posted by aerosayan

Many developers seem to accept using third-party libraries like BLAS, LAPACK, METIS, ParMETIS, Eigen 3 (C++ vector algebra library) etc. It makes sense for some teams, because it helps them reduce development time and cost.

I understand there's possibly no right or wrong answer, as every dev's requirement and philosophy will be different. What's your take on the issue?

Have you observed better results, by writing your own code? I remember once paolo said that one of his procedures was performing and scaling better than a commercial CFD code. Have you experienced such improvements in your own code?

edit : removed rant about intel mkl

Assuming paolo is me, I don't remember the exact circumstances leading to that statement, so let me digress to clarify my point of view.

In general, I don't think there is anything wrong in using a library, especially if you're not on par with the required complexity, robustness, speed, etc. This might be in relation with the developer actual capability and understanding of the matter or just in relation to the required time to reach certain goals.

I also think that in academic settings (or similar ones) this pretty much sounds always as the right thing to do, considering the typical limited scope of the program as a whole. For example, if I want to test my new convective scheme, it totally makes sense that I just try it in OpenFOAM, not that I develop a full CFD code from scratch just to test the scheme.

Also, I claim that any scripted language is, in fact, just that, using someone else library for anything, limiting yourself to the glue code.

But, if we look at the business side of the thing, I think there are several cases where developing your own solution is the best or the only actual option in the end. From one side, I think that if your core business depends on a certain code, you must both know enough of that to be able to implement it in full autonomy and also need to in order to have control on your product.

Let's take the linear algebra case, to stay close to your example. Am I able to develop a similarly equipped algebraic solver for my code as, say, PETSc? Hell no, and I will probably never be, that's why I use PETSc in my code, but I still have a backup internal LU-SGS solution in case people at PETSc, say, simply change their licence or get mad or whatever. I only have two parts of my code that have no backup solution, xml file processing and the SVD from lapack. In both cases, the standard I required for their implementation was clearly out of my league. I will probably try to have backup solution for them in time, but it's not a priority now. This, I think, pretty much clarify what I think about using a library in general.

Let's get now to the point. There is no guarantee that any piece of software, just because the author felt confident enought to put it in an open source library, is actually well tought or even reasonable. It might or not. For me, whoever wrote and/or maintains Lapack is better than me at the SVD, so I don't need to investigate further. But there are things for which I couldn't find a decent implementation and I decided to go on my own.

Sometimes, even if you can't do better, you might just need a very limited subset of a feature that using a full library might be an overkill or just kill performances because the library does too little in the end (besides making maintainability more complex). Do I need a library to sort and search, numerically integrate, find a root? Not me, I can delve deep enough in my problems to be able to come up with a solution that, in the end, is always the better for me. Let me highlight it again, better for me. And I always quantify this before making any decision. That's how I typically build unit tests, investigating the options out there.

If you want, in the end, it is (or should be) always like that: you need something, you see what you got and test it, you take the better one. Sometimes it is clear that you are going to write something better and, simply enough, you do it. So, there is really not a recipe, just common sense.