OpenFOAM on Amazon EC2
 

Cluster compute instances announced...
http://aws.amazon.com/ec2/instance-types/

I hope to be able to share an openfoam-extend machine image and instructions of how to sign up and start using it on the wiki before too long.

Questions, comments, recommendations much appreciated.

Regards,
Ben Racine

And the AMI is?

You can try the CAELinux image in EC2, it contains openfoam 1.7 and many other software packages like Discretizer, enGrid, netgen, Salome, GMSH
see http://www.caelinux.com for more details...

If there is a gpu card we could accelerate OF simulations. We have already good results with simplefoam. See http://sourceforge.net/projects/openfoamspeedit/ for details or drop me a line.

lukazs, you are annoying me and coworkers more and more .you spam in many threads to advertise a commercial offer even if nobody has asked anything remotely related .you even ask people to leave the discussion on this forum and use your site instead .

i remind you of the forum rules that you have agreed :

"Link spammers and other forms of spam are hunted very actively, both automatically and manually."

• "State your affiliation explicitly in your message if you:
  • Recommend a product sold by your employer"
• "Be very restrictive in posting links to other web sites, especially commercial sites"

"Forbidden content:

• Advertising, both commercial and personal"

i ask you to stop your practice .
without regards ,zhijun !

I agree with zhijun. We should not provide links to our web pages. Apologies for that. Our intention was to share our GPL-based technology to OF users, and not to advertise it on the forum.

Therefore, from now on we will stop providing a link to our web page but only to sourceforge where the OF plugin and the opensource speedit library can be downloaded: http://sourceforge.net/projects/openfoamspeedit/

Apologies again to those who felt offended.

Best regards,
Lukasz

Here they are :

'us-east-1' :

• 'openfoam171_0-1ubuntu2' - 'ami-ecf50385',
• 'openfoam171_0-1' - 'ami-62fa0c0b',
• 'openfoam-dev-1.5' - 'ami-98f701f1',
• 'openfoam-1.6-ext' : 'ami-b2fd0cdb',

'eu-west-1' :

• 'openfoam171_0-1ubuntu2' - 'ami-519ca925',
• 'openfoam171_0-1' - 'ami-8d9da8f9',
• 'openfoam-dev-1.5' - 'ami-4d9ca939',
• 'openfoam-1.6-ext' - 'ami-f998ad8d',

'ap-southeast-1' :

• 'openfoam171_0-1ubuntu2' - 'ami-d2423c80',
• 'openfoam171_0-1' - 'ami-28423c7a',
• 'openfoam-dev-1.5' - 'ami-2c423c7e',
• 'openfoam-1.6-ext' - 'ami-3a433d68',

'us-west-1' :

• 'openfoam171_0-1ubuntu2' : 'ami-bb7e2efe',
• 'openfoam171_0-1' - 'ami-8d7e2ec8',
• 'openfoam-dev-1.5' - 'ami-a97e2eec',
• 'openfoam-1.6-ext' - 'ami-5f60301a'

There is a SourceForge project dedicated to the "Cloud Computing for OpenFOAM (R) users" - CloudFlu

And corresponding documentation page

Dear Alexey,

could you tell me something more about "vulashaka" or how to use openfoam with python.
It would be great to get a starting point.

Another question what is the difference between CloudFlu and balloonfoam

thank you in advance.

elvis

We were forced to change the initial "balloon-foam" project name to the new one - "CloudFlu" according to OpenCFD Trademark Guidelines

Hi Ben,
How'd this experiment go? How does OpenFoam run on the Amazon cloud in terms of performance? Can you also run it in parallel?

Thx

Has anyone tried the Cluster Compute instance? I saw some results for the High-CPU Extra Large Compute instance and they weren't that impressive, but the Cluster Compute instance is supposedly tailor made for CFD type computing.

For some reason I can't seem to connect to the amazon machine image posted on openfoam.com. If someone can help me connect to an AMI with OpenFoam, I'd be glad to run a few tests and share the results.

Hi Kevin,
Could you share the sources of your information, please?

From my turn, I read "Benchmarking Amazon EC2 for high-performance scientific computing" written by Edward Walker.
It is not OpenFOAM specific, but what he found out is "approximately 40%–1000% performance degradation in the EC2 High-CPU Medium Instance runs compared to the NCSA (National Center for Supercomputing Applications, USA) runs"
And I have experienced the same situation when I tested my Amazon EC2 front-end for OpenFOAM - cloudFlu

As a consequence, our conclusion for using Amazon EC2 for HPC could be, "use Amazon EC2 if you have no access to the real cluster, only".
At the same time, this investigation by Edward Walker (Oct 2008) were made before Amazon had introduced specific Cluster Computing Instances (July 2010).
So, there is a real hope that things are changed from that time. Unfortunately, I still could see no data or articles which would indicate this shift.

Ye, I will try to prepare specific Amazon EC2 Cluster Computing Instance with OpenFOAM pre-installed.

Best regards,
Alexey

Hi Alexey,

I'm just going by the description by Amazon:

I don't know how Walker was able to test a cluster of High-CPU Medium Instances, as this description says that clusters can only be done with Cluster Compute Instances. But however he did it, I doubt it compares well to the new Cluster Compute instance. Although I am definitely not an expert on any of this stuff.

I did finally did connect to the OpenFoam AMI last night and got it running on a High-CPU instance with solid results. I'm looking to test the Cluster Compute Instance asap.

Hi Kevin,
You see, cluster notion is not defined in hardware terms. According to wikipedia - computer cluster is a group of linked computers, working together closely thus in many respects forming a single computer could be build. You can make a cluster even from a few interconnected PC (know as Beowulf cluster). What enables group of computers become a cluster is low latency interconnection.

I have used this understanding in my cloudFlu implementation and saw the same idea implemented in the widely used common purpose StartCluster - it is really working.

If you look at the Amazon description on account of Cluster Computing Instance, it will be possible to see specification of "I/O Performance: 10 Gigabit Ethernet", which provides comparable performance with the "Infiniband switch" available at 2008 year, claimed by Walker's in his article. So, once we will prepare a Cluster Computing Instance AMI (with OpenFOAM pre-installed), we will be somehow close to the referenced Walker NCSA cluster configuration (except CPU performance; Amazon's should be faster).

I just wasn't sure if you could link instances other than the Cluster Compute Instance (as Amazon's descriptions seems to say that is the only clusterable instance). I suppose there's a way to make it happen, but the fact that the Cluster Compute Instance was specifically built for this purpose gives me confidence.

And like you said, the 10 Gigabit I/O performance is comparable to non-cloud clusters used for CFD type calculations. I did see that that is comparable to some Infiniband Switch specs, but I didn't know which was typical at the time of Walker's report. To hear that he probably had something around 10 Gigabits is very encouraging. I suppose the latency could be problematic, but again, the fact that this instance is build for HPC applications gives me hope.

I don't suppose anyone has an openfoam AMI on US East that I could copy. I'm not familiar enough with openfoam, linux, and amazon AWS to figure out how to do this...it was painful enough just to start running on the EU region.

Hi Kevin,

Is this useful perhaps? http://www.openfoam.com/resources/ec2.php

Hi Gijsbert,

That's actually the tutorial I used to run openfoam on the EU region server, but the best I can do on that server is a single 8-core processor. In order to use the cluster compute instance, I have to be on the US-east region server, so I have to copy the instance from EU to US-east somehow.

Hi Kevin,
Here it is - ami-4fad6a26, Amazon EC2 AMI with the pre-installed OpenFOAM-2.0.0 especially dedicated to Cluster Computing Instances ('cc1.4xlarge' type, in Amazon terms). As it was expected, this image could be run in 'us-east-1' region, only.

Important : It is not enough just to run a number of AMI to get a cluster. Configuration of the cluster itself from the given set of machine instances is the next important step. And this requires a lot of Linux experience and accuracy. I could suggest to use cloudFlu library (especially developed for cloud based HPC OpenFOAM calculations), to automate all these steps and make your way to cloud computing really simple.

Enjoy,
Alexey

Cluster Computing Instance for OF-1.6-ext
 

Hi Alexey and all the others,

my goal is to do cluster computing on amazon using OpenFOAM-1.6-ext. I found this thread and cloudFlu and am now trying to get it working.

Using cloudflu I only get ami-4fad6a26 working, that was mentioned in the last post.

Am I right that the ami's mentioned in an earlier post by Alaxey for earlier Openfoam versions and the extended project version do not work anymore for the new cluster computing instances (like e.g. cc2.8xlarge).

Did anyone already prepare such an AMI or can give me a bit support in building it up by myself.
Thanks in advance!

Best regards,
Matthias