CFD Online Discussion Forums

CFD Online Discussion Forums (http://www.cfd-online.com/Forums/)
-   OpenFOAM Installation (http://www.cfd-online.com/Forums/openfoam-installation/)
-   -   OpenFOAM 1.7.x git problem (http://www.cfd-online.com/Forums/openfoam-installation/77821-openfoam-1-7-x-git-problem.html)

olauer July 5, 2010 10:28

OpenFOAM 1.7.x git problem
 
Hello guys,

as 1.7.x now is available as git source
(see http://www.openfoam.com/download/git.php )
I tried to get source files from there.

But my result is:

git clone http://github.com/OpenCFD/OpenFOAM-1.7.x.git

Initialized empty Git repository in /home/OpenFOAM/OpenFOAM-1.7.x/.git/
got 28a9bc1de4a03eeda021adb6c528c4ca755a804a
walk 28a9bc1de4a03eeda021adb6c528c4ca755a804a
got aff6cfa058687c5827c80b4aa4f7ed2d6ca83d5e
got 1f3faad7168d9bc63331e9a320a1575ecba1543c
walk 1f3faad7168d9bc63331e9a320a1575ecba1543c
Getting alternates list for http://github.com/OpenCFD/OpenFOAM-1.7.x.git
Getting pack list for http://github.com/OpenCFD/OpenFOAM-1.7.x.git
Getting index for pack dd4ad2b185c4f91aeba18b3b1ac3fedd57bbd70c
Getting index for pack 2758df1ccc1415152fa9b4bcb7b9a081c6c03cf6
Getting pack dd4ad2b185c4f91aeba18b3b1ac3fedd57bbd70c
which contains 2798a1bcf4bd2fde806d6d7ff29d4b448aadf137
error: Unable to get pack file http://github.com/OpenCFD/OpenFOAM-1...d57bbd70c.pack
transfer closed with 46747107 bytes remaining to read
error: Unable to find 2798a1bcf4bd2fde806d6d7ff29d4b448aadf137 under http://github.com/OpenCFD/OpenFOAM-1.7.x.git
Cannot obtain needed blob 2798a1bcf4bd2fde806d6d7ff29d4b448aadf137
while processing commit 1f3faad7168d9bc63331e9a320a1575ecba1543c.
fatal: Fetch failed.

It seems that there is something missing somewhere.

Any ideas what I could do?

Thanks for any help

Oliver

olesen July 6, 2010 02:46

Quote:

Originally Posted by olauer (Post 265777)
Hello guys,

as 1.7.x now is available as git source
(see http://www.openfoam.com/download/git.php )
I tried to get source files from there.

But my result is:

git clone http://github.com/OpenCFD/OpenFOAM-1.7.x.git

Initialized empty Git repository in /home/OpenFOAM/OpenFOAM-1.7.x/.git/
got 28a9bc1de4a03eeda021adb6c528c4ca755a804a
walk 28a9bc1de4a03eeda021adb6c528c4ca755a804a
got aff6cfa058687c5827c80b4aa4f7ed2d6ca83d5e
got 1f3faad7168d9bc63331e9a320a1575ecba1543c
walk 1f3faad7168d9bc63331e9a320a1575ecba1543c
Getting alternates list for http://github.com/OpenCFD/OpenFOAM-1.7.x.git
Getting pack list for http://github.com/OpenCFD/OpenFOAM-1.7.x.git
Getting index for pack dd4ad2b185c4f91aeba18b3b1ac3fedd57bbd70c
Getting index for pack 2758df1ccc1415152fa9b4bcb7b9a081c6c03cf6
Getting pack dd4ad2b185c4f91aeba18b3b1ac3fedd57bbd70c
which contains 2798a1bcf4bd2fde806d6d7ff29d4b448aadf137
error: Unable to get pack file http://github.com/OpenCFD/OpenFOAM-1...d57bbd70c.pack
transfer closed with 46747107 bytes remaining to read
error: Unable to find 2798a1bcf4bd2fde806d6d7ff29d4b448aadf137 under http://github.com/OpenCFD/OpenFOAM-1.7.x.git
Cannot obtain needed blob 2798a1bcf4bd2fde806d6d7ff29d4b448aadf137
while processing commit 1f3faad7168d9bc63331e9a320a1575ecba1543c.
fatal: Fetch failed.

It seems that there is something missing somewhere.

Any ideas what I could do?

Thanks for any help

Oliver

This is a github issue that sometimes occurs with the http:// transport. I've seen it with other projects too. It only seems to occur with large fetches. Smaller, incremental fetches don't seem to be a problem.

If you can use the git:// protocol (at least for your initial fetch) this problem should not occur.

If you can't use the git protocol due to a corporate firewall, you'll need a ssh tunnel.
Otherwise we might need to figure out yet another way for you to get at the source.

olauer July 6, 2010 04:13

Hello Mark,

thank you for your quick response.

GIT protocol is blocked by the corporate firewall but SSH could be an option. Port 22 is also blocked but I can connect to github.com on port 443.

Is there anything else I need to know?

In the moment

git clone git+ssh://github.com/OpenCFD/OpenFOAM-1.7.x.git

ends with

ssh_exchange_identification: Connection closed by remote host
fatal: The remote end hung up unexpectedly

.ssh/config contains

Host github.com
User git
HostName github.com
Port 443

Thank you for any further help

Best regards
Oliver

olesen July 6, 2010 04:51

Quote:

Originally Posted by olauer (Post 265882)
Hello Mark,

thank you for your quick response.

GIT protocol is blocked by the corporate firewall but SSH could be an option. Port 22 is also blocked but I can connect to github.com on port 443.

Is there anything else I need to know?

In the moment

git clone git+ssh://github.com/OpenCFD/OpenFOAM-1.7.x.git

ends with

ssh_exchange_identification: Connection closed by remote host
fatal: The remote end hung up unexpectedly

.ssh/config contains

Host github.com
User git
HostName github.com
Port 443

Thank you for any further help

Best regards
Oliver

I'm not a tunnel expert, but maybe your tunnel isn't getting used?

I have this,

Host github-git
Port 2203
HostName localhost

But still redirect via port 22:

ssh -N -L 2203:github.com:9418 someMachine

AFAIK you don't need the "User git" in the .ssh/config if you are simply fetching from another git repository, only if you are pushing to your own.


In the .git/config, the corresponding entry looks like this:
[remote "github-git"]
url = git://localhost:2203/OpenCFD/OpenFOAM-1.7.x.git
fetch = +refs/heads/*:refs/remotes/github-git/*


I hope any of this help further. If nothing else works, you can fetch from somewhere else (eg, at home) and move across a bundle via usb-stick.
Another alternative could be to have the repo.or.cz site simply mirror the github repo.
The OpenFOAM-1.7.x would be a fork from OpenFOAM-1.6.x, and then set to mirror the github.

olauer July 6, 2010 05:44

Hmm, it seems that ssh works only if you are a registered git user as you need a ssh code pair. Anonymous read access (git clone) only works with git or http protocol :(.
So I will try to get it at home with a live linux cd, but I think I won't be the only one having this problem behind a corporate firewall.

bastil July 15, 2010 06:15

Quote:

Originally Posted by olauer (Post 265902)
but I think I won't be the only one having this problem behind a corporate firewall.

Definitely not. Same problem here. I think we need another solution for this. repo.cz works fine with 1.6.x.

Regards Bastian

olesen July 15, 2010 07:10

Quote:

Originally Posted by olauer (Post 265902)
Hmm, it seems that ssh works only if you are a registered git user as you need a ssh code pair. Anonymous read access (git clone) only works with git or http protocol :(.
So I will try to get it at home with a live linux cd, but I think I won't be the only one having this problem behind a corporate firewall.

Would you have a chance to try a newer git version?
This issue seems to be related:
http://support.github.com/discussion...-with-git-1642

bastil July 15, 2010 07:45

Quote:

Originally Posted by olesen (Post 267419)
Would you have a chance to try a newer git version?
This issue seems to be related:
http://support.github.com/discussion...-with-git-1642

Mark,

thanks for this hint. Using a newer git might be be possible. I have to check this.

Regards Bastian

olesen July 16, 2010 02:09

Quote:

Originally Posted by olesen (Post 267419)
Would you have a chance to try a newer git version?
This issue seems to be related:
http://support.github.com/discussion...-with-git-1642


This is indeed the solution.

Starting somewhere fresh:
Code:

$ mkdir /tmp/git-test
$ cd /tmp/git-test
$ git init
$ git remote add github http://github.com/OpenCFD/OpenFOAM-1.7.x.git

WIth the older git:
Code:

$ git --version

git version 1.6.4.2

$ git fetch github

walk a7b9d1f5aedf70324c6147577ead05068c415d9a
walk 16e22e94a9df524af03f4f75c097ff3a19abc821
Getting alternates list for http://github.com/OpenCFD/OpenFOAM-1.7.x.git
Getting pack list for http://github.com/OpenCFD/OpenFOAM-1.7.x.git
Getting index for pack dd4ad2b185c4f91aeba18b3b1ac3fedd57bbd70c
Getting index for pack 2758df1ccc1415152fa9b4bcb7b9a081c6c03cf6
Getting index for pack e0edc0ec1b175dfec482c608e13cbf44db2e4526
Getting index for pack e9fc5e4f1a4de529d3a709982cff59d9918cd18d
Getting pack dd4ad2b185c4f91aeba18b3b1ac3fedd57bbd70c
 which contains 2798a1bcf4bd2fde806d6d7ff29d4b448aadf137

^C

After a while it hits some pack that it cannot resolve.

On my openSUSE system, I added the corresponding download repository (http://download.opensuse.org/reposit...l:/tools:/scm/) and updated my git.

With the newer git
Code:

$ git --version
 
 git version 1.7.1
 
$ git fetch github

remote: Counting objects: 36653, done.
remote: Compressing objects: 100% (13535/13535), done.
remote: Total 36653 (delta 22741), reused 36315 (delta 22441)
Receiving objects: 100% (36653/36653), 45.40 MiB | 3.44 MiB/s, done.
Resolving deltas: 100% (22741/22741), done.
From http://github.com/OpenCFD/OpenFOAM-1.7.x
 * [new branch]      master    -> github/master

It seems to now use the same type of protocol as the 'git://' uses -- which is to say that the output now looks the same. It not only seems to work without any issues, it also much faster.
My conclusion: upgrading to a new git should solve your problems.

poplar July 17, 2010 01:56

Mark.
Thank you very much. I had the same problems of Olauer, but it now was resloved by using Git 1.7.1.

bastil July 27, 2010 10:50

Our admin now did an update to git here, too:

Code:

user@host:/home/user> git --version
git version 1.7.2

However, I still cannot clone from github:

Code:

user@host:/home/user> git clone http://github.com/OpenCFD/OpenFOAM-1.7.x.git
Cloning into OpenFOAM-1.7.x...
remote: Counting objects: 36766, done.
remote: Compressing objects: 100% (13611/13611), done.
error: RPC failed; result=18, HTTP code = 200MiB | 152 KiB/s

This results in an hanging git for me.... The paraview git-Server works for me as well as repo.cz for 1.6.x.

Regards Bastian

wyldckat July 27, 2010 18:32

Greetings to all!

Uhm, has anyone tried to simply download the git version directly from github via web browser? This link will give you a choice to either download as zip or tar.gz: http://github.com/OpenCFD/OpenFOAM-1...rchives/master

Then, if you really need to use git for posterior minor updates, I suppose that adapting the tutorial that Olesen wrote for merging OpenFOAM git versions should do the trick!

Best regards,
Bruno

bastil July 28, 2010 11:33

Quote:

Originally Posted by wyldckat (Post 269077)
Greetings to all!

Uhm, has anyone tried to simply download the git version directly from github via web browser? This link will give you a choice to either download as zip or tar.gz: http://github.com/OpenCFD/OpenFOAM-1...rchives/master

I have not yet tried this. However to be honest this is not what I am aiming for since it ran fine for 1.6.x behind our company proxy.

Regards Bastian

olesen July 28, 2010 15:43

Quote:

Originally Posted by wyldckat (Post 269077)
Greetings to all!

Uhm, has anyone tried to simply download the git version directly from github via web browser? This link will give you a choice to either download as zip or tar.gz: http://github.com/OpenCFD/OpenFOAM-1...rchives/master

Then, if you really need to use git for posterior minor updates, I suppose that adapting the tutorial that Olesen wrote for merging OpenFOAM git versions should do the trick!

The only problem with this approach is that you get a tar/zip snapshot without any of the git history. You can't easily update anything from git afterwards.

olesen July 28, 2010 15:56

Quote:

Originally Posted by bastil (Post 269179)
I have not yet tried this. However to be honest this is not what I am aiming for since it ran fine for 1.6.x behind our company proxy.

In worst case (necessity is the mother of invention) you can do the following.

One machineA without any firewall issues (eg, at home):
Code:

git clone --no-checkout git://github.com/OpenCFD/OpenFOAM-1.7.x.git
The directory will only contain the .git directory, but that's enough for our purposes.

Then tar/zip it onto USB stick and transport elsewhere.

On machineB that has the firewall issues:
Untar the file.
Get everything out of the git repo:
Code:

git checkout -f HEAD
Now you have two choices, both should work equally well I believe.

Either add in the http:// protocol as a second remote and fetch it:
Code:

git remote add github http://github.com/OpenCFD/OpenFOAM-1.7.x.git
git fetch github

This should work well enough, since git will notice that you've already got most of the stuff and won't need to fetch much.

OR else:
simply change the transport protocol for the previously fetched 'origin' by editing the .git/config file in the OpenFOAM-1.7.x directory and then re-fetch again.

Also note that the 1.7.x repository shares history with 1.6.x, so you can also recycle most of the git contents from there.

bastil July 29, 2010 02:47

Thanks Mark,

I will try your proposal.
However, I still don't understand the problems that github has in combination with http checkout....?

Regards Bastian

olesen July 29, 2010 03:43

Quote:

Originally Posted by bastil (Post 269278)
Thanks Mark,

I will try your proposal.
However, I still don't understand the problems that github has in combination with http checkout....?

I don't get it either.
Note that the workaround only works when the total size of packs that you fetch later is still relatively small (without anyone actually knowing how small it has to be).

bastil July 29, 2010 05:59

Quote:

Originally Posted by olesen (Post 269295)
I don't get it either.
Note that the workaround only works when the total size of packs that you fetch later is still relatively small (without anyone actually knowing how small it has to be).

Ok... this is quite complex. I am wondering if we could not mirror the repository somewhere else (e.g. repo.cz?)?

Edit I simply did it: Use

http://repo.or.cz/r/OpenFOAM-1.7.x.git

to checkout using http. This is just a mirror of git://github.com/OpenCFD/OpenFOAM-1.7.x.git. It worked fine for me.

Regards Bastian

olesen July 29, 2010 06:36

Quote:

Originally Posted by bastil (Post 269326)
Ok... this is quite complex. I am wondering if we could not mirror the repository somewhere else (e.g. repo.cz?)?

Edit I simply did it: Use

http://repo.or.cz/w/OpenFOAM-1.7.x.git

to checkout using http. This is just a mirror of git://github.com/OpenCFD/OpenFOAM-1.7.x.git. It worked fine for me.

Perfect - this should solve any future issues (I hope).


All times are GMT -4. The time now is 02:14.