||March 4, 2010 09:03
Originally Posted by mike_jaworski
Thanks. It's been most helpful. I guess it was just a server glitch.
Sorry for the inconvenience these outages annoy me too. The problem is that the webserver runs out of swap-space when it shouldn't (it's got plenty).
The rest of this posting is of not interest to anyone, I just want to get it off my chest
Nevertheless. I investigated a bit and found that there are several processes that try to convert an animated gif U.gif to a thumbnail. The upload time of that gif was just before tonights outage. Further investigations showed that the convert command is "guarded" by a ulimit that makes sure that a shell command can't grab the whole server. That ulimit is to tight for that particular conversion making it fail with a sigv and it seems that the httpd-thread never recovers from that, thus not freeing the memory. The next time that thumbnail is requested the same thing happens until the server runs out of memory. For the time being I fixed the problem by increasing the amount of memory that is allowed for such operations and now the thumbnail is generated without problems (but this can't be a permanent fix because one of these days a "bigger" file will come along and then it will start all over again)