tag:blogger.com,1999:blog-4604437282989290623.post165846732525923148..comments2022-11-10T21:19:22.503-08:00Comments on The Harmonious Programmer: Gunicorn dyno death spiral on Heroku -- Part IIPeter J. Farrellhttp://www.blogger.com/profile/01908969705391941469noreply@blogger.comBlogger3125tag:blogger.com,1999:blog-4604437282989290623.post-30969886964017822642015-01-16T10:29:42.000-08:002015-01-16T10:29:42.000-08:00[…] ← Python / Django / Selenium: Set viewport siz...[…] ← Python / Django / Selenium: Set viewport size (window size) Gunicorn dyno death spiral on Heroku — Part II → […]Gunicorn dyno death spiral on Heroku | The Harmonious Programmerhttp://blog.maestropublishing.com/2015/01/12/gunicorn-dyno-death-spiral-on-heroku/noreply@blogger.comtag:blogger.com,1999:blog-4604437282989290623.post-12784999584320188072015-01-16T11:40:22.000-08:002015-01-16T11:40:22.000-08:00The report on the Postgres list actually came from...The report on the Postgres list actually came from New Relic (Amjith). We have tried to work with a Postgres committer to come up with a resolution since then but there was no simple obvious change that could be made in libpq as dynamic unloading of libraries from memory complicated things. We have not heard anything since and since Postgres doesn’t have a public issue tracker that we know of, no where to track what is going on.<br><br>As to this just affecting Gunicorn on Heroku, that isn’t the case. Any multithreaded WSGI server can be affected and we identified it when using mod_wsgi in testing. I have also had sporadic reports of mod_wsgi lock ups ever since mod_wsgi was released. Given that since we tracked this down I have separately pinned two such lock ups seen by mod_wsgi users to the libpq issue, I suspect it may be the problem I have been seeing for many years even when New Relic wasn’t being used.<br><br>Both mod_wsgi and uWSGI have solid mechanisms for dealing with locked up processes and then shutting down, when configured, which aren’t dependent on Python code being able to run as they operate at the C code level. Unfortunately one can only run mod_wsgi on Heroku with Python 3.3 and 3.4. Not possible to run it on 2.7 on Heroku because of limitations in the Heroku Python installation, so if on 2.7 it isn’t really an option in that case.grahamdumpletonnoreply@blogger.comtag:blogger.com,1999:blog-4604437282989290623.post-301319697836346512015-01-16T13:22:20.000-08:002015-01-16T13:22:20.000-08:00Graham, thanks for the reply and adding to the fut...Graham, thanks for the reply and adding to the future finders of this post. You're right that this affects other WSGI servers NOT just Gunicorn, but it fails WAY harder on Gunicorn than others (IMO).<br><br>And it's not specific to Heroku either as you point out. Apologies for inferring otherwise -- this blog post is mostly a one-up copy of a message with Heroku support with little editing. My blog is more of a notebook than anything well written admittedly.<br><br>I thought it was better to post it than there be little information out there because the major number of complaints / forum posts / stack overflow pleas I've found about this are specifically aimed around Heroku. Also, they pretty much wrongly blame everything on Gunicorn and makes app servers like uWSGI / others into magic bullets (which is wrong as well, they just do better when this circumstance comes up).Peter J. Farrellhttp://maepub.wordpress.comnoreply@blogger.com