Pylons and mod_wsgi Memory Management

Posted Thursday, 03 September 2009 at 12:11 by Andrew Liu
Tagged: python | web applications | web development
Read more blogs...

 

Memory management with Pylons on mod_wsgi and apache seems to be one of the major issues, especially when running multiple instances.  I happen to fall in this category, and I have found that each Pylons instance takes up a relatively large chunk of memory.  When working with limited memory constraints (such as a VPS), then this can become a major problem.  Trying to mimimise memory consumption on these thus becomes a critical issue.

 

Searching forums has found the issue to be related to the following:

 

Thread Stack Size

 

High virtual memory usage is almost solely caused by the huge stack allocated on Linux by default (default Linux stack size is 8MB). It is still not quite clear whether this is really troublesome (it looks like in typical configurations this stack remains purely virtual number), but in some configurations may theoretically cause trouble (like reaching VPS limits).


This can be controlled in several ways.


If the startup scripts running the process can be modified, the following can reduce the stack size allocation.  This example reduces the stack size to a 512kB stack.

 

#!/bin/sh
ulimit -s 512

 

This can be done in a python startup script as well with the following:

 

import thread

thread.stack_size(512 * 1024)


 

Pylons Tuning

 

Tuning Pylons can help in reducing memory consumption of the associated processes.  Editing your development.ini or production.ini file with the following can be helpful, although I didn't personally do this myself:

 

[server:main]
threadpool_workers = 10    # 10 is default

[app:main]
sqlalchemy.default.pool_size = 3     # 5 is default
sqlalchemy.default.max_overflow = 7  # pool_size+max_overflow = max simultaneous database sessions


 

Apache Tuning

 

Because of the use of threads with mod_wsgi, ensure that apache has the mpm_prefork and mpm_worker modules installed.

More information on these modules can be found at the apache httpd site:

 

http://httpd.apache.org/docs/2.2/mod/prefork.html

http://httpd.apache.org/docs/2.2/mod/worker.html

 

When configured, try adjusting the options, as the defaults are (in my opinion) very generous.  In particular, the StartServers and MinSpareServers should be reduced to just 2 (unless you are a very heavily hit site).

 

 30 # prefork MPM
 31 # StartServers: number of server processes to start
 32 # MinSpareServers: minimum number of server processes which are kept spare
 33 # MaxSpareServers: maximum number of server processes which are kept spare
 34 # MaxClients: maximum number of server processes allowed to start
 35 # MaxRequestsPerChild: maximum number of requests a server process serves
 36 <IfModule mpm_prefork_module>
 37     StartServers          2
 38     MinSpareServers       2
 39     MaxSpareServers      10
 40     MaxClients          150
 41     MaxRequestsPerChild   0
 42 </IfModule>
 43
 44 # worker MPM
 45 # StartServers: initial number of server processes to start
 46 # MaxClients: maximum number of simultaneous client connections
 47 # MinSpareThreads: minimum number of worker threads which are kept spare
 48 # MaxSpareThreads: maximum number of worker threads which are kept spare
 49 # ThreadsPerChild: constant number of worker threads in each server process
 50 # MaxRequestsPerChild: maximum number of requests a server process serves
 51 <IfModule mpm_worker_module>
 52     StartServers          2
 53     MaxClients          150
 54     MinSpareThreads       5
 55     MaxSpareThreads      75
 56     ThreadsPerChild      25
 57     MaxRequestsPerChild   0
 58 </IfModule>

 

mod_wsgi Tuning

 

Lastly, and this is the one configuration that I believe did the most for me, is to change the WSGI configuration options within apache.

Whereas the mod_wsgi installation and configuration posts give us what is required to make it work, it is the options that I think help reduce the memory.  My setup is:

 

  1 <IfModule mod_wsgi.c>
  2 WSGIDaemonProcess myapplication processes=1 threads=4 maximum-requests=100 inactivity-timeout=300 display-name=myapplication 
  3 WSGIScriptAlias /site.wsgi/ /apps/myapplication/myapplication.wsgi/
  4 WSGIProcessGroup myapplication
  5 </IfModule>

 

The main piece of advice is to use a single process (processes=1).  The number of threads could be higher, but I kept it at 4 for now.  The maximum requests seemed to make a big difference, as originally I had 5000, then reduced to 1000, then even further to 100.  I think this will have to increase depending on the number of hits you expect to receive, but keeping this low means that the life of a thread will be, at most, 100 requests before it resets.  This shouldn't affect a user's experience (possibly just the time taken to destroy and create a new thread).  

 

Conclusion

 

Doing some or all of the above should help you in managing the memory consumption of multiple Pylons processes on mod_wsgi and apache.

 

 

Resources

 

http://code.google.com/p/modwsgi/wiki/ApplicationIssues

http://wiki.pylonshq.com/display/pylonscookbook/Configuring+Pylons+for+low+memory+usage

http://www.mail-archive.com/pylons-discuss@googlegroups.com/msg06081.html

 

 

 

Web News

In security update, Apple backports Meltdown fix to older macOS versions
The fix, previously reserved for newer Macs, is now available on older versions of macOS.
--- 24 Jan 2018 06:10 AM

Intel: Stop firmware patching until further notice
The chip giant believes it has found the root cause of the issue forcing Haswell and Broadwell chips to unexpectedly reboot.
--- 24 Jan 2018 05:58 AM

iOS 11.2.5 out for iPhone and iPad, patches annoying 'text bomb' bug
Bug fixes, new Siri tricks, and support for the upcoming release of the HomePod smartspeaker.
--- 24 Jan 2018 05:52 AM

Amazon Go: The impact on human jobs, retail innovation, Amazon's bottom line
Amazon Go has had a big splash, but now it's time to ponder the big picture and what it means for business, innovation and automation.
--- 24 Jan 2018 05:04 AM

Your iPhone 8 or iPhone X battery could wear out in about 18 months
Wireless charging and fast charging might be a great solution to the pressures that more power-thirsty devices are placing on batteries, but these solutions could very well cause other problems down the line.
--- 24 Jan 2018 04:46 AM

Company Blog


Search Behaviour

Posted Tuesday, 19 October 2010 at 05:58 by Andrew Liu


As an SEO provider, you have one main goal. Get your clients website to show up in search results fo...

Read more...



sm bus drivers missing in Device Manager

Posted Sunday, 18 April 2010
Updated Sunday, 24 February 2013 at 06:39 by Andrew Liu


When installing a new Windows XPinstallation, I seemingly always miss some drivers. One that trouble...

Read more...



Multiple Domains for SEO performance?

Posted Friday, 05 March 2010 at 23:13 by Andrew Liu


Online businesses and websites that cover a broad range of topics or one large topic are sometimes b...

Read more...



Tag Clouds - SEO or not?

Posted Thursday, 04 March 2010 at 04:34 by Andrew Liu


A tag cloud or word cloud is a visual depiction of tags or words related to a site, typically used t...

Read more...



Mozilla Thunderbird and Gmail IMAP Attachments Bug

Posted Wednesday, 03 March 2010 at 20:15 by Andrew Liu


I've been using Gmail since its early inception, and Iwas one of the first to utilise Gmail's IMAPfe...

Read more...



Read more blogs...