Installing WSGI adaptor for Apache and Pylons using mod_wsgi

Posted Thursday, 04 March 2010 at 03:36 by Andrew Liu
Tagged: python | web applications | linux | fedora | web development
Read more blogs...

OS: Fedora 10

Ideally, we want to run WSGI applications under our Apache web server, rather than on standalone ports. We do so by using mod_wsgi, the Apache module developed by Graham Dumpleto. This tutorial is largely based on components from this post by Albert Valverde. Our WSGI application we are trying to run is a Pylons installations.

1. Install mod_wsgi module

Firstly, we need to grab the mod_wsgi module. We can grab it from the SVN repository. If you dont have SVN installed, you can grab a default installation of it.

> yum install svn

Loaded plugins: refresh-packagekit
Setting up Install Process
Parsing package install arguments
Resolving Dependencies
...

Complete!
> svn co http://modwsgi.googlecode.com/svn/trunk modwsgi
A    modwsgi/mod_wsgi
A    modwsgi/mod_wsgi/configure
A    modwsgi/mod_wsgi/LICENCE
A    modwsgi/mod_wsgi/configure.ac
A    modwsgi/mod_wsgi/Makefile-1.X.in
A    modwsgi/mod_wsgi/Makefile-2.X.in
A    modwsgi/mod_wsgi/mod_wsgi.c
A    modwsgi/mod_wsgi/README
A    modwsgi/README
Checked out revision 1210.

Try to compile the source code.

> cd modwsgi

> cd mod_wsgi

> ./configure

checking for apxs2... no
checking for apxs... no
checking Apache version... ./configure: line 1695: apxs: command not found
./configure: line 1695: apxs: command not found
./configure: line 1696: /: is a directory

checking for python... /usr/local/Python/bin/python
./configure: line 1843: apxs: command not found
configure: creating ./config.status
config.status: error: cannot find input file: Makefile.in

More often than not, it will complain that you don't have "apxs" installed. We will need the source code for apache for this. We can grab the httpd-devel package, then finish off the standard "./configure; make; make install" routine to build the module.

> yum install httpd-devel

Loaded plugins: refresh-packagekit
Setting up Install Process
Parsing package install arguments
Resolving Dependencies
...

Complete!
> ./configure
checking for apxs2... no
checking for apxs... /usr/sbin/apxs
checking Apache version... 2.2.10
checking for python... /usr/local/Python/bin/python
configure: creating ./config.status
config.status: creating Makefile

> make

...

> make install

/usr/sbin/apxs -i -S LIBEXECDIR=/usr/lib/httpd/modules -n 'mod_wsgi' mod_wsgi.la
/usr/lib/httpd/build/instdso.sh SH_LIBTOOL='/usr/lib/apr-1/build/libtool' mod_wsgi.la /usr/lib/httpd/modules
/usr/lib/apr-1/build/libtool --mode=install cp mod_wsgi.la /usr/lib/httpd/modules/
cp .libs/mod_wsgi.so /usr/lib/httpd/modules/mod_wsgi.so
cp .libs/mod_wsgi.lai /usr/lib/httpd/modules/mod_wsgi.la
cp .libs/mod_wsgi.a /usr/lib/httpd/modules/mod_wsgi.a
chmod 644 /usr/lib/httpd/modules/mod_wsgi.a
ranlib /usr/lib/httpd/modules/mod_wsgi.a
PATH="$PATH:/sbin" ldconfig -n /usr/lib/httpd/modules
----------------------------------------------------------------------
Libraries have been installed in:
   /usr/lib/httpd/modules

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,--rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
chmod 755 /usr/lib/httpd/modules/mod_wsgi.so

The reason I didnt bother with a --prefix option here is that I know I will only ever have one instance of httpd on my server, so I might as well install the modules in the "standard" place. I doubt I will be ever using multiple mod_wsgi modules (will I? I hope not!). This is one of those modules that I'd rather just let run, install, and know that my box has it.

2. Add mod_wsgi to apache

Once installed, we can change the "httpd.conf" file (the configuration file for apache) to add the new module. I normally stick it in the "LoadModule" section, so I dont get confused in the future. Keeping all the LoadModule bits in one area is a bit tidier.

> vi /etc/httpd/conf/httpd.conf

...

197 LoadModule mem_cache_module modules/mod_mem_cache.so
198 LoadModule cgi_module modules/mod_cgi.so
199
200 #
201 # The following modules are not loaded by default:
202 #
203 #LoadModule cern_meta_module modules/mod_cern_meta.so
204 #LoadModule asis_module modules/mod_asis.so
205 LoadModule wsgi_module modules/mod_wsgi.so
206
207 #
208 # Load config files from the config directory "/etc/httpd/conf.d".
209 #
210 Include conf.d/*.conf

Restart httpd.

> /etc/rc.d/init.d/httpd restart

If you get the following error:

Cannot load /etc/httpd/modules/mod_wsgi.so into server: \
/etc/httpd/modules/mod_wsgi.so: cannot restore segment prot after reloc: \
Permission denied

Then most likely you have SELinux enabled. SELinux is a security feature of Linux, and to be honest, everytime I've encountered it, I've just turned it off. It just stops everything and anything from working. A bit like Norton, or Windows Vista security alerts. Damn annoying. Turn it off.

> vi /etc/selinux/config

1 # This file controls the state of SELinux on the system.
2 # SELINUX= can take one of these three values:
3 #   enforcing - SELinux security policy is enforced.
4 #   permissive - SELinux prints warnings instead of enforcing.
5 #   disabled - SELinux is fully disabled.
6 SELINUX=disabled
7 # SELINUXTYPE= type of policy in use. Possible values are:
8 #   targeted - Only targeted network daemons are protected.
9 #   strict - Full SELinux protection.
10 SELINUXTYPE=targeted
11

You may also need to double check your iptables configuration. Again, I tend to turn off iptables as I have a firewall/router in front of my servers that handles firewalling.

> chkconfig --list iptables
iptables 0:off 1:off 2:on 3:on 4:on 5:on 6:off

> chkconfig iptables off

> chkconfig --list iptables
iptables 0:off 1:off 2:off 3:off 4:off 5:off 6:off

While we're here, might as well do the same to ip6tables too.

> chkconfig --list ip6tables
ip6tables 0:off 1:off 2:on 3:on 4:on 5:on 6:off

> chkconfig ip6tables off

> chkconfig --list ip6tables
ip6tables 0:off 1:off 2:off 3:off 4:off 5:off 6:off

You'll need to restart your system if you turn off SELinux, for that to take effect. Now you should be able to restart httpd without any dramas.

> /etc/rc.d/init.d/httpd restart

Stopping httpd: [FAILED]
Starting httpd: [ OK ]

With the module loaded, we need to hook up apache and your WSGI application so they can talk.

3. Configure your WSGI Application

Your WSGI application (in my case Pylons) runs fine by itself on its own specified port. But for it to run under mod_wsgi, it needs almost like a "command" for mod_wsgi to run. Assuming your Pylons installations is located at "/apps/myapp", lets create a new file here called "myapp.wsgi". This will be the entry point for mod_wsgi.

> vi /apps/myapp/myapp.wsgi

1 #!/usr/bin/env python
2
3 import site
4 site.addsitedir('/apps/myapp/lib/python2.6/site-packages')
5
6 import os, sys
7
8 sys.stdout = sys.stderr
9
10 sys.path.append('/apps/myapp/myapp')
11 os.environ['PYTHON_EGG_CACHE'] = '/apps/myapp/.python-eggs'
12
13 from paste.deploy import loadapp
14 from paste.exceptions.errormiddleware import ErrorMiddleware
15
16 application = loadapp('config:/apps/myapp/myapp/development.ini')
17 application = ErrorMiddleware(application, debug=True)
18

This seems to do the trick for me. I'm not exactly sure of the meanings of all this, but I'll try to explain.

a) The import site / addsitedir section emulates the virtual environment that we are using. Effectively, this loads our application specific site-packages. A bit like our "source activate" command if we were running Pylons as a standalone server.

b) stdout / stderr just dumps all errors into standard out. Hopefully we'll pick these up in /etc/httpd/logs/error_log (or equivalent).

c) sys.path.append is a bit like a "cd /apps/myapp/myapp". Everything will be running from this directory.

d) os.environ seems to require a python egg cache to extract things to. It appears to be a temporary directory. So we need to create this - I just created it with permissions 777 (ick) and left it empty. That seems to do the trick.

e) application object is then created.

One last thing I needed to do (which you should do here as well) is configure the debugging properties of your Pylons application. Pylons complains that because Apache runs using multi-processes, its ErrorMiddleware doesn't work properly. Personally, I'm not a big fan of the interactive debugger, and while it does show me a lot of useful things, you can more often than not access variable values anyway (if the error can be reproduced of course). Anyway, edit your "development.ini" file as follows:

> vi /apps/myapp/myapp/development.ini

...

17
18 [app:main]
19 use = egg:case
20 full_stack = false
21

The full_stack option means that the interactive debugger will show you the value of all variables at all points in the stacktrace of an error - is that really necessary? To me, no.

4. Configure Apache to locate your WSGI Application

Lastly, we need a final apache configuration to point to this WSGI application, in particular the "myapp.wsgi" file we created in the previous step. I put this into "/etc/httpd/conf.d/wsgi.conf".

> vi /etc/httpd/conf.d/wsgi.conf

1 WSGISocketPrefix /var/run/httpd
2
3 WSGIDaemonProcess myapp processes=2 threads=4 maximum-requests=5000
4 WSGIScriptAlias /myapp /myapp/myapp/myapp.wsgi
5 WSGIProcessGroup myapp
6 <Directory /apps/myapp>
7 Order allow,deny
8 Allow from all
9 </Directory>
10

The first line tells the WSGI module to place sockets into the /var/run/httpd directory. If you do not have this line, it seems to put the sockets into a directory where it cannot access, causing the following error in your /etc/httpd/logs/error_log:

(13)Permission denied: mod_wsgi (pid=26962): Unable to connect to WSGI \
daemon process '<process-name>' on '/etc/httpd/logs/wsgi.26957.0.1.sock' \
after multiple attempts.

The WSGIDaemonProcess sets a few configuration parameters for this application.

The WSGIScriptAlias tells apache that any file that is requested within the directory "/myapp" is to be served by the WSGI application, and it points to the entry point of that WSGI application - the "myapp.wsgi" file we created in the previous step.

The <Directory> directive allows apache to read from the directory that holds the WSGI application.

Restart apache, and all should be fine! Be sure to check your error logs (/etc/httpd/logs/error_log) in case something is amiss.

> /etc/rc.d/init.d/httpd restart

Stopping httpd: [ OK ]
Starting httpd: [ OK ]

And of course, browse to the relevant location to see if your Pylons application appears! Good luck!

How Can WebTop Help?

Learn how people like you are using WebTop to solve their database needs

Watch a video demonstation of CASE, our flagship web management system

Web News

Combine Ajax and JSON to Transmit Complex Presentation Data
Learn how to combine Ajax and JSON to override the browser's default locale and perform retrievals of complex data.
2010-09-01T07:04

Ten Tools Web Developers Can't Live Without
What are the 10 open source and online tools that all Web developers should have in their toolboxes? Here's one developer's list of must-haves.
2010-08-25T12:56:04-04:00

Implementing Flexible Website Navigation with WordPress 3.0
WordPress 3.0 allows you to build a flexible navigation system easily using custom menus and just a few lines of code.
2010-08-23T07:04

Setting Up the All-in-One SEO Pack for WordPress
Like every website, your WordPress blog has to be optimized for search engines. The All in One SEO Pack plugin is a one-stop shop for most of the SEO tweaks you need to make.
2010-08-18T15:34:34-04:00

Add Multilingual Support to Your Java Web Pages
Give your Web visitors the option of selecting their preferred languages using JavaScript, Ajax, Java Servlets and JSPs.
2010-08-16T15:19:46-04:00