Google App Engine with Python and Pylons

Posted Tuesday, 21 April 2009 at 23:25 by Andrew Liu
Tagged: web development
Read more blogs...

 

I had tried a few goes at Google App Engine (GAE) a few times in the past, but today tried it with a bit more vigour.  Surprisingly, it was worked a treat, so much as I got a simple application going.  I expected the installation to fall over, and a few times it did, but that was because I was trying things that were different to the norm as I went along.  For this exercise, I wanted to install Pylons as the app server on GAE, and the best source to do that is, of course, Ian Bicking, as he explains well in this post.

 

The first headache I encountered was that I had Python 2.6.1 installed.  The appengine-monkey application only works on Python 2.5, so I had to install that on my server.  See my tutorial on how I install Python, and you will get a good idea on how to install dual python installations.

 

So here's my take on installing GAE on Fedora 10.

 

1. Get appengine-monkey

 

I wanted to create a new application called 'webtop', so I installed into a directory called '/apps/webtop'.  Of course, all this was done as the 'root' user.

 

$ cd /apps/webtop
$ svn checkout http://appengine-monkey.googlecode.com/svn/trunk/ appengine-monkey

 

 

2. Install Python 2.5

 

The standard installation, with a defined prefix, does well here.

 

$ cd /root

$ wget http://www.python.org/ftp/python/2.5.4/Python-2.5.4.tar.bz2

$ tar xvfj Python-2.5.4.tar.bz2

$ mv Python-2.5.4 /usr/local/src

$ cd /usr/local/src/Python-2.5.4

$ ./configure --prefix=/usr/local/Python-2.5.4

$ make

$ make install

$ cd /usr/local/bin

$ ln -s /usr/local/Python-2.5.4/bin/python2.5 python2.5

 

 

3. Get the Google Appengine

 

Download the appengine SDK source code.

 

$ cd /root
$ wget http://googleappengine.googlecode.com/files/google_appengine_1.2.0.zip
$ unzip google_appengine_1.2.0.zip
$ mv google_appengine /apps/webtop

 

I think my directory structure ended up different to Ian Bicking's directory structure, but it still worked for me.

 

 

4. Create the environment for GAE

 

Now we see why we grabbed appengine-monkey.  This makes installation of GAE for Pylons fairly straight forward.  The following creates a virtualenv environment for our new application.

 

cd /apps/webtop
python2.5 appengine-monkey/appengine-homedir.py --gae /apps/webtop/google_appengine webtop

 

Our directory structure is now as follows:

  • apps/
    • webtop/ (holds the source directories)
      • appengine-monkey/ (appengine-monkey source)
      • webtop/ (your new application source)
      • google_appengine/ (GAE source)

Immediately after this, I activated the virtualenv.  This is a must, otherwise you never know what (and where) you are installing features.  This caught me out a few times, and I had to clean up my base installations after tainting them with unwanted third party modules.

 

$ cd /apps/webtop/webtop
$ ./bin/activate

(webtop)$

 

 

5. Install Pylons

 

Instead of using setuptools, a new installation tool is provided that works with GAE.  Pip effectively replaces easy_install here.  The homepage for Pip is located at http://pip.openplans.org/.

 

(webtop)$ cd /apps/webtop/webtop
(webtop)$ ./bin/pip install Pylons

 

 

6. Create the Pylons application

 

Now we physically create our pylons application.  Interesting to note that pip is located in '/apps/webtop/webtop/bin', but paster is located in '/apps/webtop/webtop/app/bin'.

 

(webtop)$ cd /apps/webtop/webtop/app
(webtop)$ ./bin/paster create --template=pylons webtop sqlalchemy=false

 

The next step required moving directories around, and I started to get a little confused with the number of 'webtop' directories that I was using.  So I'll try to be verbose.  Now, we have the following directory structure, and highlight the important files.

  • apps/
    • webtop/ (holds the source directories)
      • appengine-monkey/ (appengine-monkey source)
      • webtop/ (your new application source)
        • app/ (where our GAE application will reside)
          • bin/
            • paster (used to install Pylons)
          • webtop/ (** NOTE 1 ** Lets call this directory 'webtopOne')
            • webtop/ (** NOTE 2 ** Lets call this directory 'webtopTwo')
        • bin/
          • activate (script to activate the virtualenv)
          • pip (location of the pip installer)
          • python2.5 (our local python binary)
      • google_appengine/ (GAE source)

Now, you can see why 'webtop' is getting a bit confusing!  Maybe its just me wanting to use the same name for the application.  Anyway, the red directories above need a bit of TLC.  Basically, we need to blow away webtopOne (except the child directory webtopTwo), and replace it with webtopTwo.  If this is confusing, or if lines of code make more sense, try this out:

 

(webtop)$ cd /apps/webtop/webtop/app
(webtop)$ mv webtop/webtop apptemp
(webtop)$ rm -rf webtop
(webtop)$ mv apptemp webtop

 

Now our directory structure is as follows:

  • apps/
    • webtop/ (holds the source directories)
      • appengine-monkey/ (appengine-monkey source)
      • webtop/ (your new application source)
        • app/ (where our GAE application will reside)
          • bin/
            • paster (used to install Pylons)
          • webtop/ (** Formerly 'webtopTwo' **)
        • bin/
          • activate (script to activate the virtualenv)
          • pip (location of the pip installer)
          • python2.5 (our local python binary)
      • google_appengine/ (GAE source)

 

7. Edit environment.py

 

Edit the file in '/apps/webtop/webtop/app/webtop/config/environment.py, and remove the following line

 

    module_directory=os.path.join(app_conf['cache_dir'], 'templates'),

 

8. Edit config.py

 

Edit the file in '/apps/webtop/webtop/app/config.py, and replace the contents with this.  The first line, you can replace the word 'webtop' with the name of your application that you have used.

 

APP_NAME = 'webtop.config.middleware:make_app'
APP_ARGS = ({},)
APP_KWARGS = dict()
APP_KWARGS.update({'beaker.session.type': 'google', 'beaker.session.table_name': 'beaker_session',
                   'beaker.session.key': 'pylonstestapp', 'beaker.session.secret': 'secret',
                   'beaker.cache.type': 'google', 'beaker.cache.table_name': 'beaker_cache'})
# You can overwrite these separately for different dev/live settings:
DEV_APP_ARGS = APP_ARGS
DEV_APP_KWARGS = APP_KWARGS
REMOVE_SYSTEM_LIBRARIES = ['webob']

 

9. Run the application

 

Now we can run the application.  By default, it will run on localhost on port 8080.

 

(webtop)$ deactivate

$ cd /apps/webtop/google-appengine

$ python2.5 dev_appserver ../webtop/app

 

To upload the application to google:

 

$ python2.5 appcfg.py update ../webtop/app

 

nb. Quirks

 

a) Python 2.5 installation didnt work with --enable-shared.  I kept getting the following error:

 

$ bin/paster create --template=pylons webtop sqlalchemy=false
Traceback (most recent call last):
  File "bin/paster", line 8, in <module>
    load_entry_point('PasteScript==1.7.3', 'console_scripts', 'paster')()
  File "/apps/webtop/webtop/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg/pkg_resources.py", line 277, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/apps/webtop/webtop/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg/pkg_resources.py", line 2180, in load_entry_point
    return ep.load()
  File "/apps/webtop/webtop/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg/pkg_resources.py", line 1913, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/apps/webtop/webtop/app/lib/python/paste/script/command.py", line 24, in <module>
    from paste.script.util.logging_config import fileConfig
  File "/apps/webtop/webtop/app/lib/python/paste/script/util/logging_config.py", line 30, in <module>
    import sys, logging, logging.handlers, string, socket, struct, os, traceback, types
  File "/usr/local/Python-2.5.4/lib/python2.5/logging/handlers.py", line 30, in <module>
    import sys, logging, socket, types, os, string, cPickle, struct, time, glob
ImportError: /apps/webtop/webtop/lib/python2.5/lib-dynload/cPickle.so: undefined symbol: PyUnicodeUCS2_DecodeUTF8

 

After much frigging around, I ended up blowing the Python 2.5 installation away, and reinstalling it WITHOUT the --enable-shared option.  This fixed it up.  The fact that the cPickle library was located in lib-dynload made me suspect the shared libraries were at fault.  Of course, this seems strange, because I had Python 2.6 installed and running with --enable-shared, and importing cPickle worked fine there.

 

 

Web News

Symantec distrust to begin in Chrome from April 2018
Google's browser will start the process of removing trust from old Symantec TLS certificates in Chrome 66.
--- 28 Jul 2017 01:42 PM

Ok Google: Telstra adds Google Home to smart home offering
The telco giant has welcomed Google into its Smart Home play.
--- 28 Jul 2017 01:08 PM

Huawei reports 15 percent revenue surge for H1 2017
The networking giant generated 283.1 billion yuan in revenue during the first six months of 2017.
--- 28 Jul 2017 12:47 PM

LG posts healthy Q2 profit despite mobile setback
Marketing costs for the new G6 tinted an otherwise strong quarter for LG Electronics.
--- 28 Jul 2017 11:53 AM

Singapore should not criminalise good intent to encourage data sharing in cybersecurity
People who fear prosecution may be less willing to share threat information, which is vital in fending off attacks, and Singapore government should clearly define its proposed mandate for companies to report data breaches within 72 hours.
--- 28 Jul 2017 11:23 AM

Company Blog


Search Behaviour

Posted Tuesday, 19 October 2010 at 05:58 by Andrew Liu


As an SEO provider, you have one main goal. Get your clients website to show up in search results fo...

Read more...



sm bus drivers missing in Device Manager

Posted Sunday, 18 April 2010
Updated Sunday, 24 February 2013 at 06:39 by Andrew Liu


When installing a new Windows XPinstallation, I seemingly always miss some drivers. One that trouble...

Read more...



Multiple Domains for SEO performance?

Posted Friday, 05 March 2010 at 23:13 by Andrew Liu


Online businesses and websites that cover a broad range of topics or one large topic are sometimes b...

Read more...



Tag Clouds - SEO or not?

Posted Thursday, 04 March 2010 at 04:34 by Andrew Liu


A tag cloud or word cloud is a visual depiction of tags or words related to a site, typically used t...

Read more...



Mozilla Thunderbird and Gmail IMAP Attachments Bug

Posted Wednesday, 03 March 2010 at 20:15 by Andrew Liu


I've been using Gmail since its early inception, and Iwas one of the first to utilise Gmail's IMAPfe...

Read more...



Read more blogs...