Python web app as an egg with Paste, Fabric and Spawning

One of the major success of PHP is how easily you can deploy PHP applications. Thanks mod_php for that. Python or Ruby requires quite more work even though it’s not that painful if your using something like Passenger with Ruby. I’m still looking for a way that I find brilliant. So I played with Paste, Spawning and setuptools to build a very minimalist WSGI application you deploy without any pain using Fabric.

Before we start, you can find the code bellow on GitHub.

First the setup.py file that describe our application:

from setuptools import setup, find_packages

setup(name="app",
      version="0.1",
      description="example application",
      long_description="",
      keywords="",
      author="Yoan Blanc",
      author_email="yoan.at.dosimple.ch",
      url="http://yoan.dosimple.ch/blog/",
      license=’All rights reserved’,
      packages=find_packages(),
      zip_safe=False,
      install_requires=[],
      entry_points={"paste.app_factory": ["main=app:make_app"]},
     )

It’s all standard except the entry point called paste.app_factory that will be used by Paste to create the WSGI application. For the Djangoist here, Paster acts like your manage.py.

Make the WSGI application itself, into app/__init__.py:

def application(environ, start_response):
 start_response("200 OK",
                [("Content-Type", "text/html")])
 return ["<h1>Hello World!</h1>"]

And the factory (called make_app inside setup.py) is required as well in the same file.

def make_app(global_conf, **app_conf):
    return application

Now, you might be wondering how this make_app is called. Paste works with ini files that decribe your configuration, from the app your using to the database, the server, logs, and so on. It’s very handy to replicate the same kind of environment you find when working with Ruby on Rails by having different ini files for each.

So there is the app.ini:

[app:main]
use = egg:app

[server:main]
use = egg:Spawning
host = 127.0.0.1
port = 8000
num_processes = 1
threadpool_workers = 1

The first section describes the application to use. It’s the egg that setuptools will create and install. That’s right, your application is exactily like any other python package. It means dependencies resolution, updates,… And the second part describe which server paster will have to use to run it. Spawning is a very efficient server that handles graceful code reloading. Pure happiness.

Before you can run it using :

$ paster serve app.ini

You’ll have to install it.

# python setup.py develop

develop means it symlinks the package to your directory and you won’t have to reinstall it later to take into account the changes made.

It should run just perfectly. Now let’s worry about deploying it somewhere else. Local web applications are nice but sharing them is cool too.

This is the fabfile.py, Fabric is more or less like Capistrano (Ruby):

from __future__ import with_statement
from fabric.api import env, run, put, local, cd

env.hosts = ["localhost"]
env.path = "~/www/app"

def build_egg():
    local("python setup.py bdist_egg")

def cleanup():
    local("rm -rf build")
    local("rm -rf dist")

def start():
    with cd(env.path):
        run("PYTHONPATH=app.egg paster serve app.ini"
            "--daemon --pid-file=app.pid")

def stop():
    with cd(env.path):
        run("paster serve --stop-daemon "
            "--pid-file=app.pid")

def deploy():
    build_egg()
    put("app.ini", env.path)
    put("dist/*.egg", env.path+"/app.egg")
    cleanup()

A pythonic Makefile, more or less that has special power to deal with deployment. This how you can use it.

$ fab deploy
$ fab start

It created a .egg, uploaded it to the destination and started the Spawning server using Paster. It will run on the port 8080, to see it on the port 80 I’d use something like Nginx as a proxy. But it’s a different story (and very simple by the way).

After you made some changes simple run:

$ fab deploy

to see them live.

$ fab stop

Will stop the server.

How awesome is that?

There is a lot more that can be done regarding how you’re playing with the eggs. Using virtualenv to keep multiple, sane environments and maybe your own cheeseshop where all your eggs can live (see Ask’s Chishop)

I’m very fresh on that field but am very happy seeing that such good tools exist. Please share your experience, I’m looking forward knowing what kind of process is used in the real word out there to.

Yoan Blanc’s weblog