Jacob Repp

I tried using web.py today for a small project at work. The basic idea of the project is that a simple python script converts a tree of log files into a database file. The same script can be run to view, search and slice any previously created database file. I’m planning on pulling out some of the small bits to show some example python code. For today I’ll focus on a problem I ran into with web.py and how I solved it.

Web.py on the surface is a very simple and slick python web framework. I’m using version .22 which is the latest stable release. The documentation is a bit hard to parse, it’s really just a dump of the comments from the code. Really you’re better off just going to read the code which itself is hackish.

In web.py examples you start so:

import web

class index:
    def GET(self):
        print 'hello!'

urls = ('/', index)
web.run(urls, globals())

This starts up the built-in web server on port 8080 and you can see ‘hello!’ in your browser right away. Pretty slick.

After working with it for awhile you’ll soon be sick of restarting the server and the documentation describes the auto-reload feature so:

web.run(urls, globals(), web.reloader)

All ‘middleware’ goes into the *rest parameter of the run() method. Using the web.reloader module doesn’t work right away like this though.

First on a design level I should point out a few problems. Instead of using web.reloader in an abstract way the web.run() method is specialized for this particular module and changes it’s internal behavior when you pass it in. Additionally, you cannot use the same model of calling web.run() with and with-out web.reloader. This seems like a long standing problem in the request.py file. I looked at a recent development snapshot and it’s still the same way.

Personally, I think that auto-reloading of web request modules should be a built-in feature anyways as it is a core element of a web programming framework even one as light weight as this. Perhaps other containers implement it for you automatically so this feature has not gotten the love it needs.

At first I didn’t understand how the internals worked correctly so to get around this I rolled a very simple auto-reload functionality into my web viewer module. The concept was so simple and frankly I was frustrated after looking at some web.py internals:

def _OnImport():
  """Execute on initial import of module"""
  import sys, os
  global g_mtime
  g_mtime = os.stat(__file__).st_mtime
_OnImport()

def _AutoReload():
  """Call to reload this module as needed"""
  import sys, os
  global g_mtime
  fn = __file__
  if __file__.endswith('.pyc'): 
    fn = __file__[:-1]
  mtime = os.stat(fn).st_mtime
  if mtime > g_mtime:
    reload(sys.modules[__name__])
    g_mtime = mtime
    return True
  return False

class Request(object):
  def __init__(self):
    self.reloaded = _AutoReload()

class Main(Request):
  def GET(self):
    print 'hello! (reloaded: %s)' % self.reloaded

def Start():
  import web
  urls = ('/', 'Main')
  web.run(urls, globals())

The style is a bit non-python idiomatic, it’s roughly our coding style at work.

Then I ran into a problem: using my method there was no way to add new url mappings to web.py while it was running. Then I finally realized what the correct (undocumented) way of using web.reloader is:

urls = ('/', 'Main')
def Start():
  import web
  web.run('urls', globals())

The key is moving urls into a global of the module and specifying the name of the global variable instead of it’s value. All this so that the following special-case code in request.py works as intended:

def webpyfunc(inp, fvars, autoreload=False):
  #...
  mod = __import__(modname(), None, None, [""])
  #@@probably should replace this with some inspect magic
  name = utils.dictfind(fvars, inp)
  func = lambda: handle(getattr(mod, name), mod)
  #...

The utils.dictfind() method returns the tuple mapping from the module dictionary and passes it to handle() in the lambda. Every time the lambda is invoked it does the lookup of the tuple so that it can change on the fly.

In the end this works much better than my home grown autoloader but the usage of it took me way to long to get right.

Related posts:

• Globals Are Still Considered Harmful to Your Health

Trackback URI | Comments RSS