Thursday, July 23, 2009

DRY Logging in Django

I have found custom logging a very useful tool for dealing with production issues. Especially when the users are not easily reachable. I recently configured custom logging messages for a Django project I have been working on. As logging is usually only initiated once per application I chose to do it in my settings.py. The code looked something like this:

This configuration worked out well in development. However I maintain a separate settings file for production, production.py. Production.py merely imports everything from settings.py and selectively overrides some of the attributes (set DEBUG = False, for instance).

My problem was that I wanted the production logs to be written to a different location and perhaps even use a different handler. I could not think of a good way to do this without calling the logging code fragment again in production.py. I wrote a utility function to do this (configure_logging(filename)). This reduced code repetition but revealed another problem. When production.py imports everything from settings.py it triggers logging to be configured once using the LOG_FILENAME attribute used in development and then *again* using the LOG_FILENAME attribute used in production.

This was ugly. For one there is no need to configure logging twice; for another the initial configuration throws an error if the path described by LOG_FILENAME in development is not present in the production machine.

I wasn't sure how to proceed and I posed this question at Stack Overflow. I got an answer suggesting to switch LOG_FILENAME based on the DEBUG attribute. Something like this:

This looked good, but almost immediately revealed a problem. DEBUG is overridden *inside* production.py whereas this code snippet is inside settings.py. Consequently when the above given snippet gets executed DEBUG is still True and LOG_FILENAME will be pointed at the development environment.

I eventually found a solution from an answer to a different question. User Bluebird75 suggested using the 'module singleton' pattern to ensure that logging is only configured once. I extended his(?) suggestion to come up with the following solution:

The singleton module ensures that logging is only configured once. As this module resides within the app directory Django would already have loaded settings before this module is loaded, thereby ensuring the presence of settings.LOG_FILENAME. I can add as many settings files as I want and all I have to do is override the LOG_FILENAME attribute in each file.

6 comments:

Prabs said...

To have custom settings for a specific machine, do something like

try:
from localsettings import *
except:
pass

at the end of settings.py

Now in your devel machine, create a localsettings.py with whatever config you want.

Manoj Govindan said...

@Prabs:

Doesn't that mean having multiple files named 'localsettings.py', one for each unique environment (development/staging/production)?

Anonymous said...

Use socket.gethostname() and branch on the hostname.

Manoj Govindan said...

@Anonymous:
I want my code to be host-agnostic. If I branch on hostname I have to add branches for all possible machines I might choose to develop in (home laptop/home desktop/work laptop etc.)

John said...

the localsettings solution is the only one that isn't a hack... your production.py solution is definitely a hack

Manoj Govindan said...

@John: I am not sure what you are referring to when you say "localsettings solution". Care to explain?