DRY Logging in Django

I have found custom logging a very useful tool for dealing with production issues. Especially when the users are not easily reachable. I recently configured custom logging messages for a Django project I have been working on. As logging is usually only initiated once per application I chose to do it in my settings.py. The code looked something like this:
import logging import logging.handlers logger = logging.getLogger('project_logger') logger.setLevel(logging.INFO) LOG_FILENAME = '/path/to/log/file/in/development' handler = logging.handlers.TimedRotatingFileHandler(LOG_FILENAME, when = 'midnight') formatter = logging.Formatter(LOG_MSG_FORMAT) handler.setFormatter(formatter) logger.addHandler(handler)
This configuration worked out well in development. However I maintain a separate settings file for production, production.py. Production.py merely imports everything from settings.py and selectively overrides some of the attributes (set DEBUG = False, for instance).

My problem was that I wanted the production logs to be written to a different location and perhaps even use a different handler. I could not think of a good way to do this without calling the logging code fragment again in production.py. I wrote a utility function to do this (configure_logging(filename)). This reduced code repetition but revealed another problem. When production.py imports everything from settings.py it triggers logging to be configured once using the LOG_FILENAME attribute used in development and then *again* using the LOG_FILENAME attribute used in production.

This was ugly. For one there is no need to configure logging twice; for another the initial configuration throws an error if the path described by LOG_FILENAME in development is not present in the production machine.

I wasn't sure how to proceed and I posed this question at Stack Overflow. I got an answer suggesting to switch LOG_FILENAME based on the DEBUG attribute. Something like this:
# Inside settings.py if DEBUG: LOG_FILENAME = '/path/to/log/file/in/development' else: LOG_FILENAME = '/path/to/log/file/in/production' # Configure logging here.
This looked good, but almost immediately revealed a problem. DEBUG is overridden *inside* production.py whereas this code snippet is inside settings.py. Consequently when the above given snippet gets executed DEBUG is still True and LOG_FILENAME will be pointed at the development environment.

I eventually found a solution from an answer to a different question. User Bluebird75 suggested using the 'module singleton' pattern to ensure that logging is only configured once. I extended his(?) suggestion to come up with the following solution:
# Inside settings.py LOG_FILENAME = '/path/to/log/file/in/development' # Inside production.py from settings import * LOG_FILENAME = '/path/to/log/file/in/production' # Singleton module log.py import logging, logging.handlers from django.conf import settings LOGGING_INITIATED = False def init_logging(): logger = logging.getLogger('project_logger') logger.setLevel(logging.INFO) handler = logging.handlers.TimedRotatingFileHandler(settings.LOG_FILENAME, when = 'midnight') formatter = logging.Formatter(LOG_MSG_FORMAT) handler.setFormatter(formatter) logger.addHandler(handler) if not LOGGING_INITIATED: LOGGING_INITIATED = True init_logging()
The singleton module ensures that logging is only configured once. As this module resides within the app directory Django would already have loaded settings before this module is loaded, thereby ensuring the presence of settings.LOG_FILENAME. I can add as many settings files as I want and all I have to do is override the LOG_FILENAME attribute in each file.

Ticket #11475

Logged a Django bug related to test client. Part of my continuing series of adventures in Django.

Learning to write smaller sentences

I have started to use Twitter. There are many things I'd like to write about that do not merit a blog post. Twitter is the right place to scribble in such cases. I hope to be a more regular writer using the new medium :)