Thursday, November 15, 2007

Quick and dirty 'type' for Django's message model

In an earlier post I had mentioned using a 'flash' variable passed to templates as a mechanism for passing messages to users. Following an anonymous tip I started using user.message_set instead to pass messages. This mechanism makes use of Django's message model and is easy to use. However it does have a limitation. Messages cannot be assigned types - say, 'Error' or 'Success' or 'Warning'. This has been discussed at some length at the Django site. A design decision is awaited as the suggested changes will not be backward compatible.

In the meantime I needed message types for a pet application and came up with a quick and dirty way of providing some. Changes were required in three places: the view, where messages are created; the template which displays the messages and the style sheet used for styling the rendered templates.

The solution (did I say it was dirty?) involved prefixing a digit and a separator to all messages. The digit would indicate the type of message. I chose 0 to indicate success, 1 for warnings, 2 for errors etc. Corresponding styles were then created to appropriately style the messages.

Here is a sample view: The template looks something like this: There is a corresponding style sheet as well:

Tuesday, October 16, 2007

An Unique Arrangement

I visited Leh, Laddakh in mid 2001 for trekking and seeing the place. Four of us completed our trek along Laddakh's Markha Valley in about eight days and decided to travel south to Srinagar in the Kashmir valley. Although Laddakh and Srinagar are both part of the state of Jammu and Kashmir they have dissimilar cultures and political atmosphere. The predominantly Buddhist Leh has been spared most of the violence that has plagued the state while the predominantly Muslim Kashmir valley has been at the heart of incidents.

We hired a taxi to ferry us to from Leh to Srinagar. The road connecting the two passes through several towns including Kargil that bore witness to a small scale war as recently as 1999. We started at noon from Leh. In spite of the modest distance (~475 Km/300 miles) ahead of us we were obliged to stop in Kargil at nightfall for various reasons. Traffic through the narrow Zoji-La pass was regulated so that vehicles driving North-South could only travel through from sunrise to noon. For the rest of the day the pass would be used by vehicles traveling in the opposite direction. Moreover nighttime civilian travel was not common, even discouraged.

In Kargil we checked into a hotel for the night. During dinner the manager came over and enquired if we had hired the cab in Leh. After we answered he told us to expect the driver to come up with an excuse to end the journey in Kargil and go no further. We would be offered an identical vehicle with a new driver to take us to our destination at no extra cost. It happened exactly as he predicted. Early next day our driver informed us that the car wouldn't start but we could continue our journey in another vehicle that had miraculously appeared out of nowhere in the predawn cold.

The incident was puzzling until we discovered the reason behind it. Our first cab driver was a Buddhist from Laddakh. He didn't dare to venture into Kashmir valley for fear of his life. His replacement was a Muslim from Srinagar who stayed away from Leh. The desire to survive had prompted them to come to an arrangement through which they could still work and split fares.

The rest of our journey was uneventful save for the hair raising transit through the narrow paths of Zoji-La. We reached Srinagar safely and settled into a house boat in Nagin Lake.

I haven't been to Kashmir since. I wonder if things have changed sufficiently for the cabbies to drop their arrangement.

Bank Notes

I recently had to deposit money into a friend's account. He has an account with State Bank of India, Trivandrum branch. For those of you not from around here, that branch is in a city in a different state. All I had to do was:
  1. Locate the nearest SBI branch
  2. Make the deposit
I tried to locate the nearest branch using SBI's Branch Locator. After a few minutes of fiddling with the unwieldy search interface I located a branch using - what else - Google. According to the page the branch had the following "alternate channels" (facilities?) : Internet banking, Core banking, ATM(s). Great, that is all I needed.

Armed with this information I visited the bank and presented my request. I was promptly informed that this particular branch would only let me make deposits if the target account belonged to this branch. So much for "core banking".

All was not lost yet. I recalled another branch located in St. Marks road and asked if visiting that one would help. Unfortunately, no. That particular branch was for "specialized personal banking" whereas I needed a "general branch". The nearest one was a good 10 Km away. I later found that "general branch" is a vaguely defined concept; the "advanced options" section of SBI's Branch Locator does not include it among the more than a dozen 'branch type' options listed.

With that I gave up on SBI. Fortunately my friend had an account with Indian Bank. IB was a little more forthcoming with information about their branches. I located the nearest one and went over. I found the printed pay-in slip a bit confusing in spite of instructions written in three languages. Luckily one of the employees hinted that what mattered was the inclusion of relevant information in the form; the "where" did not matter. So I filled up the form as best as I could and paid the money.

This episode left me with some questions.
  • What is "core banking"? Does it exclude paying in money to accounts established in other states/cities/branches, especially in this day and age when all banks are touting their internet facilities?
  • Why does a national bank with vast outreach make depositing money to a non-local account so difficult? Particularly in cities like Bangalore where a significant percentage of inhabitants are from other states.
  • Who designs the pay-in slips used in banks? I feel that at least some of them need to be redesigned to make them easier to use. Something akin to usability in software applied to banking.

Saturday, June 02, 2007

Django Testing Gotchas

I recently started using Django's newly introduced capability to make use of test fixtures. In order to do this all test classes that need fixtures have to be subclasses of Django's custom TestCase class rather than the customary unittest.TestCase. I had written about 17 or so test cases split across two test classes. One of these test classes used Django's fixtures.

Each Django test run (kicked off by running python manage.py test) involves the following steps:
  1. Create test database
  2. Create the necessary tables
  3. Load "initial fixtures" if any. These have to be located in a file named initial_data with suitable extension denoting fixture type (.xml, .json etc.)
  4. Load fixtures specified in each test case
  5. Execute any one test
  6. Remove fixtures loaded in step #4
  7. Repeat steps 4 through 6 until all tests have been executed.
Sounds nifty, doesn't it? That is what I thought too, until I ran into reality. All goes well until step 3. After that, the tests take an eternity to execute. OK, it took 159 seconds to execute 17 puny tests. I was surprised. My application was rather simple, involving less than a dozen tables and a few hundreds of rows. And I was only using a subset of the data to test. What was going on? I looked around and found that I was not the only one to face this problem.

After reading up more about the problem I figured that I could probably cut down on the time taken by "refactoring" my tests and moving unnecessary fixtures out to "initial fixtures" so that they would only be loaded once. Luckily this would work in my case as my app only used the data for reference; it did not modify any. I was therefore not constrained to load the data before each test.

I made the necessary changes and re-ran my test cases. The time taken to execute the test cases improved ever so slightly. It now took about ~140 seconds.

I was still unhappy with the results. Since I was not using any fixtures, I decided to substitute Django's TestCase with unittest.TestCase. And voilĂ ! It was as I had engaged hyper drive. 24 tests now took a mere 0.2 seconds to run!

Naturally the question that came to my mind was that in the absence of fixtures, what on earth was taking up so much time? Django is known for its speed vis a vis Rails; I had come to expect the same kind of superior performance while running tests too. I don't have an answer right now. I hope to come up with one after I take a look at Django's innards.

Wednesday, February 14, 2007

Moving from Rails to Django: Replacing flash[:notice]

One of the most common practices I came across while learning Rails is the flash[:notice] mechanism used to send (error) messages to the template. Consequently it was also one of my first targets to migrate when I started moving a pet application from Rails to Django.

I came up with a simple (and all too obvious) way to send error messages to the template. I am sure that there are better ways to do this. If you know of any, please take a minute to share them.

First, the Rails snippet. Next, the equivalent in Django. The template in Rails ... ... and the template in Django. While testing in Django, flash can be treated just like any other context variable.

Request to Blogger

Please, please add a formatting tool/plug in/some other mechanism to highlight code syntax. It took me two frustrating hours of cutting and pasting to add code samples to this post. Even now, I am not completely happy with the results. And as automatically inserted line breaks (BR tags) and highlighter do not go together, I have to manually insert BR tags in all my posts :( Thankfully, I am not a prolific blogger ;) but even then it sucks.

I am not partial to any specific utility. In this case I followed the example set by other bloggers and used dp.SyntaxHighlighter. But I will (and so will most others, I suspect) gladly use your tool if you were to provide one.

C'mon guys, after all you are Google :)

Tuesday, February 13, 2007

Moving from Rails to Django: Migrating Tests

Context
I am in the process of migrating one of my small "pet" applications developed in Ruby on Rails to Python using Django. One of my first concerns was migrating tests, both unit (testing models) and functional (testing 'controllers' in Rails/'views' in Django) from one framework to the other. After looking around the documentation, asking questions and conducting some experiments I have some answers. Here is a listing of what I have found out so far.

Note
These examples will only work with the latest development version of Django.

Locating your tests
Rails
Usually located under the path [path_to_app]/test/functional/. There is usually one test source per controller, named as [controller_source_without_extension]_test.rb. For example if the source for a controller, say, 'Generator' is written in [path_to_app]/app/controllers/generator_controller.rb, then the corresponding test should be present in a file named generator_controller_test.rb

Django
The easiest way (see the official documentation) is to add all your tests to a file called tests.py and drop it in the application folder, at the same level as your models.py and views.py.

However I prefer to have a separate test source structure if I can help it. This has the added advantage of keeping I decided to adopt the convention used by Rails and came up with the following tree:
  • [project]/[application]/test/functional/ for functional tests and
  • [project]/[application]/test/unit/ for model tests.
Don't forget to add the __init__.py files to the tree as applicable to make these accessible as python modules. You will still need to retain the tests.py except that you will have to add a line to import your test case classes and nothing else. Test runner will automatically pick up the test classes and run them.
Fixtures
Fixtures are readily available in Rails. There are not quite there yet in Django and I am as righteously angry as the next ex-Rails user ;) But do not worry, they are on their way and will be available shortly. I am experimenting with a really dirty solution in the meantime. More on this in a subsequent post.

Writing Tests - setup
Rails
A typical setup() method in Rails looks like this: Django
The framework provides a test client to simulate browser calls. The best place to create a test client object would be the setup method.
Writing Tests - GET requests
Rails
My first objective was to test if a GET request to the controller/URL 'generator/index' was being handled properly. "Properly" in this case means three things:
  • Ensure that the response came back with an HTTP status of 200 OK;
  • Ensure that the appropriate template was used; and
  • Ensure that instance variables were present and had expected values.
The resulting test looked like this:
Django
While the objectives remain the same, the steps vary. The key differences I noticed were:
  • There are no built-in wrapper methods for GET/POST. It should be noted that it is easy to enough to code wrappers should one so wish.
  • Likewise there are no equivalents for wrappers such as "assert_template" or "assert_response"
  • Django uses a different philosophy when it comes to passing variables to templates. Instance variables are not directly passed to templates; rather they have to be explicitly passed using a context object or a call to locals()
The resultant code looks like this:
Writing Tests - POST requests
Rails
A simple "happy path" POST request test looks like this in Rails: Here, in addition to checking the response I am also verifying that a particular session variable was set properly.

Django
Here again we find that there are no wrappers for session and that it is easy enough to add some. The session object itself is readily available and can be accessed using the test client.
Running Tests
Rails
The easiest way is to execute ./rake from the command line. There are separate tasks available (such as test:functional, for instance) in case you wish to run the functional tests only.

Django
The easiest way is to execute ./manage.py test from the command line. This will run the tests for all applications in the project. If you wish to test only one application, feed its name as an additional parameter.

Django Template Nuances
Django provides simple inheritance mechanism for templates. I personally like it :) What should be remembered is that using inheritance changes the testing process slightly. For instance, consider a view index that renders a template say, index.html. This template inherits from another, named base.html. A GET request test for the view would look like this: Note the difference from the earlier test. When template inheritance comes into play, response.template will resolve into a list of all the templates involved in the hierarchy, listed parent first. Corresponding to each template, there will be a separate context object at available at the same index in response.context (which will also resolve into a list)

Testing Code Highlighting