Archive for the ‘Programming’ Category

Source Control Commit Visualisation

Wednesday, June 25th, 2008

Software development relies on source control management software such as CVS, Subversion, SourceSafe, Bitkeeper, Mercurial, Git and the like to track and manage the changes in the source code over time. As a project progresses, developers come and go, contractors come and go and the activity on a given project ebs and flows as required.

Attempting to visualise who, what and how much of a project is changing is quite complex as there are so many variables - however Michael Ogawa has built a project named code_swarm which does just that. Instead of providing tabular or static images to help visualise a projects changes, he has managed to animate it into something quite spectacular.

Following are five different code swarm visualisations of popular open source projects:

The amazing thing that a visualisation such as code_swarm provides, is to show just how many people actively participate in a given open source project, how much each of them participates and what sort of tasks they are normally performing on that project. As an example, comparing the number of different people in SQLAlchemy compared to Django isn’t a competition - Django is ahead by a mile, though compared to Apache, the others seem insignificant.

.htaccess Inheritance Gotcha

Wednesday, April 30th, 2008

I recently set up a subdomain within a CPanel hosting account and ran into a strange problem.

After the subdomain was built, DNS propogated, FTP access and a database set up - I uploaded the latest version of the ever propular WordPress blogging software. As the installation instructions suggested, about two minutes later WordPress was up and running. Unfortunately the image overlays for the TinyMCE rich text editor were not displaying correctly; the buttons were present but the overlay that depicts a capital B for bold or an I for italics were not showing.

Knocking off the simplest things first:

  • F5
  • CTRL+R and CTRL+F5 to force the browser to reload all content
  • Cleared browser cache
  • Uploaded the wp-includes folder again, in case any of the images failed to upload successfully
  • Checked that I could view the images in question manually in the browser by typing in the URL, which worked

at this point I was beginning to draw a bit of a blank as to what it might have been.

As is often the case, I left the problem alone for a little while and the solution popped into my head. To enable the friendly URL’s within WordPress, I needed to either make the .htaccess file writable or upload one with the appropriate configuration in it. As a matter of simplicity, I copied the .htaccess file from the main WordPress installation on the site and dropped it into the subdomain installation. Copy and paste, the bain of all evil.

Back in May 2006, people from MySpace were hotlinking images from my site. The cost of popularity from my anonymous MySpace friends was pushing my web hosting account well over its monthly data limits and I was forced to block their access using some simple rules in my .htaccess file.

Since I copy and pasted my .htaccess file into the subdomain (which resides in a folder under the primary account) - the settings in the parent .htaccess file were inheriting into the subdomain account. The result was that any requests from the subdomain that didn’t meet the hotlinking requirements were being blocked by Apache and mod_rewrite.

The solution of course was straight foward, remove the restrictions in the .htaccess file that I had uploaded into the subdomain and suddenly the images from the WordPress admin and TinyMCE started showing up as you’d expect.

A work collegue of mine would term this a junior error.

ASP.NET Permanent (301) Redirect

Thursday, April 3rd, 2008

Within ASP.NET, the System.Web.HttpResponse class providers a method to handle HTTP redirects, aptly named Redirect. The Redirect method has two overloads:

  • Redirect (string)
  • Redirect (string, bool)

The former accepts a string representation of the URI that you’d like to redirect to. The latter accepts the string URI and a boolean indicating whether you want ASP.NET to stop processing the request immediately or to continue throughout the life cycle of the request.

Unless you’re a HTTP junkie or you’ve had a specific reason to issue a permanent redirect (reorganising a web site is a prime candidate), you can be forgiven for not realising that the Redirect method issues a temporary redirect (HTTP 302). Not having needed to issue a permanent redirect through ASP.NET before, I assumed (there is that word again) that there would be additional overloads to handle both temporary (HTTP 302) and permanent (HTTP 301) redirects or optionally another method; I was wrong.

To issue a HTTP 301 permanent redirect through ASP.NET, you need to do a little bit of manual labour; don’t worry it’s only two lines of code:

  • Response.Status = "301 Permanently Moved";
  • Response.AppendHeader ("Location", URI);

At this point, so long as nothing else interferes with the response - the client will receive a standard run of the mill 301 HTTP permanent redirect. If you know or are worried that something following the above lines may modify the response, you can issue an optional Response.End() as the third line to make it take effect immediately.

Rapid Development Using Django

Sunday, March 30th, 2008

In February, Jesper Nøhr wrote about taking an idea from conception to profitable web site in 24 hours. The project involved building an advertising product for the indie crowd, so they could advertise their products throughout other web sites in a similar fashion to how Google Adwords & Google Adsense works.

The final product named The Indiego Connection, allows advertisers to sign up and their account is manually verified to make sure that it meets their indie requirements. Once their account has been approved, they can go about configuring their advertising, which is then displayed throughout other web sites.

The really interesting thing for me about this project was the technical aspects of it, which involved:

Jesper wrote the front end of the site using Django, as he uses it for his day job. Given the demanding time frame that the product was built in, I expect that as many of the existing applications were utilised - such as auth. The prototype for the advertising server was built using CherryPy and once Jesper was satisfied with how it was constructed, moved that into Erlang for the lightweight threading and performance.

About 24 hours after starting the project, Indiego Connection was pushed into the wild. Word got out about a free advertising product for the indie crowd quickly and within hours they had over 100 users.

In any sort of normal environment, working an idea from start to finish in 24 hours would seem nearly impossible; especially if technology is involved. Through clever use of the tools, its allowed Jesper to rapidly develop a complete product in a short space of time.

Change The Django Source Directory With A .pth File

Monday, February 11th, 2008

John DeRosa recently wrote about using junctions (the Windows version of a symbolic link within Unix) to change the Django source directory. While that solution clearly works, I got the impression it seemed like a lot of hard work and John asked if there was a better way of handling that scenario.

If you’re wondering why a developer might want different versions of Django installed - it’s common if you need to upgrade an existing site from one version to another or if you offer a distributed application that is meant to work on the latest and also older versions of Django.

Under normal circumstances, developers place a copy of the Django source code underneath their ${install_path}\Lib\site-packages\ folder. While this works, it isn’t particularly convenient if you want or need to switch between the different versions. Under a unix based system, the common way of switching between the different versions is using a symbolic link; so you’d change the symbolic link from pointing from version A to version B. Unfortunately, in a Windows environment the concept of a symbolic link either doesn’t exist or isn’t common knowledge. As a by product, the average developer is only left with the option of copy and pasting the Django source code around in their file system or renaming folders - not particular convenient.

Fortunately, a little known feature of Python is that it supports special files with a .pth file extension. The .pth files or path files, are using by the site module (found under ${install_path}\Lib\) when the interpreters environment is initialised. The .pth files are standard text files, which contain one path per line, which when loaded are added into the sys.path collection.

A quick run down, place a django.pth file under ${install_path}\Lib\site-packages\, with a single line pointing towards your preferred Django source code version. When you need to change the version of Django you’re developing against - change the path within the file. Remember that if you have a Python interpreter open at the time, you’ll need to restart it and likewise if you’re running Django via something like Apache.

Installing Django On Windows

Thursday, February 7th, 2008

Ever needed to install Django in a Microsoft Windows environment, here is a quick start guide to make that happen:

  1. Read through the official Django installation documentation, it might just save you a world of hut down the road.
  2. Download Python for your version of Windows.
  3. Install Python, my preference here is to put it into the Program Files folder under a folder named Python<Version>
  4. Add your chosen Python installation path into your Windows path environment variable. This is an optional step, however it allows you to just type python in the command line and have it fire up the Python interpreter. An easy way of adding it is going into Control Panel, System and into the Environment Variables section.
  5. Download Django, you can either download a compressed file or if you’re comfortable with using version control - check it out from the Django Subversion repository.
  6. Create a folder named django under your <Python installation directory>\Lib\site-packages\ folder. Using my example above that would have been C:\Program Files\Python25\Lib\site-packages\.
  7. If you chose to download the compressed file, open it and extract the contents of the django folder into your newly created folder. If you’d prefer to check it out from Subversion, the normal check out points are http://code.djangoproject.com/svn/django/trunk/ for the latest development copy or a named release which you’ll find under http://code.djangoproject.com/svn/django/tags/releases/.

Done, you now have a working Django installation on Windows.

At this point, it’d be pertinent to confirm that everything is working properly, which you can do by following the first Django tutorial. The tutorial will make mention of django-admin.py, which is a utility which offers some basic functionality to get you off the ground. The file is located in the bin folder under your Django installation directory. When you need to use it, you can either type in the full path to it or simply add that file path into your environment variables as well.

Extending Nintendo Wii

Sunday, December 23rd, 2007

Johnny Lee, a human computer interaction student has released a series of videos demonstrating alternative uses for the Nintendo Wii gaming console. At this stage, Johnny has released videos showcasing:

  • Minority Report style hand gestures to control your computer
  • low cost interactive white boards
  • head tracking for desktop virtual reality displays

The cool factor of the videos has certainly caught the eyes of a lot of people as Johnny is reporting the 30 days following his initial video has seen over half a million unique visitors into the site and coverage from half a dozen of the top technology sites on the net.

Also, for those that care the videos above are accompanied by their respective .NET code samples.

Django Northwind Coming Soon

Saturday, December 15th, 2007

Whenever someone in the Microsoft development world has needed to demonstrate virtually anything utilising a database, the Microsoft Northwind database has been used.

The beauty of the Northwind database being so widely recognised and frequently reused, is that developers around the world don’t need to concern themselves with learning or understanding a new database schema every time another developer wishes to provide an example. Instead, the developer doing the example can simply state that his or her example is based off of the Northwind database and by proxy of its popularity, the majority of the readers will immediately understand.

Since the Microsoft Northwind database is so popular and used so widely, it seemed like a good idea to make it available for anyone wanting to do some simple examples using Django. I’ll be releasing the initial version of it in the coming days, keep your eyes peeled if you’re interested.

Django, Initial Data & Many To Many Tables

Friday, November 9th, 2007

Django provides functionality through manage.py to manage the database that you’re using. The various utility commands range from building your database using syncdb command to tearing it down using flush.

The functionality to manage the database through Django is great, however it’s often required to load initial data into a database when it is first created. The initial data might be used for running unit tests against or actually providing some useful default information into a system every time it is built.

Django provides two mechanisms to handle this functionality:

Initial SQL
The initial SQL way relies on providing a plain text file with SQL statements in it. The SQL files are located in a folder named ’sql’ under each application. There is no guarantee on what order the files will be executed, just that they will be executed some time after the database creation has been completed. Each SQL file provided should be named after the lower case version of the model name it is for; more about this point later.
Fixtures
Fixtures are essentially blocks of data, which do not necessarily need to be related to one another. Django supports various formats for the fixture files, ranging from JSON to YAML but accepts anything which is has a serialiser for. Using fixtures means that the developer can group various bits of information together, lets say specifically for a unit test and have it loaded each time the unit test is run. As a convenient feature, Django provides a way to dump the data currently in the database into a fixture file in whatever format you’d like and then reload it at a later time using the loaddata command.

Not knowing any better, I followed the SQL path initially simply because it was familiar to me. Everything was going just fine, every model I created I produced a useful collection of SQL statements to match. This process continued until the model definitions got a little more complex and I hit a wall when I defined a many to many relationship.

The problem was pretty straight forward, the SQL method states that the SQL file must be named after the lower case name of the model that it refers to. There is no problem with this until you produce a model that automatically generates tables, such as a many to many table relationship. I tried a number of different file names, however since the table is automatically generated off an existing model; I couldn’t find a file named that worked.

Given how complete Django seems to be in general, I’ll be surprised if I don’t find out that there is a way to do this down the road. Until that road is crossed though, I moved onto using fixtures to load the initial data into the database and it works a treat. I can conveniently use the automatic admin that Django provides to create some sample data and then just dump it into a fixture to reload at a later time.