Tag Archives: python

Project Euler, Problem #1

The first problem within Project Euler isn’t meant to twist the brain in knots and is a gentle introduction to what is coming further down the road. The problem states:

If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23.

Find the sum of all the multiples of 3 or 5 below 1000.

As I mentioned in my initial post about exploring Project Euler, I noted that I often write in a lowest common denominator with whatever language I’m using at the time. The net effect of that decision is often a working but poorly optimised solution or a solution that doesn’t fit well with the spirit of the programming language.

The solution to problem #1 took three distinct evolutions, the first of which is below and looks like a solution you’d likely see in any programming language:

i = 1
sum = 0

while i < 1000:
  if i % 3 == 0 or i % 5 == 0:
    sum += i
  i += 1

print sum

I wasn’t a fan of needing to declare the variables to start with for such a simple problem, so removed the need for the i but using the Python range function to generate a series of numbers. The range function allows you to generate any length series of numbers, starting and ending where you want with any stepping. By default, it starts at 0 and a stepping of 1 – which means range(1000) is going to generate a list of numbers from 0 – 999.

sum = 0

for x in range(1000):
  if x % 3 == 0 or x % 5 == 0:
    sum += x

print sum

Python provides functionality for its lists called list comprehensions. The idea behind a list comprehension is that you can perform the same action, whatever that may be, against each item in the set. In the final solution below, I have applied a filter (the if statements), which now only returns those items from the for x in range that match and I’m not applying an expression to the resulting matches (the solitary x before the for). Once collected the new list is passed to the sum function to yield our result.

sum([x for x in range(1000) if x % 3 == 0 or x % 5 == 0])

Project Euler, An Exercise In Exploration

Some time ago I stumbled onto a math based problem solving site for programmers named Project Euler. The Project Euler site describes itself as:

Project Euler is a series of challenging mathematical/computer programming problems that will require more than just mathematical insights to solve. Although mathematics will help you arrive at elegant and efficient methods, the use of a computer and programming skills will be required to solve most problems.

The motivation for starting Project Euler, and its continuation, is to provide a platform for the inquiring mind to delve into unfamiliar areas and learn new concepts in a fun and recreational context.

Possible solutions to the problems are verified through the Project Euler web site, where your successes are recorded if you sign up for an account. You don’t need to work through the problems in order, however doing so may be beneficial as work completed and previous questions is often reused.

I thought Project Euler would be a great exercise to explore more of the Python programming language. While I’ve read quite a bit about the Python language, I’ve not written anything substantial in it and I often find myself using the lowest common denominator in my approaches to solving problems as a by product.

My intention while working through the problems presented within Project Euler is to find a neater, cleaner method of solving the problem which is hopefully more pythonic. I’ll present my solution to the problems and with a little luck, I’ll receive some helpful improvements from the great programming community as well.

Source Control Commit Visualisation

Software development relies on source control management software such as CVS, Subversion, SourceSafe, Bitkeeper, Mercurial, Git and the like to track and manage the changes in the source code over time. As a project progresses, developers come and go, contractors come and go and the activity on a given project ebs and flows as required.

Attempting to visualise who, what and how much of a project is changing is quite complex as there are so many variables – however Michael Ogawa has built a project named code_swarm which does just that. Instead of providing tabular or static images to help visualise a projects changes, he has managed to animate it into something quite spectacular.

Following are five different code swarm visualisations of popular open source projects:

The amazing thing that a visualisation such as code_swarm provides, is to show just how many people actively participate in a given open source project, how much each of them participates and what sort of tasks they are normally performing on that project. As an example, comparing the number of different people in SQLAlchemy compared to Django isn’t a competition – Django is ahead by a mile, though compared to Apache, the others seem insignificant.

Rapid Development Using Django

In February, Jesper Nøhr wrote about taking an idea from conception to profitable web site in 24 hours. The project involved building an advertising product for the indie crowd, so they could advertise their products throughout other web sites in a similar fashion to how Google Adwords & Google Adsense works.

The final product named The Indiego Connection, allows advertisers to sign up and their account is manually verified to make sure that it meets their indie requirements. Once their account has been approved, they can go about configuring their advertising, which is then displayed throughout other web sites.

The really interesting thing for me about this project was the technical aspects of it, which involved:

Jesper wrote the front end of the site using Django, as he uses it for his day job. Given the demanding time frame that the product was built in, I expect that as many of the existing applications were utilised – such as auth. The prototype for the advertising server was built using CherryPy and once Jesper was satisfied with how it was constructed, moved that into Erlang for the lightweight threading and performance.

About 24 hours after starting the project, Indiego Connection was pushed into the wild. Word got out about a free advertising product for the indie crowd quickly and within hours they had over 100 users.

In any sort of normal environment, working an idea from start to finish in 24 hours would seem nearly impossible; especially if technology is involved. Through clever use of the tools, its allowed Jesper to rapidly develop a complete product in a short space of time.

Django Friendly Hosting

If you’re about to purchase hosting for your Django application, everything you’ll need to make a good decision is in one place at Django Friendly.

Ryan Berg is the man behind Django Friendly and put together the site as a way to consolidate the plethora of hosting options available for Django. As with some other scripting languages such as Ruby, Python also has some special hosting requirements which makes it inconvenient to host it within standard hosting configurations. The development of mod_wsgi for Apache is aiming to provide a simple, high performance option for hosting Python applications within shared hosting environments.

Allowing users the ability to filter web hosts by price and shared/dedicated hosting types is a great step forward. My suggested improvement for the site would be the ability for users to dynamically build a search query. As an example, being able to filter by server location, hosting type, price ranges, ratings and so on. Maybe an interface in a similar fashion to what is offered by a custom ticket search within the popular Trac software could be used as a starting point.

Either way, it’s another fantastic looking Django specific site which has been offered up to the greater community – you’ve got to love it.

Change The Django Source Directory With A .pth File

John DeRosa recently wrote about using junctions (the Windows version of a symbolic link within Unix) to change the Django source directory. While that solution clearly works, I got the impression it seemed like a lot of hard work and John asked if there was a better way of handling that scenario.

If you’re wondering why a developer might want different versions of Django installed – it’s common if you need to upgrade an existing site from one version to another or if you offer a distributed application that is meant to work on the latest and also older versions of Django.

Under normal circumstances, developers place a copy of the Django source code underneath their ${install_path}\Lib\site-packages\ folder. While this works, it isn’t particularly convenient if you want or need to switch between the different versions. Under a unix based system, the common way of switching between the different versions is using a symbolic link; so you’d change the symbolic link from pointing from version A to version B. Unfortunately, in a Windows environment the concept of a symbolic link either doesn’t exist or isn’t common knowledge. As a by product, the average developer is only left with the option of copy and pasting the Django source code around in their file system or renaming folders – not particular convenient.

Fortunately, a little known feature of Python is that it supports special files with a .pth file extension. The .pth files or path files, are using by the site module (found under ${install_path}\Lib\) when the interpreters environment is initialised. The .pth files are standard text files, which contain one path per line, which when loaded are added into the sys.path collection.

A quick run down, place a django.pth file under ${install_path}\Lib\site-packages\, with a single line pointing towards your preferred Django source code version. When you need to change the version of Django you’re developing against – change the path within the file. Remember that if you have a Python interpreter open at the time, you’ll need to restart it and likewise if you’re running Django via something like Apache.

Installing Django On Windows

Ever needed to install Django in a Microsoft Windows environment, here is a quick start guide to make that happen:

  1. Read through the official Django installation documentation, it might just save you a world of hut down the road.
  2. Download Python for your version of Windows.
  3. Install Python, my preference here is to put it into the Program Files folder under a folder named Python<Version>
  4. Add your chosen Python installation path into your Windows path environment variable. This is an optional step, however it allows you to just type python in the command line and have it fire up the Python interpreter. An easy way of adding it is going into Control Panel, System and into the Environment Variables section.
  5. Download Django, you can either download a compressed file or if you’re comfortable with using version control – check it out from the Django Subversion repository.
  6. Create a folder named django under your <Python installation directory>\Lib\site-packages\ folder. Using my example above that would have been C:\Program Files\Python25\Lib\site-packages\.
  7. If you chose to download the compressed file, open it and extract the contents of the django folder into your newly created folder. If you’d prefer to check it out from Subversion, the normal check out points are http://code.djangoproject.com/svn/django/trunk/ for the latest development copy or a named release which you’ll find under http://code.djangoproject.com/svn/django/tags/releases/.

Done, you now have a working Django installation on Windows.

At this point, it’d be pertinent to confirm that everything is working properly, which you can do by following the first Django tutorial. The tutorial will make mention of django-admin.py, which is a utility which offers some basic functionality to get you off the ground. The file is located in the bin folder under your Django installation directory. When you need to use it, you can either type in the full path to it or simply add that file path into your environment variables as well.

Django, Initial Data & Many To Many Tables

Django provides functionality through manage.py to manage the database that you’re using. The various utility commands range from building your database using syncdb command to tearing it down using flush.

The functionality to manage the database through Django is great, however it’s often required to load initial data into a database when it is first created. The initial data might be used for running unit tests against or actually providing some useful default information into a system every time it is built.

Django provides two mechanisms to handle this functionality:

Initial SQL
The initial SQL way relies on providing a plain text file with SQL statements in it. The SQL files are located in a folder named ‘sql’ under each application. There is no guarantee on what order the files will be executed, just that they will be executed some time after the database creation has been completed. Each SQL file provided should be named after the lower case version of the model name it is for; more about this point later.
Fixtures are essentially blocks of data, which do not necessarily need to be related to one another. Django supports various formats for the fixture files, ranging from JSON to YAML but accepts anything which is has a serialiser for. Using fixtures means that the developer can group various bits of information together, lets say specifically for a unit test and have it loaded each time the unit test is run. As a convenient feature, Django provides a way to dump the data currently in the database into a fixture file in whatever format you’d like and then reload it at a later time using the loaddata command.

Not knowing any better, I followed the SQL path initially simply because it was familiar to me. Everything was going just fine, every model I created I produced a useful collection of SQL statements to match. This process continued until the model definitions got a little more complex and I hit a wall when I defined a many to many relationship.

The problem was pretty straight forward, the SQL method states that the SQL file must be named after the lower case name of the model that it refers to. There is no problem with this until you produce a model that automatically generates tables, such as a many to many table relationship. I tried a number of different file names, however since the table is automatically generated off an existing model; I couldn’t find a file named that worked.

Given how complete Django seems to be in general, I’ll be surprised if I don’t find out that there is a way to do this down the road. Until that road is crossed though, I moved onto using fixtures to load the initial data into the database and it works a treat. I can conveniently use the automatic admin that Django provides to create some sample data and then just dump it into a fixture to reload at a later time.