Learning Django by Example(6): AJAX File Upload

Python, Web October 4th, 2007

In the last post, I just added the entities in the database, but did not bind the meta data with eBook files. This post will demonstrate how to upload the file in AJAX flavor:

The idea is inspired by this tutorial , using dojo.io.iframe, we could submit a form with file upload asynchronously. In client side, a hidden field meta is added to store the marshaled JSON string. dojo.io.iframe.send shares the same defer concept as dojo.xhrPost:

var meta = dojo.query("input[@name=meta]", form)[0];
    meta.value = dojo.toJson(cache[i]);

    // submit the form
    dojo.io.iframe.send({
        url: form.action,
        method: "post",
        handleAs: "json",
        form: form,
    }).addCallback(function(ret) {
    … …

NOTE: the handleAs is json, so the argument ret is JavaScript object.

In server side, the JSON string needs to be wrapped by textarea, this is the hard requirement of dojo.iframe.send.

return HttpResponse("<textarea>%s</textarea>" % simplejson.dumps(retset) , mimetype=‘text/html’);

Conceptually, add-by-search will issue INSERT operation to the database, the result may fall in the following categories:

  • Success
  • FileAlreadyExist: the eBook file exists, so the uploaded file is discarded
  • UnexpectedError: an 500 error happens in server side for unknown reason.

Check r34 for the implementation.

Learning Django by Example(5): Time to Attack

Python, Web September 19th, 2007

In the last four section(1, 2, 3, 4) ,we have been familiar with the environment, it is time to do some real work.

I will add more fresh to models.py , linking eBook file and its meta data, thumbnail of the cover, regular size cover etc. Two other models are added for eBook:

class FileType(models.Model):
    type = models.CharField(maxlength=10)

class File(models.Model):
    type = models.ForeignKey(FileType)
    handle = models.FileField(upload_to="files")
    meta = models.ForeignKey(Book)

Book to File is one-to-many as one book may have more than one formats, a PDF version for print, a CHM version for reference.

To server the images of book covers, , we need to setup the MEDIA_ROOT and MEDIA_URL:

MEDIA_ROOT = ‘/home/share/eBooks/’
MEDIA_URL = ‘http://localhost:8000/repo/’

and setup the url mapping as well:

(r’^repo/(?P .*)$’, ‘django.views.static.serve’, {‘document_root’: ‘/home/share/eBooks’}),

To save an image file, we can easily call save_FOO_file method, it works with two drawbacks: first, you need to read the image into the memory; second, it is synchronous function call aka it is blocked until the file is fetched, read and written. urlretrive is preferred in this way:

from django.conf import settings
from urllib import urlopen, unquote, urlretrieve
from urlparse import urlparse
from os import path
… …
                thumb = unquote(urlparse(item[‘thumburl’])[2].split(‘/’)[-1])
                # using hard-coded image temporary
                urlretrieve(item[‘thumburl’], path.join(settings.MEDIA_ROOT, ‘images’, thumb))
                book.thumb = path.join(‘images’, thumb);
                book.save()

A further optimization may introduce Twisted or asyncore.

More work needs to be done in the UI side: the Book objects are populated in brief or detail modes. Brief mode just highlight the title, cover, authors and ISBN of the book, commonly used in general listing, search results, my favorite and so on; while detail mode shows more detailed information like editorial review, related book, tags etc. It is worthy extracting the code snippet of brief mode to a template, so we can reuse it later by {% include book_item_brief.html %}. Detail view would fetch the editorial view from Amazon using the same AJAX hack as discussed before.

With stylesheet, it looks much better, Brief view:

Breif view



and Detail view:

Detail view



Check r30.

Learning Django by Example(4): First user authenticated

Python, Web September 18th, 2007

In the last post, we succeeded to update the database, in this post we would add more functionalities to the view of normal users.

In r23, I re-organize the file directory in the repository, media is added to the repository for your convenience.

Though Gelman is designed for personal use, the multi-user functionality is added just in case you want to share you collection with family members and friends later.The built-in authentication is quite neat, if you deploy Gelman into an enterprise environment, you may consider this. Now, just follow the documentation, and add the very first reader to Gelman by running python manage.py shell:

>>> from django.contrib.auth.models import User
>>> u = User.objects.create_user(‘python’, , ‘spam’)
>>> u.save()

Now, let’s build the gateway for the user to login. Copy the basic login form in the doc to templates/login.html, The default success login action is to redirect to user’s profile, we may override it to the index first:

<input type="hidden" name="next" value="/library/" />

add the URL mapping in urls.py:

(r‘^accounts/login/$’, ‘django.contrib.auth.views.login’, {‘template_name’: ‘login.html’, }),
       (r‘^accounts/logout/$’, ‘gelman.library.views.logout_view’, ),

For logout action, just logout the user and redirect the user to the front page, in library/views.py:

def logout_view(request):
    logout(request)
    return HttpResponseRedirect(‘/library’)

We would greet the users in the top-left corner of the page just as WordPress’s dashboard does. Add the following code snippet to base_generic.html:

{% if user.is_authenticated %}
Howdy, {{ user.username }} [Logout, My Profile]
{% else %}
[Login or Register]
{% endif %}

as the following(left: logged-in user, right: logged-out user):

Login and Logout demo



Check out r25 for the implementation details.

Learning Django by Example(3): Just works

Python, Web September 7th, 2007

In the last post, we succeeded to post the Javascript objects via JSON to the server, the data is to be serialized to the database in this section.

Before we move on, let’s remove some bumps in our way, the database scheme is modified in this way:

class Book(models.Model):
isbn = models.CharField(maxlength=10, primary_key=True)
pub_date = models.DateField(‘data published’)

The reasoning lies in that AWS returns ISBN-10 as ASIN, we may take advantage of this. Another fix is to replace DateTimeField by DateField for pub_date. Django does not support seamless database scheme refactory so far. So we just dropped the database and re-generate it as described in the tutorial. The username/password for new database is set as admin/gelman as common practice.

It is quite annoying that a 500 error returned without any reason, Django server ate the exception due to the nature of Web server. We could wrap the suspicious buggy code with try/except block like this:

try :
    // database access
except :
    print sys.exc_info()[0]

Well, InterfaceError or IntegrityError pinned the bug in database insertion, but where? Interactive shells are our best friends, you can either use pdb or python manage.py shell.

Oh, my fault, I forgot the return value of get_or_create is a tuple, instead of the object; and for many to many relation, we could not simply set a list as its value, we need to use add method after the object is constructed:

for item in simplejson.loads(request.POST[‘items’]):
        publisher, created = Publisher.objects.get_or_create(name=item[‘publisher’])
        authors = [Author.objects.get_or_create(name=au) for au in item[‘authors’]]
        authors = [author for (author, created) in authors];
        y,m,d = [int(x) for x in item[‘pub_date’].split(‘-’)]
        pub_date = date(y, m, d)
        book, created = Book.objects.get_or_create(isbn=item[‘isbn’], name=item[‘title’],
            pages=300, pub_date=pub_date, publisher=publisher)
        book.authors.add(*authors)
        #print sys.exc_info()[0]
    return HttpResponseRedirect(‘/admin/library/book/’)

At the exit, we just redirect the browser to a new URL.

It seems the database is updated, but the page is not redirected. Since the POST operation is launched in an AJAX request, we need to send JSON object as the response. In server side, simplejson.dump is used:

xhr = {‘succeed’: [], ‘failed’: []};
       for item in simplejson.loads(request.POST[‘items’]):
               # … ….
               try :
                       book, created = Book.objects.get_or_create(isbn=item[‘isbn’], name=item[‘title’],
                               pages=300, pub_date=pub_date, publisher=publisher)
                       book.authors.add(*authors)
               except :
                       xhr[‘failed’].append(item[‘title’])
               xhr[‘succeed’].append(item[‘title’])

       return HttpResponse(simplejson.dumps(xhr), mimetype=‘text/javascript’);

In client side, we evaluate the JSON string then populate the DOM nodes:

dojo.xhrPost({
                url: dojo.byId("book_form").action,
                content: { items: dojo.toJson(formdata) },
            }).addCallback(function(response) {
                var result = dojo.fromJson(response);
                var messages = dojo.query("ul[@class=messagelist]")[0]
                dojo.forEach(result.failed, function(item) {
                    var msg = document.createElement("li");
                    msg.innerHTML = "<li><em>" + item + "</em> <strong>failed</strong> to add.</li>"
                    messages.appendChild(msg);
                });
                dojo.forEach(result.succeed, function(item) {
                    var msg = document.createElement("li");
                    msg.innerHTML = "<em>" + item + "</em> successfully added."
                    messages.appendChild(msg);
                });
            });

The final result looks like this:

Add books with feedback



Check r19 for details.

Learning Django by Example(1): Start the Engine

Python, Web September 3rd, 2007

This is yet another Django tutorial, it makes a difference since

  • the author is totally a noob about Django programming, he has little experience on database and Web framework. You may find it interesting how a newbie benefit from the elegant design of Django.
  • each step is carefully recored via Google Code, you can check out the snapshot to understand how the project evolves.

Translations are freely permitted as long as they are released under Attribution-Noncommercial-Share Alike Creative Common License. Learning Django by Example has already been fully or partially translated into several languages. If you translate it into another language and would like to be listed here, just let me know.

Think Big

I really love the intuitive interface of Delicious Library, unfortunately, it is Mac-only, and it may not scale due to the lack of database server support. Personally I prefer a small daemon running silently in my old Gentoo box to serve contents to all clients, cross-platform and it is supposed to scale when the library grows, and I may share the resources with my friends. So I decide to brew my tea: Gelman, named after the library in the main campus of the George Washington University.

I choose Django for the python on rail hype. I just need a lightweight, pythonic Web framework. Hopefully I could enjoy the development process.

NOTE: In the following tutorial, I would show the code snapshot in each step, when I mention check r#, you can checkout the code via:

svn checkout http://gelman.googlecode.com/svn/trunk/gelman -r3 gelman

in the case, # is 3. If Google code supported Trac, that make it much easier.
Read the rest of this entry »