Learning Django by Example(7): Act like a desktop application: Dnd

Web October 6th, 2007

After some tests, I found the add-by-search has a big design flaw, I took the wrong direction: the focus of eBook management ultimately is the file, not the meta data. Current implementation is too meta-data centric, it adds the complexity since all dynamic features are created dynamically, and it looks like a little desperate to put so many controls in one page.

So in r37, a brand new UI is developed inspired by WordPress’s dashboard.

Dnd Overview



The user may drag the thumbnail from the search result and drop it in the left dropbox, attach a local file, then click Save, done. Here is another screen shot of drag-n-drop in action:

Dnd in action


Unlike the canonical Dnd example, our dropbox only accept no more than one item at any time, the newly added item would replace the old one. So we need to subscribe the /dnd/drop event:

dojo.subscribe("/dnd/drop", function(source, nodes, iscopy) {
        var t = dojo.dnd.manager().target;
        if(t == dropbox) {
            t.selectAll();
            t.deleteSelectedNodes();
            t.clearItems();
        }
        var jsnode = source.getItem(nodes[0].id);
        t.insertNodes(false, [jsnode.data]);

    });

The first action is to remove all the items, then we extract the data, and reconstruct the nodes by calling insertNodes. The rest part is quite the same.

An advanced version is in the blueprint: how about a file manager on FTP’s incoming directory? But I am lazy to copy-n-paste the code, I would rather grill myself to figure it out how to put the two options in one page more intuitively. If you have any suggestion or mock up, please just drop you comment here. Thanks.

Learning Django by Example(6): AJAX File Upload

Python, Web October 4th, 2007

In the last post, I just added the entities in the database, but did not bind the meta data with eBook files. This post will demonstrate how to upload the file in AJAX flavor:

The idea is inspired by this tutorial , using dojo.io.iframe, we could submit a form with file upload asynchronously. In client side, a hidden field meta is added to store the marshaled JSON string. dojo.io.iframe.send shares the same defer concept as dojo.xhrPost:

var meta = dojo.query("input[@name=meta]", form)[0];
    meta.value = dojo.toJson(cache[i]);

    // submit the form
    dojo.io.iframe.send({
        url: form.action,
        method: "post",
        handleAs: "json",
        form: form,
    }).addCallback(function(ret) {
    … …

NOTE: the handleAs is json, so the argument ret is JavaScript object.

In server side, the JSON string needs to be wrapped by textarea, this is the hard requirement of dojo.iframe.send.

return HttpResponse("<textarea>%s</textarea>" % simplejson.dumps(retset) , mimetype=‘text/html’);

Conceptually, add-by-search will issue INSERT operation to the database, the result may fall in the following categories:

  • Success
  • FileAlreadyExist: the eBook file exists, so the uploaded file is discarded
  • UnexpectedError: an 500 error happens in server side for unknown reason.

Check r34 for the implementation.

Learning Django by Example(5): Time to Attack

Python, Web September 19th, 2007

In the last four section(1, 2, 3, 4) ,we have been familiar with the environment, it is time to do some real work.

I will add more fresh to models.py , linking eBook file and its meta data, thumbnail of the cover, regular size cover etc. Two other models are added for eBook:

class FileType(models.Model):
    type = models.CharField(maxlength=10)

class File(models.Model):
    type = models.ForeignKey(FileType)
    handle = models.FileField(upload_to="files")
    meta = models.ForeignKey(Book)

Book to File is one-to-many as one book may have more than one formats, a PDF version for print, a CHM version for reference.

To server the images of book covers, , we need to setup the MEDIA_ROOT and MEDIA_URL:

MEDIA_ROOT = ‘/home/share/eBooks/’
MEDIA_URL = ‘http://localhost:8000/repo/’

and setup the url mapping as well:

(r’^repo/(?P .*)$’, ‘django.views.static.serve’, {’document_root’: ‘/home/share/eBooks’}),

To save an image file, we can easily call save_FOO_file method, it works with two drawbacks: first, you need to read the image into the memory; second, it is synchronous function call aka it is blocked until the file is fetched, read and written. urlretrive is preferred in this way:

from django.conf import settings
from urllib import urlopen, unquote, urlretrieve
from urlparse import urlparse
from os import path
… …
                thumb = unquote(urlparse(item[‘thumburl’])[2].split(‘/’)[-1])
                # using hard-coded image temporary
                urlretrieve(item[‘thumburl’], path.join(settings.MEDIA_ROOT, ‘images’, thumb))
                book.thumb = path.join(‘images’, thumb);
                book.save()

A further optimization may introduce Twisted or asyncore.

More work needs to be done in the UI side: the Book objects are populated in brief or detail modes. Brief mode just highlight the title, cover, authors and ISBN of the book, commonly used in general listing, search results, my favorite and so on; while detail mode shows more detailed information like editorial review, related book, tags etc. It is worthy extracting the code snippet of brief mode to a template, so we can reuse it later by {% include book_item_brief.html %}. Detail view would fetch the editorial view from Amazon using the same AJAX hack as discussed before.

With stylesheet, it looks much better, Brief view:

Breif view



and Detail view:

Detail view



Check r30.

Learning Django by Example(3): Just works

Python, Web September 7th, 2007

In the last post, we succeeded to post the Javascript objects via JSON to the server, the data is to be serialized to the database in this section.

Before we move on, let’s remove some bumps in our way, the database scheme is modified in this way:

class Book(models.Model):
isbn = models.CharField(maxlength=10, primary_key=True)
pub_date = models.DateField(’data published’)

The reasoning lies in that AWS returns ISBN-10 as ASIN, we may take advantage of this. Another fix is to replace DateTimeField by DateField for pub_date. Django does not support seamless database scheme refactory so far. So we just dropped the database and re-generate it as described in the tutorial. The username/password for new database is set as admin/gelman as common practice.

It is quite annoying that a 500 error returned without any reason, Django server ate the exception due to the nature of Web server. We could wrap the suspicious buggy code with try/except block like this:

try :
    // database access
except :
    print sys.exc_info()[0]

Well, InterfaceError or IntegrityError pinned the bug in database insertion, but where? Interactive shells are our best friends, you can either use pdb or python manage.py shell.

Oh, my fault, I forgot the return value of get_or_create is a tuple, instead of the object; and for many to many relation, we could not simply set a list as its value, we need to use add method after the object is constructed:

for item in simplejson.loads(request.POST[‘items’]):
        publisher, created = Publisher.objects.get_or_create(name=item[‘publisher’])
        authors = [Author.objects.get_or_create(name=au) for au in item[‘authors’]]
        authors = [author for (author, created) in authors];
        y,m,d = [int(x) for x in item[‘pub_date’].split(‘-’)]
        pub_date = date(y, m, d)
        book, created = Book.objects.get_or_create(isbn=item[‘isbn’], name=item[‘title’],
            pages=300, pub_date=pub_date, publisher=publisher)
        book.authors.add(*authors)
        #print sys.exc_info()[0]
    return HttpResponseRedirect(‘/admin/library/book/’)

At the exit, we just redirect the browser to a new URL.

It seems the database is updated, but the page is not redirected. Since the POST operation is launched in an AJAX request, we need to send JSON object as the response. In server side, simplejson.dump is used:

xhr = {’succeed’: [], ‘failed’: []};
       for item in simplejson.loads(request.POST[‘items’]):
               # … ….
               try :
                       book, created = Book.objects.get_or_create(isbn=item[‘isbn’], name=item[‘title’],
                               pages=300, pub_date=pub_date, publisher=publisher)
                       book.authors.add(*authors)
               except :
                       xhr[‘failed’].append(item[‘title’])
               xhr[’succeed’].append(item[‘title’])

       return HttpResponse(simplejson.dumps(xhr), mimetype=‘text/javascript’);

In client side, we evaluate the JSON string then populate the DOM nodes:

dojo.xhrPost({
                url: dojo.byId("book_form").action,
                content: { items: dojo.toJson(formdata) },
            }).addCallback(function(response) {
                var result = dojo.fromJson(response);
                var messages = dojo.query("ul[@class=messagelist]")[0]
                dojo.forEach(result.failed, function(item) {
                    var msg = document.createElement("li");
                    msg.innerHTML = "<li><em>" + item + "</em> <strong>failed</strong> to add.</li>"
                    messages.appendChild(msg);
                });
                dojo.forEach(result.succeed, function(item) {
                    var msg = document.createElement("li");
                    msg.innerHTML = "<em>" + item + "</em> successfully added."
                    messages.appendChild(msg);
                });
            });

The final result looks like this:

Add books with feedback



Check r19 for details.

Learning Django by Example(1): Start the Engine

Python, Web September 3rd, 2007

This is yet another Django tutorial, it makes a difference since

  • the author is totally a noob about Django programming, he has little experience on database and Web framework. You may find it interesting how a newbie benefit from the elegant design of Django.
  • each step is carefully recored via Google Code, you can check out the snapshot to understand how the project evolves.

Translations are freely permitted as long as they are released under Attribution-Noncommercial-Share Alike Creative Common License. Learning Django by Example has already been fully or partially translated into several languages. If you translate it into another language and would like to be listed here, just let me know.

Think Big

I really love the intuitive interface of Delicious Library, unfortunately, it is Mac-only, and it may not scale due to the lack of database server support. Personally I prefer a small daemon running silently in my old Gentoo box to serve contents to all clients, cross-platform and it is supposed to scale when the library grows, and I may share the resources with my friends. So I decide to brew my tea: Gelman, named after the library in the main campus of the George Washington University.

I choose Django for the python on rail hype. I just need a lightweight, pythonic Web framework. Hopefully I could enjoy the development process.

NOTE: In the following tutorial, I would show the code snapshot in each step, when I mention check r#, you can checkout the code via:

svn checkout http://gelman.googlecode.com/svn/trunk/gelman -r3 gelman

in the case, # is 3. If Google code supported Trac, that make it much easier.
Read the rest of this entry »