This has been a long update to come and I hope you will all realise the sort of step forward it represents.

First of all, let’s talk a little bit of the smaller updates.
- there has been quite a bit of improvement on previous/next navigation. Using the left or right arrow keys bring the previous/next picture into view in a more convenient manner by moving and scaling the display appropriately. Reviewing a set of pictures is now much more convenient. Let me know if you have more suggestions.
- the right-click menu should be working better now, but still can take some time to show up (pretty much like in the Windows Explorer itself). Note: it works on images, not on year/months/day/group icons
- I’ve started integrating Intel’s Threading Building Blocks for new features and it works amazingly well. This will be retrofitted into the rest of the app and might make things even smoother.

Now, the main new feature: SIFT integration

SIFT is a computer vision technology that enables the computer to recognize elements in an image. The general principle is that it associates a number of keypoints that it can later detect elsewhere. This has received quite a bit of attention in the particular application of automatic panorama stitching, where it has shown great precision and convenience.

In gpuViewer, my aim is to provide a means to find a picture by providing a variation. Let’s take one simple example: I keep posting pictures online in a hurry, and sometimes I just can’t remember where the original is. With this new version, it’s only a matter of copying the web version in the clipboard and asking gpuViewer to find it. Thanks to the keypoints, it can find pictures even when they have been well photoshop’ed. On my two current test databases, which contain 20000 and 15000 SIFT-indexed pictures, the search time is in the order of a few seconds (on a QuadCore Q6600/2GB/Vista PC). This is partly thanks to SSE&multicore optimisation of the matching code but mainly down to an original indexing algorithm that I’m still tuning but that seems extremely promising. Revolutionary, even, as I don’t know of other efficient ways to index SIFT data.

The process of indexing is reasonably fast, when you take into account that, for each photo, this involves decoding it, detecting up to a thousand keypoints, storing the resulting 128K on disk and keeping indexes in place. Now, when you are looking at indexing tens of thousands of images, this can mean days… Fortunately you don’t add 10K pictures everyday in your database . To make it easier to do partial tests, the database is indexed going back in time (ie the newest stuff first). If you cancel and restart later, it does recognise what’s already been done.

Let’s look a bit into the future: what is this all really for? Surely, looking up one’s own pictures is cool, and can come in handy when someone brings in a picture that he wants an enlargement of (scan it, and let gpuViewer search), but isn’t there something cooler? Well, it all depends on how well my indexing method scales. If it scales as much as I hope, then it will be possible to have central indexes (on the web, on Intranets) that will be able to let anyone find information about anyone else’s (indexed) picture. That’s right: imagine showing gpuViewer a copy of the famous surfer+shark picture (at the bottom of this post) and be directed to Kurt Jones’ site… Where you might find more pictures, licensing info, prints ordering details, or whatever the author might want (hey, it’s his site…)

Ok, this might all sound quite crazy but it isn’t and the best way for you to test is to check it out.

Go to the dowload area and download your version now!

Attention! This version is *not* SIFT enabled by default. To enable the features, you will need David Lowe’s “SIFT demo program” and copy it in the gpuViewer install directory. Please, take note of the important notice he puts forward: This demo software is provided for research purposes only. A license must be obtained from the University of British Columbia for any commercial applications. The sofware is protected under a US patent as listed below. This demo software is a research implementation, while the licensed software has been further optimized for speed and to provide a range of other capabilities. See the LICENSE file provided with the demo software. Make sure this applies to your test !

Once you have added the siftWin32.exe file to the directory, there will be a new menu in the main menu bar and new items in the context menu for photos. I’m afraid you will have to experiment by yourself until I update the user manual for gpuViewer (hopefully, tomorrow).

As always, please, keep the feedback coming!

This entry was posted on Wednesday, January 23rd, 2008 at 8:28 pm and is filed under announcements. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.