Home | Blog | Brainjam | Publications | People

Dilbert Author Invents ZoneTag

From a recent post by Scott Adams on his Dilbert Blog:

First, my digital camera should have GPS so it always knows where I am. When I download my photos, a Google map would pop up, and the photos would go into storage according to the points on the map where the pictures were taken, ordered by date. The map forms the backdrop for organizing the scrapbook.

Second, I would use a special credit card for all purchases on my vacation, from gas stations to hotels to restaurants. The special part is that the records of my purchases would feed into my automatic scrapbook software and coordinate it with the camera’s GPS data. That would be enough data for the scrapbook system to intelligently guess the name of the restaurant or attraction where I was at the time of the picture.

Third, the system needs face recognition software so it can label photos with at least the names of family and friends who appear in them. It doesn’t need to be 100% accurate, but it could give you a big head start.

Minus the face recognition (which ZoneTag compensates for by suggesting tags based on your history and the tag’s likelihood, which often gets the names of the people in your images), we’re already there. And there’s no need to get into your credit report, Scott! ZoneTag knows where you are and will show you names of restaurants, landmarks and attractions around you as you take a photo. Just click, and it’s captures.

Get a Nokia N95, Scott, and start ZoneTagging. If you need help setting up, feel free to drop us a line. We’ll see what we can do (an originally-signed strip of Dilbert will get you up and running — and you know what — we’ll send you the phone pre-installed as well…).

via Andy.

Flickr Fountain of Knowledge

What can we learn from Flickr? Well, for one, we have learned that there are a lot of people who like to take photographs and share them publicly. Who would have guessed! However, my question refers to a different type of knowledge: information about the world that is implicitly encoded in the activity on Flickr.

You do not need to go far to see a simple yet brilliant example of such knowledge: check out Flickr’s tag clusters (here are the clusters for love, jaguar, Taj Mahal, hack). Using tag co-occurrence on Flickr photos, Flickr’s clustering can break down a term into multiple semantics or meanings: Jaguar, for example, is the animal as well as the car and the guitar: the first co-occurs with the tags “zoo” and “cat”; the second meaning of “jaguar” appears with “car” and “auto”. Note that these meanings are not mined from any other resource: they represent some “knowledge” that is generated automatically from the implicit contributions of Flickr users uploading and tagging their photos.

In other examples, Patrick Schmitz developed a different co-occurrence model that allowed him to generate subsumption data in Flickr tags (e.g. San Francisco is subsumed by California). The work at Yahoo! Research on TagLines and at our own lab on Tag Maps had shown that Flickr community activity generates descriptive labels for events and locations.

Last week, in Amsterdam, as part of SIGIR 2007, we added yet another method of extracting knowledge from Flickr. The paper, “Towards Automatic Extraction of Event and Place Semantics from Flickr Tags”, by Tye Rattenbury, Nathan Good (two of our star interns) and myself*, begins to answer a simple question: given a tag that appears on Flickr (such as “dog”, “SIGIR 2007″, or “Yahoo! Research Berkeley”), can we automatically determine whether or not that tag refers to a specific place, and whether or not the tag refers to a specific event? As you may guess, SIGIR 2007 refers to an event, Yahoo! Research Berkeley is a place, and “dog” is neither a place not an event.

Knowing if a tag is a place or event leads to better image search, but can also help us to better visualize the Flickr data; generate automatic event and place gazetteers; associate missing time/location metadata based on tags, and more.

I will not get into the details of how we propose to do extract the place/event knowledge from Flickr; you can get these details in our paper (pdf). I will just mention that we are using the dataset of geotagged Flick photos, and looking at the time and location distributions for each individual tag in the dataset. If the location or time distribution for a tag have specific “structure” to them, we classify that tag as a place or event, accordingly.

Below, you can follow the presentation slides I gave at SIGIR, or just jump directly to the paper to get the full story.

While the debate on the “Is the semantic web is dead?” question continues, “emerging semantics” are alive and kicking. What other knowledge can be extracted from the Flickr dataset?

* “Towards” is a code word in research papers meaning “we didn’t take the research all the way quite yet but want to make the paper sound important nevertheless” - we try not use it too much.

Got N95? ZoneTag Now Runs on Nokia 3rd Edition Phones

You have a spiffy new Nokia N95, N73 or one of Nokia’s latest and greatest new cameraphones, but you were not able to run ZoneTag? Be sad no more. We’re very happy to announce ZoneTag now works for Nokia Series 60 3rd edition phones. For phones like the N95 with built-in GPS, ZoneTag will make geotagging easier than ever.

So, what are you waiting for? Get ZoneTag now. If you are a new user, just click the big orange “Try it now button”. If you are an existing ZoneTag user looking to upgrade (come back to ZoneTag!), click on the My ZoneTag link on top and you’ll see instructions on how to upgrade.

While you’re at it, make sure you check out ZoneTag’s faithful companion, Zurfer, the coolest mobile photo viewing experience on the planet. And let us know how it goes.

Looking forward to seeing all your super hi-res photos up on Flickr!

Oh Bay Area people, Learn about the Fire Eagle this Monday

Are you in the Bay Area? Are you dying to learn about Fire Eagle and get early Super Alpha VIP access? Then come to Mobile Monday this week, July 2nd. Even if you are not dying to learn about Fire Eagle, it would be a great chance to learn and discuss location based services.

* What: July 2007 Mobile Monday (Location Based Services)
* When: July 2nd, 2007 7:00pm
* Where: TellMe Networks Inc., 1310 Villa Street, Mountain View, CA 94041
* Who: Anyone interested in mobility
* Cost: Nothing!

Official announcement; Upcoming event. See you there!

World Explorer: A JCDL best paper!

Last night, we discovered our work on TagMaps / World Explorer won best paper at ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL). Rahul’s in British Colombia where he presented the paper, World Explorer: Visualizing Aggregate Data from Unstructured Text in Geo-Referenced Collections.

Read the paper, check out the demo and many shouts to Shane, Mor, Rahul, & Jeannie. Fantastic!

Oh and thanks Rahul for letting us know via this Flickr photo.