Home | Blog | Brainjam | Publications | People

Flickr Fountain of Knowledge

What can we learn from Flickr? Well, for one, we have learned that there are a lot of people who like to take photographs and share them publicly. Who would have guessed! However, my question refers to a different type of knowledge: information about the world that is implicitly encoded in the activity on Flickr.

You do not need to go far to see a simple yet brilliant example of such knowledge: check out Flickr’s tag clusters (here are the clusters for love, jaguar, Taj Mahal, hack). Using tag co-occurrence on Flickr photos, Flickr’s clustering can break down a term into multiple semantics or meanings: Jaguar, for example, is the animal as well as the car and the guitar: the first co-occurs with the tags “zoo” and “cat”; the second meaning of “jaguar” appears with “car” and “auto”. Note that these meanings are not mined from any other resource: they represent some “knowledge” that is generated automatically from the implicit contributions of Flickr users uploading and tagging their photos.

In other examples, Patrick Schmitz developed a different co-occurrence model that allowed him to generate subsumption data in Flickr tags (e.g. San Francisco is subsumed by California). The work at Yahoo! Research on TagLines and at our own lab on Tag Maps had shown that Flickr community activity generates descriptive labels for events and locations.

Last week, in Amsterdam, as part of SIGIR 2007, we added yet another method of extracting knowledge from Flickr. The paper, “Towards Automatic Extraction of Event and Place Semantics from Flickr Tags”, by Tye Rattenbury, Nathan Good (two of our star interns) and myself*, begins to answer a simple question: given a tag that appears on Flickr (such as “dog”, “SIGIR 2007″, or “Yahoo! Research Berkeley”), can we automatically determine whether or not that tag refers to a specific place, and whether or not the tag refers to a specific event? As you may guess, SIGIR 2007 refers to an event, Yahoo! Research Berkeley is a place, and “dog” is neither a place not an event.

Knowing if a tag is a place or event leads to better image search, but can also help us to better visualize the Flickr data; generate automatic event and place gazetteers; associate missing time/location metadata based on tags, and more.

I will not get into the details of how we propose to do extract the place/event knowledge from Flickr; you can get these details in our paper (pdf). I will just mention that we are using the dataset of geotagged Flick photos, and looking at the time and location distributions for each individual tag in the dataset. If the location or time distribution for a tag have specific “structure” to them, we classify that tag as a place or event, accordingly.

Below, you can follow the presentation slides I gave at SIGIR, or just jump directly to the paper to get the full story.

While the debate on the “Is the semantic web is dead?” question continues, “emerging semantics” are alive and kicking. What other knowledge can be extracted from the Flickr dataset?

* “Towards” is a code word in research papers meaning “we didn’t take the research all the way quite yet but want to make the paper sound important nevertheless” - we try not use it too much.