Home | Blog | Brainjam | Publications | People

Some dinner tables know your friends

It’s been a month since we all saw the shores of Santa Barbara at ACM Multimedia. For some of us, our brains are just now leaving the conference (or coming off the beach). For others, we return to work after a long weekend of family, friends, and food. On Thanksgiving day in the United States, dining tables get the most attention.

Our daily interaction with tables is an indication of our society and culture. Your non-virtual social network is a snapshot of people you meet during the day. I’m talking about real people, in person, outside what we do with Email/IM or in online social groups (like LinkedIn, Facebook, and MySpace). How about capturing and growing your everyday social networks? Think about it for a moment in the physical sense. Who ate dinner with you at Thanksgiving? Who sat in your Monday morning meeting? Who do you go with to get a coffee? At ACM Multimedia 2006, artist Noriyuki Fujimura, seen here with his previous installation Remote Furniture (SIGGRAPH 2004), enhanced an ordinary table to track his real world social network and externalize the communities built around everyday social gatherings.

In his installation, an omni-directional camera takes photos of the people at the table (who are identified by their RFID badges). Noriyuki calls this a Community Snapshot:

Out of an accumulation of “Community snapshot” data from multiple tables, a computer working behind analyses it and produces a hypothetical community map projected onto the screen. Participants can browse the map interactively by focusing on a person, viewing the whole community or finding pathways between persons. the map here is created solely from human connectedness and multimedia data of the scene of gatherings…. Artwork as a community device has been conceived of for a few decades in the field of contemporary/public art such as Joseph Beuys’s “7000 Oaks”. Considering the social aspect of emerging ubiquitous computing technology, there is much to explore in the intersection of communication technology and public art.

The Community Snapshot grows as you join more table gatherings. It connects you with people in your social space and allows you to visualize and explore your real social network.

Noriyuki is an artist at Japan’s National Insitute of Advanced Industrial Science and Technology (AIST). His work was presented in the 2006 Interactive Arts Program at ACM Multimedia. Since 2004, the arts program provides venue for artists to present and discuss their work in publication or installation. The program also connects the artists with the scientists, allowing practice and theory to meet. And it gives people like Nori and myself (who live the life of artist and scientist) a place to swim.

Tag Maps: Visualizing the Crowd’s “Mental Map” Using Flickr Geotagged Images

In 1976, social psychologist Stanley Milgram asked his subjects to list places of interest in Paris. Milgram then aggregated the results, effectively creating an “attraction map” of Paris with landmark names appearing in a larger font according to the number of subjects who mentioned each.

Can the same type of information and visualization be automatically derived from Flickr geotagged images and their associated tags? At this year’s ACM MM MIR workshop, we showed the answer to be “yes”. The idea is simple: By taking a photo, photographers essentially express their interest in a particular place. Individual pictures taken at a specific location act as “votes” in favor of that location’s interest, much like the explicit input of Milgram’s subjects. Further, additional information can be extracted from the tags attached to these photos on Flickr. Tags that frequently appear in images from a specific location but are otherwise rare suggest a topic unique to the location.

By analyzing the patterns of location, photographers, and tags in a photo data set, our system generates tag maps that mirror Milgram’s manually created attraction map. While Milgram was testing his ideas in Paris (good for him!) we were looking at London data (we’ll do Paris soon - even if we have to go there and investigate in person). The figure above shows a tag map of central London, derived from Flickr’s geotagged photos. The attractions that emerge from the London data include Buckingham Palace, London Eye, and Big Ben—all generated automatically with the implicit contributions of Flickr users. The data we used to generate the map is Flickr geotagged images from May 2006 - a lot more are available now, and we promise to run it again on the new data set very soon - stay tuned! The beauty of it is that Tag Maps usually improve as more content is added, alleviating the overload problem often associated with large collections.

Incidentally, the same algorithm that produces tag maps can also summarize a photo collection by selecting representative images based on the collection’s patterns. The system can then overlay the selected photos on a map in their capture location, effectively illustrating the region’s “vibe” via images rather than text.

Above is a SlideShare of the presentation I gave at MIR, including more details about how all this actually works. If you are still curious, I encourage you to take a look - imagine that - at the actual paper (pdf).

UPDATE, Jan 18th 2007: TagMaps is now live - see this post or go directly to the TagMaps page!

2006

• Alexander Jaffe, Mor Naaman, Tamir Tassa, Marc Davis. Generating Summaries and Visualization for Large Collections of Geo-Referenced Photographs. In proceedings The 8th ACM SIGMM international workshop on Multimedia information retrieval (MIR ‘06), Santa Barbara, CA, USA, 2006.
• Ryan Shaw, Patrick Schmitz. Community Annotation and Remix:
  a Research Platform and Pilot Deployment In proceedings The 8th ACM SIGMM international workshop on Human-Centered Multimedia (HCM2006 Workshop), Santa Barbara, CA, USA, 2006.
• (demo) Patrick Schmitz, Peter Shafton, Ryan Shaw, Samantha Tripodi, Brian Williams, Jeannie Yang. International Remix: Video Editing for the Web. The 8th ACM SIGMM international, Santa Barbara, CA, USA 2006. Flash presentation
• (poster) Shane Ahern, Marc Davis, Simon King, Mor Naaman, Rahul Nair. Reliable, User-Contributed GSM Cell-Tower Positioning Using Context-Aware Photos. In The Eigth International Conference on Ubiquitous Computing (Ubicomp ‘06), September 2006, Orange Country, CA, USA. poster
  
• (poster) Shane Ahern, Nathan Good, Simon King, Mor Naaman, Rahul Nair. Privacy Decisions for Location-Tagged Media. In The Eigth International Conference on Ubiquitous Computing (Ubicomp ‘06), September 2006, Orange Country, CA, USA. poster
• Cameron Marlow, Mor Naaman, danah boyd, Marc Davis. HT06, Tagging Paper, Taxonomy, Flickr, Academic Article, ToRead. In Proceedings of the Seventeenth ACM conference on Hypertext and Hypermedia (HYPERTEXT ‘06), Aug 2006, Odense, Denmark.
  
• (poster) Alexander Jaffe, Mor Naaman, Tamir Tassa, Marc Davis. Generating Summaries for Large Collections of Geo-Referenced Photographs. In The Fiftheenth International World-Wide Web Conference (WWW 2006), May 2006, Edinburgh, Scotland. poster

ZoneTag v2.0 released

It’s been months since we released ZoneTag v1.0, so this is very exciting (at least for us, that is).

For those of you new to ZoneTag, it is a mobile Flickr photo uploader that goes beyond just uploading your cameraphone photos to Flickr. ZoneTag helps you tag (and therefore find, share and organize) your photos through the use of location context. As a research prototype, ZoneTag is a first step in attempting to generate innovative experiences that can help you create, store, find, share and discover photos: yours, your friends’, and the world’s. Read here for a first-hand user account (no bias).

ZoneTag v2.0 takes a small evolutionary step and moves beyond cameraphone photos: it can now help tag any digital photos you take. Using the location-logging feature of the ZoneTag client, you can use your phone to create a log of your location history. This log can then be used by ZoneTag to add location data to photos that were taken with another camera. In other words, you can use your digital camera and have ZoneTag location-tag those photos using your location logs. And here’s a side benefit, or feature, specifically requested by our users. If you’re strapped for money (or, say, if your cheap-ass company doesn’t pay for your data plan), you can save on phone data costs. Do that by uploading your cameraphone photos separately (say, via the good old PC - we’re pretty sure some of you still own one) and “ZoneTag them” later using the log. This way, you won’t lose any location context information.

Besides location logging, ZoneTag v2.0 makes it easier for you to teach ZoneTag your location information. In previous versions, ZoneTag had no way of entering location on the phone. In v2.0, ZoneTag introduces an easy dialogue interface for you to type in the city name or zip code of your location. On the backend, we utilize an array of various location resources at our power to figure out that you’re in Dublin, California and not Dublin, Ireland - or maybe you are in Dublin, Ireland?

And of course, ZoneTag v2.0 keeps all the goodness of v1.0 like Matching tags from your own custom streams, Action Tags, Upcoming.org integration, ZoneTag photo browser… Join the ZoneTag group to get the latest news and updates.

Happy ZoneTagging!

Mix it up!

Late this past April, we launched the International Remixer. It was an especially rewarding project for us because it went beyond just a research prototype. The number of remixes submitted by people showed that remixing is fun. In September, Yahoo! Korea launched a localized version of the Remixer and showed that the idea of remixing transcended language barriers.

We believe that incredibly easy video editing tools like the Remixer are reshaping how we think about video programming as it transforms thousands of video viewers into video creators and enables them to share new stories remixed from the media. But, don’t just take our word for it. We migrated the International Remixer (and its great content!) onto our site for all of you to play with. So, take a spin and let us know what you think about remixing. You can also find the link under “Prototypes” in our sidebar. Happy remixing!