Facebook's new Suggested Tags feature

This week, my CBC Radio technology column was all about face recognition technology and Facebook’s new “Suggested Tags” feature, announced last Wednesday. It aired on Tuesday, 21 September 2010 on CBC Radio afternoons shows across Canada, and you can listen to one version below, or download the MP3. [audio:http://blip.tv/file/get/Dmisener-20101221_misener_facebooksuggestedtagsfacialrecognition940.mp3]

Suggested Tags 101

Basically, Facebook has announced a new feature that’s part of their photo sharing application.

For a while now, you’ve been able to tag photos with the names of people who are in them. If you upload a photo of me to Facebook, you can tag it “Dan Misener.” That way, when people are browsing through your photos, and they want to know who that incredibly handsome guy standing near the Christmas tree is, they’ll know it’s me.

Now, Facebook is rolling out a system that looks through your photos, and tries to automatically identify the people in them. You feed it a bunch of photos, and it tries to guess who’s who, in order to make it easier to tag your friends. It doesn’t actually do the tagging, but it does try to identify faces to so you can tag people more easily. Thus, “Suggested Tags.”

In other words, Facebook is starting to roll out a very, very large scale face recognition system.

Why would I want this?

On their blog, Facebook says (and I’m paraphrasing here) they’re introducing this feature because their existing tools are tedious (or their users are lazy):

many of you have said tagging photos can be a chore. (Like that time you had to tag your cousin and her fiancé over and over and over again in 64 different pictures of their engagement party, and then go back and tag the guests.)

It’s important to point out that Facebook isn’t the first company to do this kind of face recognition. Other web applications like PhotoTagger can perform this same kind of face recognition. Both iPhoto from Apple and Picasa from Google can do face recognition on your own photo library.

I’ve used the face recognition that’s built into iPhoto, and it’s kind of neat. I can say, “I want to see some photos of my grandmother now,” or, “Show me some photos of my wife.” It can be a novel way to organize and browse through your photos.

And, as you might imagine, there are definitely security and surveillance applications for this kind of face recognition technology. It could be useful in places like airports, or train stations, or other public places. In fact, at the 2001 Super Bowl, this kind of face recognition technology was used to scan and check spectators against a database of known criminals.

The underlying technology

Recognizing faces is something that’s usually pretty easy for humans to do. Most of us can look at our friends and family, and know who they are, based on their faces. But up until pretty recently, that’s been a very tricky task for computers.

But in recent years, a lot of time and money has gone into developing face recognition systems. There are several different methods of computerized face detection, but one common method has to do with what they call “facial landmarks.” A computer can look at an image of you, and measure the distance between your eyes, the width of your nose, the shape of your cheekbones, your jawline, etc. The relationship between all those landmarks makes up your faceprint. In the same way you have a unique fingerprint, you also have a unique faceprint or face pattern.

Once you have a faceprint, you can compare it to an existing database of faces to find likely matches. This is the kind of system Facebook is rolling out.

Privacy concerns

I’ve confimed with Facebook that the default setting for this new feature is “enabled.” So unless you opt out of this, by manually changing your privacy settings, your name will probably appear if your friends upload photos of you to Facebook.

But for me, the biggest concern here has to do with scale. This kind of face recognition works by comparing your faceprint to an existing database of faces. And Facebook has an absolutely enormous database of faces. Two and a half billion photos are uploaded to the site every month. I asked face recognition researcher Adam Harvey for his thoughts, and he told me:

What some people may not realize though is that every time they tag a photo on FaceBook they’re helping to make face-recognition stronger, smarter, and better. Many of the first face-detection programs were built using a database of several thousand photos. Face-recognition algorithms need far less. And FaceBook has billions of source images and billions of tags.

FaceBook’s database of images is probably the best the world for developing better face-recognition technology. If that doesn’t make you worried about your privacy, it should. Because now that FaceBook has introduced it and made it fun and “social”, others, like Google, will soon follow suit. It’s not the Google doesn’t already have great face-recognition technology, they absolutely do, it’s that they know users will freak out when they see how powerful it is.

Personally, I find all of this more than a little bit creepy, in a self-surveillance kind of way. And I think it’s very interesting how Facebook is positioning this new feature. Of course, they’re not calling it an “automated surveillance tool.” Instead, they’re positioning it as a feature that will make tagging your photos easier and more fun.

Canadian roll-out

Like a lot of Facebook features, American users will get the face recognition feature first. It was announced last Wednesday, and I’m told it’s going to be rolling out very slowly over the next few weeks. No word yet on Canadian availability.

My guess is that the slow roll-out has a lot to do with the amount of processing power required to do this. When you’re talking about two and a half-billion photos every month, it’ll take an awful lot of horsepower to perform face recognition on all of them. You don’t just flip a switch to turn on that feature for everyone all at once.

I would encourage all Facebook users to be on the lookout for this feature in the coming weeks and months, especially given that the default setting is “enabled.” Recognize that if you choose to use it, you’ll be helping Facebook build a better face detection algorithm.

According to Adam Harvey, Facebook’s announcement has some positive elements:

The good thing about making this technology transparent and accessible is that we can see what works and what doesn’t. There is nothing worse than knowing you’re being surveilled and not knowing who is doing it, what it looks like, or who is seeing it […] The silver lining is that everyone can see what this technology for what it is. It definitely is not perfect, and there’s a lot of room to exploit its vulnerabilities.

I think if a computer is going to recognize your face, you might as well be aware that a computer is recognizing your face. Plus, when the face-detecting robot overlords finally come to destroy us all, it’ll comforting to know they’ll be accurate.