Skip to main content

This software could soon be captioning your drunk Facebook photos


Two researchers at Idiap Research Institute, Switzerland, have developed an algorithm that can automatically caption photos.

They're not the first to come up with software capable of doing this, but what makes this software superior to existing auto-captioning techniques is that it's less predictive.

"Those other systems propose the first word based on the photo and then use that word to predict subsequent ones," said Pedro Oliveira Pinheiro, one of the project researchers.

Microsoft's own deep learning solution predicts after learning the first word. The Idiap solution instead creates vector representations of images and compares them to a range of possible words.

The whole image is broken into elements - nouns, verbs, prepositions - which are then put together to create the caption.

The technique is quite simple by comparison to others, and it's already got Facebook's attention. The researchers behind the project did an internship at the Zuckerberg's company, and Facebook is said to be drawing on this technology for its auto-captioning artificial intelligence, some of which will be dedicated to helping the visually impaired.

Hugh Langley

Hugh Langley is the ex-News Editor of TechRadar. He had written for many magazines and websites including Business Insider, The Telegraph, IGN, Gizmodo, Entrepreneur Magazine, WIRED (UK), TrustedReviews, Business Insider Australia, Business Insider India, Business Insider Singapore and more.