New video analytics software can identify what people are wearing.
The Graymatics company has developed software that can automatically identify specific products in visual media—a pair of Ray-Ban sunglasses in a music video, say, or the Banana Republic shirt your friend is wearing in a holiday photo. As a way to drive online purchases, it could be a revenue booster for both content publishers and content platforms like Facebook and YouTube, which are serving up an exploding number of images and video online.
Online videos, especially, haven't lived up to their potential for driving revenues, even with those annoying ads before a clip. Improving video advertisements is a niche that several young startups, including Graymatics, are looking to fill.
On stage at the Demo startup conference in Santa Clara, California, last week, the company's executives showed off software that can quickly identify items in videos and photos and then match these with the same or a similar product for sale through various online retailers and marketplaces.
As one example, the software matched the sunglasses worn by Brad Pitt and Angelina Jolie in an image accompanying a news article about the movie stars to similar pairs available on Amazon. A reader online who hovered a cursor over the object would see the tagged link for the product.
At least two other companies offer similar product-recognition services, but Graymatics' business development director, Michael Scolari, says they require humans to input some data and can handle only still images, not video. His company's software, which relies on computer-vision and machine-learning techniques developed by researchers in Singapore, is the first to be fully automated, he says.
The software learns by scanning product images available on the Web. It then recognizes objects in images or videos and uses algorithms to break them down into almost two dozen attributes like color, shape, and even texture. Finally, it finds the closest matches in a database of products, such as a feed of Amazon or eBay's stock, or on specific retailers' sites.
Jay Stanley from the ACLU writes:
Graymatics has developed software that can automatically identify specific products in visual media—a pair of Ray-Ban sunglasses in a music video, say, or the Banana Republic shirt your friend is wearing in a holiday photo…. the company's executives showed off software that can quickly identify items in videos and photos and then match these with the same or a similar product for sale through various online retailers and marketplaces.
As one example, the software matched the sunglasses worn by Brad Pitt and Angelina Jolie in an image accompanying a news article about the movie stars to similar pairs available on Amazon. A reader online who hovered a cursor over the object would see the tagged link for the product….
The software learns by scanning product images available on the Web. It then recognizes objects in images or videos and uses algorithms to break them down into almost two dozen attributes like color, shape, and even texture. Finally, it finds the closest matches in a database of products, such as a feed of Amazon or eBay's stock, or on specific retailers' sites.
The technology could be used not just to let us get more information about products, but to allow companies to get more information about us. If the technology proves effective and reliable at detecting the brand of clothing one is wearing, that could reveal a lot about a person. Human beings have been differentiating themselves by clothing for thousands of years; as this marketing document suggests, clothing and product choices reveal things about our gender, age, geography, lifestyle, and behavior. Not only that, but with modern big data techniques, it’s likely that our clothing choices could reveal many more correlations, such as our income, education, ethnicity, and preferences of all kinds.
Who would apply these analyses? Social networks like Facebook possess enormous stores of photographs posted by their users. The technology could also be applied of course to any other photographs appearing online (blogs, Flickr, news stories, etc.). And perhaps even more significantly, it could also be applied to surveillance cameras.
Imagine this scenario: you walk into a furniture store. A surveillance camera captures your image and applies product recognition. Based on the price and brand of your purse, briefcase, pants, jacket, and shoes, a computer decides in a flash that you are most likely white, middle aged, with a medium-low income and education. It makes guesses not only about your buying power but also about your lifestyle and tastes in music and much else. Right or wrong, this information is passed along to salesmen, security guards and managers and determines how you are treated. Face recognition is also applied so that if you return on another day, their product-based profile can be refined based on subsequent outfits. The store might even enter into a partnership with other establishments to capture your image across different contexts on different days to really pin you down. (Or more likely, a third-party company might emerge that will do this.) Soon, the reality of this system begins to sink in and people start to become very aware and self-conscious about what they wear and the products that they carry.
http://www.aclu.org/blog/technology-and-liberty-free-speech-national-security/newest-video-analytics-technique-product
http://www.technologyreview.com/news/429477/whered-you-get-that-cool-shirt-this-software/?utm_campaign=newsletters&utm_source=newsletter-daily-all&utm_medium=email&utm_content=20121008