Computers developing ‘common sense’ via image analysis

A computer cluster running the so-called the Never Ending Image Learner at Carnegie Mellon University runs 24 hours a day, 7 days a week searching the Internet for images, studying them on its own and building a visual database. The process, scientists say, is giving the computer an increasing amount of common sense.

“Images are the best way to learn visual properties,” said Abhinav Gupta, assistant research professor in Carnegie Mellon’s Robotics Institute. “Images also include a lot of common sense information about the world. People learn this by themselves and, with [this program], we hope that computers will do so as well.”

The computers have been running the program since late July, analysing some three million images. The system has identified 1,500 types of objects in half a million images and 1,200 types of scenes in hundreds of thousands of images, according to the university.

The program has connected the dots to learn 2,500 associations from thousands of instances.

Thanks to advances in computer vision that enable software to identify and label objects found in images and recognise colours, materials and positioning, the Carnegie Mellon cluster gains a better understanding of the visual world with each image it analyses.

The program also is set up to enable a computer to make common sense associations, like buildings are vertical instead of lying on their sides, people eat food, and cars are found on roads. All the things that people take for granted, the computers now are learning without being told.

“People don’t always know how or what to teach computers,” said Abhinav Shrivastava, a robotics Ph.D. student at CMU and a lead researcher on the program. “But humans are good at telling computers when they are wrong.”

While previous computer scientists have tried to “teach” computers about different real-world associations, compiling structured data for them, the job has always been far too vast to tackle successfully. CMU noted that Facebook alone has more than 200 billion images.

The only way for computers to scan enough images to understand the visual world is to let them do it on their own.

“What we have learned in the last five to 10 years of computer vision research is that the more data you have, the better computer vision becomes,” Gupta said.

CMU’s computer learning program is supported by Google and the Office of Naval Research.

Originally published on Computerworld (US). Reprinted with permission from IDG.net. Story copyright 2024 International Data Group. All rights reserved.