Imagine you’re in the distant future with an age-old problem: you can’t find your keys. Mark Campbell, Sibley School of Mechanical and Aerospace Engineering, says you might ask a robot for help. The robot could communicate back, searching with you until the keys are recovered.

Campbell’s version of the future doesn’t end there. With keys in hand, you would go to your car but wouldn’t need to drive it. At the interstate, your car might join a train of other autonomous cars moving safely and efficiently to their destinations.

To make elements of this world possible, however, we need intelligent systems, says Campbell. “The analogy is people,” he explains. “What people are very good at is dealing with new things, with uncertainty, or reasoning about things that aren’t clear. That’s all very hard to program.”

Campbell continues, “We’re interested in developing a set of algorithms 算法 that sit inside whatever system—a robot, spacecraft, or car—and enable it to reason and make intelligent decisions.”

Intelligent Cars

With current technology, an autonomous car performs well on basic roads, but in a busy, complicated place like Manhattan, it won’t know what to do, especially if it loses GPS. “It would basically have to pull over 靠边停车, and we’d have to go get it,” says Campbell. “That’s the current state of the art.”

To address this, Campbell and his team are working on algorithms that would enable an autonomous car to anticipate possible outcomes and make decisions. “When people are driving, we have a mental model of the scene around us and how it’s going to change,” he says. At a four-way intersection 十字路口, for example, we would expect another car to slow down, stop, and either turn or go straight. “This is your mental model, and if anything starts to go against that model, you make some different decisions. So the challenge for us is how to encode 编码 that mental model.”

Using data from multiple types of sensors 传感器, Campbell and his team have programmed a car to anticipate 预测 which way another car will turn at an intersection—100 meters before it arrives. “The sensors pick up on these subtle 细微 movements that we wouldn’t notice with our naked eye 肉眼,” he says. “Effectively, it’s a kind of perception 感知, a mental model, anticipating how a scene is going to change.”

The car’s ability to respond could make autonomous driving and the roadways in general more safe, a major goal of the field. “If you look at the number of accidents that happen, so many of those could be prevented,” Campbell says. “And people are becoming more and more distracted 分心.”

Increasing various efficiencies would be another benefit of a large-scale adoption of self-driving cars. “If you’re pulling onto the highway into a kind of train, your gas mileage goes way up because of the aerodynamics 空气动力学,” Campbell says. “Like bikers in the Tour de France环法自行车赛.” Commuters would also gain back that time spent driving.

“So how far will it go?” Campbell says. “I could imagine city officials, looking at all the benefits, making it mandatory 强制 that you’re almost fully autonomous inside city limits. It’s a long way off, but I can imagine it.” The fact that many of Campbell’s recent PhD students now work at car companies speaks to the momentum 势头 behind this dream.

Collaborating 合作 to Build the Ideal Robot, the Human’s Assistant

Robots, fluidly interacting with humans, have been a long-time fantasy of the future, as shown in movies, such as Star Wars and many others, but the reality is much harder to achieve. Campbell and his team can equip robots with all the same sensors that their cars have, but language poses particular challenges.

“Robots deal with numbers, so the algorithms have to transform those numbers into something that’s an intermediate 中间的 to my spoken information,” Campbell says. “We’re figuring out ways of making algorithms that connect the two.”

This includes programming a robot to understand not just human commands but also communicated information—enabling natural, two-way exchanges. “We want eventually to think about humans working very intimately 亲密的 with robots, rather than separately, where the robots do one set of things and people do another,” Campbell says.

This collaboration might become invaluable 无价 in search-and-rescue missions, or communicating with an unmanned spacecraft, or in the home. “If you’re going to have a robot near your family, you want to be able to work with it naturally and feel safe,” Campbell says.

In collaboration with Hadas Kress-Gazit, Sibley School of Mechanical and Aerospace Engineering, Campbell’s group is working to get a human and robot to build a pyramid together. In a broader project funded by the National Science Foundation—with Kress-Gazit, Massachusetts Institute of Technology, and University of Rochester—teams are addressing all the major challenges. They’ll work on creating a natural language between humans and robots, building robot perception with sensory data, and teaching robots tasks and decision-making.

Kickstarting the Autonomous Driving Revolution, the Student Factor

Campbell hasn’t always been so interested in robotics. “I was all aerospace宇宙空间,” he says. As a graduate student, he’d experienced  the exhilaration 高兴 of having a project on the space shuttle and wanted to give that opportunity to his own students.

“We had two opportunities to launch satellites, and it took us four years to develop each project,” Campbell says. “The first time, the shuttle had an accident. The second time, we worked for four years to develop these little cubes that launched on a Dnepr rocket that exploded over Kazakhstan. The students appreciated the experience, but it was hard.”

Still, Campbell was not drawn away from aerospace until a student’s passion led him away. “I was on sabbatical 休假 in Australia, and a very strong PhD student took on a side-project while I was gone,” Campbell says. That side project was the Defense Advanced Research Project Agency (DARPA) Robotics Challenge, sponsored by the Department of Defense 国防部. The task was to develop an autonomous car that could drive 150 miles through the desert.

Campbell’s PhD student, leading a team of undergraduates, didn’t fair too well in the first challenge, but for the next, which offered million dollar development grants 基金, they asked Campbell to serve as adviser. “We wrote the grant together while I was in Sydney. I would work all day, and my graduate student would work all night,” Campbell says. “And we were lucky enough to win one of those grants.”

Cornell’s team went on to be one of only six finishers of the challenge of more than 200 initial teams. The car they programmed drove autonomously on city streets for about 60 miles, navigating驾驶 other competing vehicles as well as cars driven by people. This all happened before the Google Car. “These contests kind of kick-started the autonomous driving revolution,” Campbell says, “and it launched me and my research into this area.

“This is a wonderful thing about academia,” he adds, “that you can pretty much change your career, which makes our jobs really refreshing.”

Some of the frustrations 失败of working in aerospace no longer apply. “The main thing I like about robotics is that I can see it operating and working right in front of me,” Campbell says.

Campbell recalls vividly the first time he and his students enabled a car to drive autonomously. “My students got me to come out at two or three in the morning. They were testing in the Vet school parking lot, and it was the very first time the car could actually drive itself,” he says. “The steering wheel was shaking, but it was driving this snake course at 15 miles per hour.”

His students had planned a surprise—they’d programed the car to go in reverse 反转. “I’ve never driven backwards 反向 at 15 miles per hour—your entire sense is different, but the algorithms didn’t care at all. They just planned a path backwards, and it’s exactly as safe as going forwards. I thought that was just awesome. It was spine-tingling. We had moments like that once a month during development.”

Campbell adds, “I always like to say that the reason I’m doing robotics now is because my students pulled me in that direction, and I’m very happy they did.”