The October 2022 issue of IEEE Spectrum is here!
IEEE websites place cookies on your device to give you the best user experience. By using our websites, you agree to the placement of these cookies. To learn more, read our Privacy Policy.
This robust research platform is still humble enough to teleoperate
The Florida Institute for Human & Machine Cognition (IHMC) is well known in bipedal robotics circles for teaching very complex humanoid robots to walk. Since 2015, IHMC has been home to a Boston Dynamics Atlas (the DRC version) as well as a NASA Valkyrie, and significant progress has been made on advancing these platforms toward reliable mobility and manipulation. But fundamentally, we’re talking about some very old hardware here. And there just aren’t a lot of good replacement options (available to researchers, anyway) when it comes to humanoids with human-comparable strength, speed, and flexibility.
Several years ago, IHMC decided that it was high time to build its own robot from scratch, and in 2019, we saw some very cool plastic concepts of Nadia—a humanoid designed from the ground up to perform useful tasks at human speed in human environments. After 16 (!) experimental plastic versions, Nadia is now a real robot, and it already looks pretty impressive.
For a long time, we’ve been pushing up against the limits of the hardware with DRC Atlas and Valkyrie. We haven’t been able to fully explore the limits of our software and our controls. With Nadia, it’s the other way around, which is really exciting from a research perspective. —Robert Griffin, IHMC
Designed to be essentially the next generation of the DRC Atlas and Valkyrie, Nadia is faster, more flexible, and robust enough to make an excellent research platform. It’s a hybrid of electric and hydraulic actuators: 7 degrees-of-freedom (DoF) electric arms and a 3 DoF electric pelvis, coupled with a 2 DoF hydraulic torso and 5 DoF hydraulic legs. The hydraulics are integrated smart actuators, which we’ve covered in the past. Nadia’s joints have been arranged to maximize range of motion, meaning that it has a dense manipulation workspace in front of itself (where it really matters) as well as highly mobile legs. Carbon fiber shells covering most of the robot allow for safe contact with the environment.
Teleoperation through immersive virtual reality helps Nadia perform complex tasks.IHMC
That large manipulation workspace is particularly relevant because there’s a lot of emphasis on immersive virtual reality teleoperation with Nadia. Not that autonomy isn’t important, but the concept behind Nadia has been shaped (at least in part) by IHMC’s experience at the DARPA Robotics Challenge Finals: Ideally, you want the robot to manage everything it can on its own, while having a human able to take more direct control over complex tasks. While it’s not the fantasy of a robot being fully autonomous, the fact is that in the near term, this approach is an immediately viable way of getting robots to reliably do useful things in unstructured environments. Overall, the goal with Nadia is to operate as close to human speed as possible when doing urban exploration and manipulation tasks. And if that involves a human directly teleoperating the robot because that’s the best way of doing things, Nadia is designed to handle it.
Nadia demonstrates how much more flexible its hips and legs are relative to DRC Atlas.IHMC
For more details about Nadia, we spoke with Robert Griffin, a research scientist at IHMC.
How is Nadia unique among humanoid robots?
Robert Griffin: I think the big thing that we’re trying to bring to the table with Nadia is the really high range of motion of a lot of the joints. And it’s not just the range of motion that differentiates Nadia from many other humanoid robots out there, it’s also speed and power. Nadia has much better power-to-weight than the DRC Atlas, making it significantly faster, which improves its general operational speed as well as its stability.
Can you put your goals with Nadia into context in the humanoid robotics space?
Griffin: We’re trying to make Nadia a robot that can function in unstructured environments where you don’t know what it needs to do ahead of time. So, instead of having a set sequence or number of tasks, we’re trying to build up a robot in hardware, software, and interfaces that enable a human-machine team to go into an unknown environment, explore it, map it, and then do meaningful tasks.
IHMC has a reputation for helping complex humanoid robots like DRC Atlas and Valkyrie with their mobility. How has all of that experience made its way into Nadia’s design?
Griffin: You’re right, and we’ve been partnering with NASA JSC since the DRC—when you see Valkyrie walking now, it’s using IHMC software and controllers. When it comes to applying some of that knowledge to Nadia, we’ve paid special attention to things like Nadia’s range of motion in its legs. Unlike a lot of humanoid platforms, Nadia has really good hip roll (or ab/adduction), which means that it can do really wide steps and also really narrow steps where the feet overlap each other. We think that’s going to help increase mobility by helping with step adjustment and reaching challenging footholds.
Beyond the hardware, we were also able to use our simulation tools and control stack to really help us specify what the hardware design needed to be in order to accomplish some of these tasks. We weren’t able to optimize all of the joints on the robot—we pretty much stuck with a single hydraulic piston in the legs, for now, but we hope that our future designs will be better integrated and optimized. We do have full ankle actuation, unlike a lot of humanoid robots. Our approach, from the beginning, was to look for the necessary control authority with Nadia’s feet to be able to do precise foot placement, and use that for relatively quick reactive mobility.
Robert Griffin gives Nadia a helpful shove, for science.IHMC
As you went from concept to hardware with Nadia, what compromises did you have to make?
Griffin: We’re not quite as flexible as a typical human, although we’re not very far off. I was surprised that we lost some range of motion in the mechanisms just because of the hydraulic hose routing, especially around the hips, where we wanted to avoid having big loops of hoses hanging off the robot. We eventually want to take Nadia into the real world, and we don’t want the hoses getting snagged on things.
Nadia is not quite as fast as a human in some of the joints, and in some of the joints it’s faster. We’re also not quite as compact as we wanted to be originally, because, well, robots are really hard. We’re hoping to get it a little more compact in the future. And we missed our weight margin by a little bit—it’s about 90 kilograms right now, without the hydraulic pump or battery on board.
Why did you decide to focus on how a human can directly control Nadia in VR, as opposed to more emphasis on autonomy?
Griffin: Autonomy works really, really well when you know what the operating conditions are, and what you’re trying to accomplish. Not that it’s easy to automate that, but it’s feasible. We’ve experimented with some autonomous planning and manipulation, and we found that it’s a little limiting in that it doesn’t take advantage of the full capabilities of our platform. And so what we’re looking at doing is trying to automate just key parts of the system—things like footstep planning and grasp planning, which can be automated pretty well. But we also need the ability to interact with novel environments, which is why we’re focusing on developing ways to team with humans through interfaces like virtual reality. With the cognitive capabilities of a human combined with the manipulation capabilities of the robot, we believe you get a ton of versatility.
How much of a difference has it made to controlling Nadia in VR, that the hardware has humanlike flexibility and a humanlike workspace?
Griffin: For manipulation, it’s been huge. We don’t want the user to be operating the robot and feeling like they’re limited in what they can do because of the robot’s constraints. A lot of the manipulation that we’ve been able to do with Nadia we could never have done with our other platforms.
There are definitely trade-offs—the arms are more designed for speed than for strength, which is going to be a little bit limiting, but we’re hoping to improve that long-term. I think that designing the robot—and especially the control system and software architecture—with the idea of having a human operating it has opened up a lot of capability that we hope will let us rapidly iterate along many different paths.
What are some things that Nadia will be uniquely capable of?
Griffin: We’re hoping that the robot will be uniquely capable of multicontact locomotion—that’s a direction that we’re really hoping to move in, where we’re not just relying on our feet to move through the world. But I don’t necessarily want to say that it’ll be better at that than other robots are, because I don’t know everything that other robots can do.
Are there practical tasks that you hope Nadia will be able to do at some point in the future?
Griffin: Things like disaster response, nuclear remediation, bomb disposal—these tasks are largely done by people right now, because we don’t have robots that are actually capable of doing them adequately despite the amount of time that roboticists have been working on it. I think that by partnering excellent robotic engineering with really incredible operating interfaces and human cognition, these tasks will start to become feasible to do by a robot.
Who needs a real head when you have googly eyes and a shark mouth?IHMC
Even though Nadia is brand new, IHMC already has a bunch of upgrades planned for the robot. The most obvious one is cutting that tether by putting batteries and a custom hydraulic pump into Nadia’s torso. Also on the road map are higher torque arms and lighter, more streamlined legs. In the meantime, the software has a lot of catching up to do to allow Nadia to reach its full potential. And if we know IHMC, that’s going to be an incredibly exciting thing to see.
Evan Ackerman is a senior editor at IEEE Spectrum. Since 2007, he has written over 6,000 articles on robotics and technology. He has a degree in Martian geology and is excellent at playing bagpipes.
Atlas, Tesla Bot, and now Nadia... Seems like there's been a push for more R&D for humanoid robots, despite their limitations.
The prosthetics industry is too focused on high-tech limbs that are complicated, costly, and often impractical
The author, Britt Young, holding her Ottobock bebionic bionic arm.
In Jules Verne’s 1865 novel From the Earth to the Moon, members of the fictitious Baltimore Gun Club, all disabled Civil War veterans, restlessly search for a new enemy to conquer. They had spent the war innovating new, deadlier weaponry. By the war’s end, with “not quite one arm between four persons, and exactly two legs between six,” these self-taught amputee-weaponsmiths decide to repurpose their skills toward a new projectile: a rocket ship.
The story of the Baltimore Gun Club propelling themselves to the moon is about the extraordinary masculine power of the veteran, who doesn’t simply “overcome” his disability; he derives power and ambition from it. Their “crutches, wooden legs, artificial arms, steel hooks, caoutchouc [rubber] jaws, silver craniums [and] platinum noses” don’t play leading roles in their personalities—they are merely tools on their bodies. These piecemeal men are unlikely crusaders of invention with an even more unlikely mission. And yet who better to design the next great leap in technology than men remade by technology themselves?
As Verne understood, the U.S. Civil War (during which 60,000 amputations were performed) inaugurated the modern prosthetics era in the United States, thanks to federal funding and a wave of design patents filed by entrepreneurial prosthetists. The two World Wars solidified the for-profit prosthetics industry in both the United States and Western Europe, and the ongoing War on Terror helped catapult it into a US $6 billion dollar industry across the globe. This recent investment is not, however, a result of a disproportionately large number of amputations in military conflict: Around 1,500 U.S. soldiers and 300 British soldiers lost limbs in Iraq and Afghanistan. Limb loss in the general population dwarfs those figures. In the United States alone, more than 2 million people live with limb loss, with 185,000 people receiving amputations every year. A much smaller subset—between 1,500 to 4,500 children each year—are born with limb differences or absences, myself included.
Today, the people who design prostheses tend to be well-intentioned engineers rather than amputees themselves. The fleshy stumps of the world act as repositories for these designers’ dreams of a high-tech, superhuman future. I know this because throughout my life I have been fitted with some of the most cutting-edge prosthetic devices on the market. After being born missing my left forearm, I was one of the first cohorts of infants in the United States to be fitted with a myoelectric prosthetic hand, an electronic device controlled by the wearer’s muscles tensing against sensors inside the prosthetic socket. Since then, I have donned a variety of prosthetic hands, each of them striving toward perfect fidelity of the human hand—sometimes at a cost of aesthetics, sometimes a cost of functionality, but always designed to mimic and replace what was missing.
In my lifetime, myoelectric hands have evolved from clawlike constructs to multigrip, programmable, anatomically accurate facsimiles of the human hand, most costing tens of thousands of dollars. Reporters can’t get enough of these sophisticated, multigrasping “bionic” hands with lifelike silicone skins and organic movements, the unspoken promise being that disability will soon vanish and any lost limb or organ will be replaced with an equally capable replica. Prosthetic-hand innovation is treated like a high-stakes competition to see what is technologically possible. Tyler Hayes, CEO of the prosthetics startup Atom Limbs, put it this way in a WeFunder video that helped raise $7.2 million from investors: “Every moonshot in history has started with a fair amount of crazy in it, from electricity to space travel, and Atom Limbs is no different.”
We are caught in a bionic-hand arms race. But are we making real progress? It’s time to ask who prostheses are really for, and what we hope they will actually accomplish. Each new multigrasping bionic hand tends to be more sophisticated but also more expensive than the last and less likely to be covered (even in part) by insurance. And as recent research concludes, much simpler and far less expensive prosthetic devices can perform many tasks equally well, and the fancy bionic hands, despite all of their electronic options, are rarely used for grasping.
Activity arms, such as this one manufactured by prosthetics firm Arm Dynamics, are less expensive and more durable than bionic prostheses. The attachment from prosthetic-device company Texas Assistive Devices rated for very heavy weights, allowing the author to perform exercises that would be risky or impossible with her much more expensive bebionic arm.Gabriela Hasbun; Makeup: Maria Nguyen for MAC cosmetics; Hair: Joan Laqui for Living Proof
In recent decades, the overwhelming focus of research into and development of new artificial hands has been on perfecting different types of grasps. Many of the most expensive hands on the market differentiate themselves by the number and variety of selectable prehensile grips. My own media darling of a hand, the bebionic from Ottobock, which I received in 2018, has a fist-shaped power grip, pinching grips, and one very specific mode with thumb on top of index finger for politely handing over a credit card. My 21st-century myoelectric hand seemed remarkable—until I tried using it for some routine tasks, where it proved to be more cumbersome and time consuming than if I had simply left it on the couch. I couldn’t use it to pull a door shut, for example, a task I can do with my stump. And without the extremely expensive addition of a powered wrist, I couldn’t pour oatmeal from a pot into a bowl. Performing tasks the cool bionic way, even though it mimicked having two hands, wasn’t obviously better than doing things my way, sometimes with the help of my legs and feet.
When I first spoke with Ad Spiers, lecturer in robotics and machine learning at Imperial College London, it was late at night in his office, but he was still animated about robotic hands—the current focus of his research. Spiers says the anthropomorphic robotic hand is inescapable, from the reality of today’s prosthetics to the fantasy of sci-fi and anime. “In one of my first lectures here, I showed clips of movies and cartoons and how cool filmmakers make robot hands look,” Spiers says. “In the anime Gundam, there are so many close-ups of gigantic robot hands grabbing things like massive guns. But why does it need to be a human hand? Why doesn’t the robot just have a gun for a hand?”
It’s time to ask who prostheses are really for, and what we hope they will actually accomplish.
Spiers believes that prosthetic developers are too caught up in form over function. But he has talked to enough of them to know they don’t share his point of view: “I get the feeling that people love the idea of humans being great, and that hands are what make humans quite unique.” Nearly every university robotics department Spiers visits has an anthropomorphic robot hand in development. “This is what the future looks like,” he says, and he sounds a little exasperated. “But there are often better ways.”
The vast majority of people who use a prosthetic limb are unilateral amputees—people with amputations that affect only one side of the body—and they virtually always use their dominant “fleshy” hand for delicate tasks such as picking up a cup. Both unilateral and bilateral amputees also get help from their torsos, their feet, and other objects in their environment; rarely are tasks performed by a prosthesis alone. And yet, the common clinical evaluations to determine the success of a prosthetic are based on using only the prosthetic, without the help of other body parts. Such evaluations seem designed to demonstrate what the prosthetic hand can do rather than to determine how useful it actually is in the daily life of its user. Disabled people are still not the arbiters of prosthetic standards; we are still not at the heart of design.
The Hosmer Hook [left], originally designed in 1920, is the terminal device on a body-powered design that is still used today. A hammer attachment [right] may be more effective than a gripping attachment when hammering nails into wood.Left: John Prieto/The Denver Post/Getty Images; Right: Hulton-Deutsch Collection/Corbis/Getty Images
To find out how prosthetic users live with their devices, Spiers led a study that used cameras worn on participants’ heads to record the daily actions of eight people with unilateral amputations or congenital limb differences. The study, published last year in IEEE Transactions on Medical Robotics and Bionics, included several varieties of myoelectric hands as well as body-powered systems, which use movements of the shoulder, chest, and upper arm transferred through a cable to mechanically operate a gripper at the end of a prosthesis. The research was conducted while Spiers was a research scientist at Yale University’s GRAB Lab, headed by Aaron Dollar. In addition to Dollar, he worked closely with grad student Jillian Cochran, who coauthored the study.
Watching raw footage from the study, I felt both sadness and camaraderie with the anonymous prosthesis users. The clips show the clumsiness, miscalculations, and accidental drops that are familiar to even very experienced prosthetic-hand users. Often, the prosthesis simply helps brace an object against the body to be handled by the other hand. Also apparent was how much time people spent preparing their myoelectric prostheses to carry out a task—it frequently took several extra seconds to manually or electronically rotate the wrists of their devices, line up the object to grab it just right, and work out the grip approach.The participant who hung a bottle of disinfectant spray on their “hook” hand while wiping down a kitchen counter seemed to be the one who had it all figured out.
In the study, prosthetic devices were used on average for only 19 percent of all recorded manipulations. In general, prostheses were employed in mostly nonprehensile actions, with the other, “intact” hand doing most of the grasping. The study highlighted big differences in usage between those with nonelectric, body-powered prosthetics and those with myoelectric prosthetics. For body-powered prosthetic users whose amputation was below the elbow, nearly 80 percent of prosthesis usage was nongrasping movement—pushing, pressing, pulling, hanging, and stabilizing. For myoelectric users, the device was used for grasping just 40 percent of the time.
More tellingly, body-powered users with nonelectric grippers or split hooks spent significantly less time performing tasks than did users with more complex prosthetic devices. Spiers and his team noted the fluidity and speed with which the former went about doing tasks in their homes. They were able to use their artificial hands almost instantaneously and even experience direct haptic feedback through the cable that drives such systems. The research also revealed little difference in use between myoelectric single-grasp devices and fancier myoelectric multiarticulated, multigrasp hands—except that users tended to avoid hanging objects from their multigrasp hands, seemingly out of fear of breaking them.
“We got the feeling that people with multigrasp myoelectric hands were quite tentative about their use,” says Spiers. It’s no wonder, since most myoelectric hands are priced over $20,000, are rarely approved by insurance, require frequent professional support to change grip patterns and other settings, and have costly and protracted repair processes. As prosthetic technologies become more complex and proprietary, the long-term serviceability is an increasing concern. Ideally, the device should be easily fixable by the user. And yet some prosthetic startups are pitching a subscription model, in which users continue to pay for access to repairs and support.
Despite the conclusions of his study, Spiers says the vast majority of prosthetics R&D remains focused on refining the grasping modes of expensive, high-tech bionic hands. Even beyond prosthetics, he says, manipulation studies in nonhuman primate research and robotics are overwhelmingly concerned with grasping: “Anything that isn’t grasping is just thrown away.”
TRS makes a wide variety of body-powered prosthetic attachments for different hobbies and sports. Each attachment is specialized for a particular task, and they can be easily swapped for a variety of activities. Fillauer TRS
If we’ve decided that what makes us human is our hands, and what makes the hand unique is its ability to grasp, then the only prosthetic blueprint we have is the one attached to most people’s wrists. Yet the pursuit of the ultimate five-digit grasp isn’t necessarily the logical next step. In fact, history suggests that people haven’t always been fixated on perfectly re-creating the human hand.
As recounted in the 2001 essay collection Writing on Hands: Memory and Knowledge in Early Modern Europe, ideas about the hand evolved over the centuries. “The soul is like the hand; for the hand is the instrument of instruments,” Aristotle wrote in De Anima. He reasoned that humanity was deliberately endowed with the agile and prehensile hand because only our uniquely intelligent brains could make use of it—not as a mere utensil but a tool for apprehensio, or “grasping,” the world, literally and figuratively.
More than 1,000 years later, Aristotle’s ideas resonated with artists and thinkers of the Renaissance. For Leonardo da Vinci, the hand was the brain’s mediator with the world, and he went to exceptional lengths in his dissections and illustrations of the human hand to understand its principal components. His meticulous studies of the tendons and muscles of the forearm and hand led him to conclude that “although human ingenuity makes various inventions…it will never discover inventions more beautiful, more fitting or more direct than nature, because in her inventions nothing is lacking and nothing is superfluous.”
Da Vinci’s illustrations precipitated a wave of interest in human anatomy. Yet for all of the studious rendering of the human hand by European masters, the hand was regarded more as an inspiration than as an object to be replicated by mere mortals. In fact, it was widely accepted that the intricacies of the human hand evidenced divine design. No machine, declared the Christian philosopher William Paley, is “more artificial, or more evidently so” than the flexors of the hand, suggesting deliberate design by God.
Performing tasks the cool bionic way, even though it mimicked having two hands, wasn’t obviously better than doing things my way, sometimes with the help of my legs and feet.
By the mid-1700s, with the Industrial Revolution in the global north, a more mechanistic view of the world began to emerge, and the line between living things and machines began to blur. In her 2003 article “ Eighteenth-Century Wetware,” Jessica Riskin, professor of history at Stanford University, writes, “The period between the 1730s and the 1790s was one of simulation, in which mechanicians tried earnestly to collapse the gap between animate and artificial machinery.” This period saw significant changes in the design of prosthetic limbs. While mechanical prostheses of the 16th century were weighed down with iron and springs, a 1732 body-powered prosthesis used a pulley system to flex a hand made of lightweight copper. By the late 18th century, metal was being replaced with leather, parchment, and cork—softer materials that mimicked the stuff of life.
The techno-optimism of the early 20th century brought about another change in prosthetic design, says Wolf Schweitzer, a forensic pathologist at the Zurich Institute of Forensic Medicine and an amputee. He owns a wide variety of contemporary prosthetic arms and has the necessary experience to test them. He notes that anatomically correct prosthetic hands have been carved and forged for the better part of 2,000 years. And yet, he says, the 20th century’s body-powered split hook is “more modern,” its design more willing to break the mold of the human hand.
“The body powered arm—in terms of its symbolism—(still) expresses the man-machine symbolism of an industrial society of the 1920s,” writes Schweitzer in his prosthetic arm blog, “when man was to function as clockwork cogwheel on production lines or in agriculture.” In the original 1920s design of the Hosmer Hook, a loop inside the hook was placed just for tying shoes and another just for holding cigarettes. Those designs, Ad Spiers told me, were “incredibly functional, function over form. All pieces served a specific purpose.”
Schweitzer believes that as the need for manual labor decreased over the 20th century, prostheses that were high-functioning but not naturalistic were eclipsed by a new high-tech vision of the future: “bionic” hands. In 2006, the U.S. Defense Advanced Research Projects Agency launched Revolutionizing Prosthetics, a research initiative to develop the next generation of prosthetic arms with “near-natural” control. The $100 million program produced two multi-articulating prosthetic arms (one for research and another that costs over $50,000). More importantly, it influenced the creation of other similar prosthetics, establishing the bionic hand—as the military imagined it—as the holy grail in prosthetics. Today, the multigrasp bionic hand is hegemonic, a symbol of cyborg wholeness.
And yet some prosthetic developers are pursuing a different vision. TRS, based in Boulder, Colo., is one of the few manufacturers of activity-specific prosthetic attachments, which are often more durable and more financially accessible than robotic prosthetics. These plastic and silicone attachments, which include a squishy mushroom-shaped device for push-ups, a ratcheting clamp for lifting heavy weights, and a concave fin for swimming, have helped me experience the greatest functionality I have ever gotten out of a prosthetic arm.
Such low-tech activity prostheses and body-powered prostheses perform astonishingly well, for a tiny fraction of the cost of bionic hands. They don’t look or act like human hands, and they function all the better for it. According to Schweitzer, body-powered prostheses are regularly dismissed by engineers as “arcane” or derisively called “Captain Hook.” Future bionic shoulders and elbows may make a huge difference in the lives of people missing a limb up to their shoulder, assuming those devices can be made robust and affordable. But for Schweitzer and a large percentage of users dissatisfied with their myoelectric prosthesis, the prosthetic industry has yet to provide anything fundamentally better or cheaper than body-powered prostheses.
Bionic hands seek to make disabled people “whole,” to have us participate in a world that is culturally two-handed. But it’s more important that we get to live the lives we want, with access to the tools we need, than it is to make us look like everyone else. While many limb-different people have used bionic hands to interact with the world and express themselves, the centuries-long effort to perfect the bionic hand rarely centers on our lived experiences and what we want to do in our lives.
We’ve been promised a breakthrough in prosthetic technology for the better part of 100 years now. I’m reminded of the scientific excitement around lab-grown meat, which seems simultaneously like an explosive shift and a sign of intellectual capitulation, in which political and cultural change is passed over in favor of a technological fix. With the cast of characters in the world of prosthetics—doctors, insurance companies, engineers, prosthetists, and the military—playing the same roles they have for decades, it’s nearly impossible to produce something truly revolutionary.
In the meantime, this metaphorical race to the moon is a mission that has forgotten its original concern: helping disabled people acquire and use the tools they want. There are inexpensive, accessible, low-tech prosthetics that are available right now and that need investments in innovation to further bring down costs and improve functionality. And in the United States at least, there is a broken insurance system that needs fixing. Releasing ourselves from the bionic-hand arms race can open up the possibilities of more functional designs that are more useful and affordable, and might help us bring our prosthetic aspirations back down to earth.
This article appears in the October 2022 print issue.