Mimic Arm (Part 1)

Whether it was born out of a lifetime fascination with lifelike robots, or out of the simple power fantasy of controlling a mech like so many adrenaline-driven films and movies, I've always wanted to build a robot that could mimic my movements. Something about having a machine follow your commands without needing a controller (known as telecontrol) is just thrilling, immediately making you feel like you're in a movie or sci-fi television show. Luckily, though, you don't need to be a Tony Stark or Asimov to put together a mimic bot. And compared to what it would have taken just a few decades ago (think two stories filled entirely with computers), it's actually fairly easy with some modern tools.

Not to say it's that easy. Mimic Arm ended up being my senior design project at the University of Rochester, along with two other engineers Frank Dinoff and Colin Murphy, and it took almost a year to finish. But if you want to put one together yourself, all it'll take is about $300 (more if you want it to be stronger) and a couple hours of assembly. That's about a $700 dollars cheaper for an arm that ended up working better than anything else we'd could find online.

Here's a video of the final thing in action:

Here's some background as to how we did it (and how you can do it too, if you want.)

The Base Concept:

We started the project with the expectation of creating a robot arm capable of imitating the natural movements of a human arm in real time. We designed the arm with 4 joints, to allow it to hit every possible human arm position minus the motions of the wrist, and rigged it to a Kinect camera to track the operating user. When somebody stepped in front of the camera, the system would track to see where their right arm was, port it into our software program, and then command the robot arm to imitate their real arm as they moved it around:

Similar projects had been done independently a couple times before by other enthusiastic makers online, as well as by some very large and well funded research groups, but almost all of them suffered serious flaws: most of the arms we could find were very slow, they had to use bulky exoskeletons mounted on the human operating them, and they just didn't look very natural. Even the big expensive ones suffered from serious latency issues, lagging far behind the motions of their human operator, and the little ones were jagged and erratic in their motion.

Having personally worked with motion tracking cameras and servos before, it seemed possible that these existing arms were suffering from excessive overhead due to software and constraints they really didn't need for humanlike motion. By stripping out unnecessary features that are not required for telecontrol, we figured we could speed up the system substantially.

With that in mind, we established three goals for the arm that would dictate our design choices.

1. Human degrees of freedom
2. Camera based tracking
3. Superior power and speed

Each of these is pretty important, and takes some engineering to achieve, but they're roughly in order of priority, and are listed as such below.

1) Human Degrees of Freedom

The biggest difference between mimic arm and the vast majority of robotic arms that exist today is actually how the mathematics work. For most arms, the most critical part of the system is the position of the "end effector", or the very end of the arm; this is where you mount the hand, or the drill, or the machine tool or whatever else. Since whatever the robot is doing probably depends on how accurately it can use that tool, it's normally critical that the entire arm is designed to move to very predictable positions with extremely high stability, with everything else being secondary design constraints.

But that philosophy doesn't really apply to Mimic Arm. We don't care where the hand is, as long as its appearance roughly mimicking what your real arm is doing. And the user is controlling it in real time, so if they need to adjust the position, they can do it very easily and intuitively. This may seem like a small change, but it dramatically changes how the arm is approached.

For one thing, we treat Degrees Of Freedom (DOF) a lot differently. Degrees Of Freedom just refers to how many joints your robot arm has, ie, how many ways it can transform its own motion. For a robot arm that only cares where it puts its own hand, it's usually more efficient for these to not be the same joints as a human arm, since what works for us usually isn't quite the same as what's needed for the job the robot needs to do. Maybe it doesn't need as many, or needs more, or maybe it needs wider or narrower flexibility to fit the application. In any case, you typically end up with something like this, which does that one robot thing really well but isn't exactly very human.

If the you want to know more about what the diagrams mean (they're called the Denavit-Hartenberg Convention for kinematic drawings), check out this link for a full explanation. Short version, the hourglass looking joints rotate into the page, and the circle ones are rotating across it. 

For mimic arm, it's much more important that it look like a human arm than for it to be precise. So we need to implement DOFs that are more similar to the shoulder of a real person, which has roughly the following setup:

This one isn't quite perfect either, since it clusters a lot of motors up at the shoulder, and it requires a lot of the DOFs drawn out to move in concert to perform some common motions. Even if you spread them out a bit (arm 1 below), the mathematics for this arm are very complex. So we came up with our own model (arm 2 below), which does things a bit more easily by moving one motor to become the bicep. It can still reach every position a human arm can, but needs fewer motors at a time for common movements, and is generally easier to handle.

The mathematics we used would also change from this decision. For regular robot arms, you typically have a destination for the hand which is then met by the motors of the arm. The mathematical operations needed to figure out how to get it there are referred to as Inverse Kinematics, or finding the appropriate arm angles needed to hit a point (or series of points) in three dimensions. Mimic Arm is given points by the Kinect and then must find the proper angles so that it looks like it's moving naturally, and thus uses Forward Kinematics. This actually made our job a little easier, since forward kinematics tend to be easier than inverse, but I'll get back to that in the implementation post.

2) Camera Based Tracking

From the start, we knew we wanted to be using a camera to track users rather than a physical control. An exoskeleton would be uncomfortable, awkward, expensive to build, and we'd have to do all the mathematics of reading it ourselves; on the contrary, the Kinect for Windows SDK does some extremely sophisticated tracking right off the bat, and would be capable of tracking the right arm of any user the moment they stepped in front of the camera without even needing to calibrate. Other bonuses of using camera tracking included my personal hopes of integrating the system with the NullSpace system, something I'm still hoping to do if I find the time...

It's worth noting that we did not originally start with the Kinect, instead opting for a linux-based camera, the Asus Xtion Pro. Our hope was to make the system entirely self contained, without needing a laptop as an intermediary, which could be done with compact linux systems like the BeagleBone Black but not with a Windows operating system. However, we later found out that the entire non-windows software community for motion tracking has been destroyed by an acquisition of the PrimeSense skeletal tracking library by Apple, meaning that the only viable platform for camera tracking is now the Kinect, with the Microsoft SDK, on a windows machine. Not a great scenario, so stay away unless you want to accidentally spend hundreds of dollars on defunct hardware like we did.

Despite not having linux support, the new Kinect camera really is a brilliant machine, doing some significant under-the-hood processing to bring you a fully mapped skeletal model of the human body in real time. The speed and accuracy of the platform meant that we had very little to do on the tracking end other than fetch the numbers that the Kinect's SDK throws at you, though we did end up smoothing out a little bit of residual noise to make the arm's movements less jumpy.

Things the Kinect can't do, though, are tell when your arm is out of sight, and track what your hand is doing. These weren't big issues for us, since we couldn't afford to build an arm powerful enough to support a fully articulated robotic hand (discussed in the next section), and since we weren't expecting users to be twirling around as they moved the arm. They're problems that could be approached by engineers in the future, possibly by using inertial tracking rather than just a camera, but that's a job for another time.

3) Superior Power and Speed

One of the biggest problems that we could identify with a lot of the hobby-type mimic arms that already existed was that the motors they were using were simply too lightweight to perform motions properly. Motors like these offer only a pitiful 50 ounce inches of torque at the max, and can only be controlled through PWM (pulse width modulation), which means they give back no information on whether they've maxed out their torque limit, destroyed their own gearing, or even just where they are physically in terms of angular position.

Making the arm move powerfully and smoothly was one of our biggest concerns from the start, so this needed to be solved in the design phase. Running the kinematic calculations, we were going to want a much higher torque than oz/in at the shoulder to support an arm of 12 inches or more. We had already sacrificed a complex end effector out of concerns for cost - motors powerful enough to lift an arm with the weight of a motorized hand on the end would almost certainly be out of our price range. But making the arm out of cheap materials and at miniaturized size was out of the question. We wanted to see it move, and we wanted users to be able to relate to that movement; that meant a balance between price and more power.

In addition, we wanted more flexibility in terms of what the arm could return to the computer; things like current position, temperature, and load torque, as well as options for emergency shutoff in case the arm got blocked or overheated. Putting all our money and time into an arm that would simply break as soon as it got stressed was the last thing anyone wanted, and we were willing to put up a little more funding upfront to prevent that.

With these conditions in mind, we ended up going with the Dynamixel servo line:

The Dynamixel AX-12s were the perfect mix of power and flexibility we were looking for, with over 200 oz/in of torque and a full onboard microcontroller with settings and feedback for position, speed, motor rotation limits, max torque, and a host of alarm settings that could shut the system off in a pinch. The weren't bad on price either, coming in at just $45 for an order of magnitude more functionality than the simple PWM servos we'd seen before. For the shoulder, though, we wanted even more power, and decided to go with the upgraded MX-64: a $300, 1000 oz/in beast of a servo that would be more than enough to lift our arm by almost twice our safety margin.

The real tradeoff for these motors was in their complexity. PWM may be limited, but it is also simple, and can be implemented on any Arduino microcontroller. These guys, on the other hand, well... they need some extra work. A whole semester's worth of work, as it turned out.

But all that shall be left for another time. Part 2 of this series will address implementation of the system, from technology to coding to completion.

See you then!

 - Lucian

Sentry Gun!

As the second UR Robotics "Lab Collab" we decided to make a sentry gun. This was in no small part due to the grad students who have been invading our lab and stealing computer mice for the past 2 years. Card access for the floor is global, so there's no real way we can track them or prevent them from entering the space, so there's little we can do to keep them from tromping through our workshops to get to their adjacent space whenever they please, nicking office supplies and using the xbox on nights when nobody's there.

To put it plainly, we hate them. And I think we can all agree the best way to deal with something you hate is to shoot it with a bunch of nerf darts.

This project was loosely based on an existing nerf turret instructible made for the Vulcan chain-fed nerf gun, which was in turn based off another tutorial for a paintball gun turret. However, given that the Vulcan dropped out of circulation a while ago and now goes on Amazon for a whopping $400 (and also given that reviews of the gun were never very good anyway) we figured we'd go with a different model for our sentry.

Obviously electric was a must, and flywheels that we could power up, but we were also attracted to the idea of visual intimidation. So we went ahead with the off-brand AirZone Quickfire Gatling Blaster, which has a great big rotating gatling barrel. Not the best reviewed, mostly due to the darts holding poorly in the gun chain, but since we're planning to build a box for them anyway this wasn't a big issue.

prepped for dissection

We had 4 objectives for this gun:

1. Get Arduino control over the internal motors for the flywheels, gatling barrel, and firing mechanism
2. Up the voltage on them up so the gun is more powerful
3. Set up a camera to do automatic facial tracking
4. Set up the turret to let the gun aim both vertically and horizontally

This first post will focus on the gun internals, while upcoming ones will focus on the software, tracking, and turret mount.

Taking the gun apart ended up being more tricky than we anticipated. The gatling end has a ring that kind of holds stuff together, and even once we took it apart the entire front end of the gun was still adhering a bit. Rather than force it and risk the delicate clockwork around the flywheels being disturbed, we opted to simply dremel off the back of the gun to access the electronics - there was nothing we needed to get to in the front of the gun anyway, so why risk it?

Alex uses a dremel tool to cut the paneling off

Once we got the panel off, you can see all the internal circuitry of the gun, including another box which turned out to be a motor casing.

Once you take that out you can see the wire and switch assembly driven by the trigger that dictates which motors in the gun are on at which times. 

We actually opened up this motor casing, but immediately regretted it - it's simply a bunch of lubricated gearing, which is awfully easy to break and awfully hard to put back together correctly. Luckily we got it to fit back together with some coaxing but after that we just left it aside. 

The trigger assembly was an entertaining puzzle. It turns out there are several inputs to the gun. One is not trigger related, but is actually a switch in the loading mechanism (not shown) which detects whether ammunition has been added. The others are all here in the trigger. When the first switch is released, the barrel starts rotating, whereas when the others are pressed, they begin the flywheels and firing mechanism, respectively. It's a good way of making sure everything starts in the right order without needing any real electronics. 

Of course, we don't really care about how it's done with the trigger, since we just need to access it with an Arduino. So we went ahead and disabled the trigger, porting all the wires out the bottom of the gun with some documentation as to what does what. 

The power is pretty useless since we'll just be attaching it to the Arduino motor drivers we bought a while back, but at least we know it's 9V as a reference. We'll probably be upping it to 12 volts to give the gun a little extra kick and inspire greater terror amongst those pesky grads.

Next up is getting the whole thing running in software. See you then.

OWI Arm revamp

We've started work on revamping an old OWI toy arm here in the UR Robotics club laboratory at the University of Rochester. We got the box of Amazon, thinking it'd be a good example of a robotic appendage at low cost to examine for our own designs.
http://www.amazon.com/OWI-OWI-535-Robotic-Arm-Edge/dp/B0017OFRCY

We'd bought the arm for a teardown project because of its low price ($40) but we were disappointed to find that it really contained no electronics whatsoever. The motors and wires are all rigged up to metal contact switches controlled manually, and there isn't even so much as a resistor in the whole thing. This means that not only is the arm very physically weak, can't be programmed, and quickly drains the batteries if used for too long, it'll also tear apart its own gearing if you run it too far in one direction.

But hey, that's what Arduinos are for!

The first objective was to get the arm off of the manual control and into something we could configure. For the first session we didn't have any appropriate drivers on hand and couldn't figure out some of the more obscure ones lying around the robot lab, so rather than give up entirely and wait a week I figured I'd throw together my own H bridge for demonstration purposes. I found a pretty good demonstration here for building one out of power mosfets, which we happened to have a good pile of on hand (IRF9Z34 PMOS and 1D22AB NMOS to be specific, and 2N4401 NPN BJTs).

The critical thing to remember about H bridges is that they're essentially a pair of CMOS inverters, but controlled in opposite polarity from each other. Thus, when you turn one side positive, the other turns negative, and vice versa. There's a pretty good overview here if the concept seems tricky. However, this whole loopy thing only controlled one motor, which is nice to show off a concept but not so much for being actually able to program the arm.

It could waggle up and down, though!

It was pretty evident at this point that we were going to need a more capable motor driver. I ended up going with a set of Arduino motor shields off of Adafruit that had some impressive current driving capability and could also be stacked, an important feature for getting all 5 of the motors on the arm powered.

Once we got these going it was a piece of cake to rig it up to my Uno and give the arm a fair bit of individual articulation. The code is still pretty simple but can be located here:
https://github.com/URRoboticsClub/OWIArmControl

The real issue now is that the arm still has no feedback control. I'll be tackling that next time with some glued on potentiometers and maybe even a force sensor or two. After that, who knows? Chess?

Whatever. Until the next project!