HAEMOBOT complexity   overview

HaemoBot required enormous complexity to enable full, autonomous IV insertion. The following list
gives a broad overview of this complexity:

• 19 mechanical DOFs.
  • NeedleBot had 9 mechanical DOFs:  Needle X, Y, Z, roll, pitch, and yaw + catheter insertion and two grab-and-release DOFs.
  • AssistoBot had 8 mechanical DOFs:  X, Y, Z, blood-cuff inflation + deflation, alcohol spraying, air spraying, and hand restraint.
  • Stereo camera had 2 mechanical DOFs for spinning the laser texture projectors.
• 23 optical DOFs:
  • 20 LEDs in grid (8-bit control each), 1 laser on Needle Driver, and 2 lasers on camera.
• 1,674 separate part designs:
  • 27% of parts designed by us for custom fabrication: 313 laser-cut, 94 machined (separate designs),
  • 38 3D-printed, 4 CNC sheet-metal bending, and 1 vacuum-formed.
  • Remainder of parts from commercial vendors such as McMaster-Carr, Misumi, and Stock Drive.
• 13,306 parts total, including duplicates (like the same screw installed 100 times).
• 4 Data Acquisition Cards, 6 custom PCBs, and 8 Atmega microprocessor/motor-controller boards.
• 20,000 lines code: 9,000 lines for vein-finding and 11,000 lines for non-vision robot controls.
• Footprint with patient chair was 2063mm x 1858mm (along patient’s arm axis) x 2105mm tall.
• Weighed 589 kg / 1298 lbf / 0.589 metric ton.

DISSERTATION ABSTRACT

Intravenous (IV) catheterization is a medical procedure wherein a metal needle is used to puncture a vein and introduce a plastic catheter that remains in the vein for the delivery of medicinal fluids. Nearly 250 million IV insertions take place in the United States annually, and 28% of those insertions fail on the first attempt in normal adults, with appreciably higher failure rates in children. Failed insertions commonly cause bruising and pain but can also lead to more serious complications, including long-term nerve damage, local tissue damage, and sclerosis of the veins. There are two main difficulties in IV insertion: locating the veins, which are often difficult to see or feel, and moving the needle precisely into a vein that is a small, movable target.

We present our robotic IV insertion system, HaemoBot, as a possible solution that offers enhanced veindetection sensory abilities and precise movement of the needle. HaemoBot can be teleoperated by a human to increase his or her sensory and motor abilities without ceding intelligent control over the insertion, or it can insert autonomously. We envision our system being used to treat people in remote or hostile locations where a human practitioner could not be physically present or in hospitals to increase the success and through-put of practitioners.

The highest-level, most far-reaching contribution of our work was to answer affirmatively the question of whether a robot can insert an IV autonomously by demonstrating that our system can 1. autonomously locate and calculate a 6D needle pose to insert on a venous bifurcation and 2. autonomously insert an IV into a realistic phantom of a venous bifurcation by detecting the force penetration-event “pop”.

We performed a clinical experiment with instrumented IV needles to record the process and mechanics of actual IV insertions, and this allowed us to determine a complete list of human IV insertion steps that could be translated into robot steps, determine the normative force, speed, and angles at key moments in IV insertions, and prove that a human can pick out the needle and catheter penetration events into the skin and veins from the force data. We further developed three novel explanations for why bifurcations can be easier insertion targets than straight sections of vein.

We proved that a robot can locate a vein and compute a 6D needle pose autonomously. To achieve this, we developed novel Near Infrared (NIR) transillumination and stereo camera hardware capable of generating uniquely accurate, high-resolution, high-contrast, and three-dimensional hand/vein point-clouds and developed novel computer-vision algorithms for locating, characterizing, and tracking venous bifurcations in a 2D image and then generating a 6D needle pose in the 3D point-cloud.

We proved that a robot can autonomously perform all of the complicated 1-handed and 2-handed mechanical steps of IV insertion on realistic hand, arm, and vein geometries (not just rubber tubes) using similar sensory cues, like the force penetration-event “pop”, that we observed with human practitioners. During this process, we developed a patented method for shining a targeting-laser through an IV needle while maintaining sterility and a patent-pending design for a Remote Center of Motion that has many superior fabrication, installation, and performance properties over existing designs.

We distilled the best lessons from our work on HaemoBot into a simple assistive device that provides more-accurate, stereo vein-visualization than currently available, makes IV insertion easier by reducing it from a 2-handed to 1-handed procedure, and theoretically allows for self-insertion. We designed our device to fill unmet clinical needs that we established during our testing of many different commercially-available vein-finding devices.

We addressed an unmet need for realistic but inexpensive hand/vein phantoms for testing HaemoBot. We developed novel methodologies and tools for fabricating an artificial hand/vein phantom that used the NIR vein images and point-clouds from our vision system to copy real patient anatomy almost exactly, including seamless, properly-sized, latex bifurcations that had a realistic force profile and a silicone hand that had accurate anatomy from the macrostructure down to the fingerprints.

Our work is unique from other research in robotic phlebotomy and IV insertion in that we are the first to:

1. Develop the necessary degrees-of-freedom for treating the needle and catheter as two, separate parts,
2. Design a system that allows for either fully-teleoperated or fully-autonomous control of all steps, including the simple ones (such as cleaning the insertion site),
3. Develop a complete step-by-step model of IV insertion and establish normative values for variables like force, insertion speed, and pitch angle at key skin and vein penetration-events,
4. Develop a 2-armed robot that can perform all of the motions that we observed human practitioners performing, including all of the preparatory, assistive, and actual needle-insertion steps,
5. Employ a mechanical Remote Center of Motion as a safer way of setting the needle’s pose,
6. Develop a high-Degree-of-Freedom, NIR-transillumination system and control algorithm that are able to provide uniquely accurate, high-resolution, high-contrast, and 3D vein point-clouds,
7. Employ a higher-Degree-of-Freedom force/torque sensor with non-penetration-detection goals in mind,
8. Target venous bifurcations and to explain technically why this is a useful, safe strategy, and
9. Develop an insertion phantom that includes venous bifurcations.