Human-machine interactions with soft robots.

Technologies using soft, stretchable materials are increasingly important for providing an interface between humans and machines, but actuating, sensing, and controlling soft, stretchable materials is a challenge. Nature, however, demonstrates remarkable control of stretchable surfaces: for example, cephalopods can project hierarchical structures from their skin in milliseconds for a wide range of textural camouflage, run across the ocean floor, change color, and hydrojet at high speeds through the ocean. Inspired by nature, we are developing theoretical and experimental techniques that enable the design, fabrication, and control of soft robotics, with applications towards helping humans.

Shape transformable materials

Inspired by cephalopod muscular morphology, we have developed synthetic tissue groupings that allowed programmable transformation of 2-D stretchable surfaces into target 3-D shapes. The synthetic tissue groupings consisted of elastomeric membranes embedded with inextensible textile mesh that inflated to within 10% of their target shapes using a simple fabrication method and modeling approach. These stretchable surfaces transform from flat sheets to 3-D textures that imitate natural stone and plant shapes and camouflage into their background environments. Figure 1 and the video below explain this approach, taken from ref. [13]

Figure. 1. Inspiration for and description of CCOARSE. A) A conical papilla (ca. 4mm high) in Octopus rubescens that dynamically extends or retracts in ca. 220 milliseconds. This small species has numerous skin papillae that provide exceptional camouflage in shallow kelp habitats in central California. Frame grabs are from video of a live animal (R. Hanlon). B) An inflated silicone membrane showing the principal strains and resulting deformation of points on the undeformed planar membrane. The radial and circumferential strains displace the points vertically and radially, but not along theta. C) An inextensible non-woven mesh embedded in the silicone membrane constrains the circumferential strain, resulting in vertical displacement based on the radial strain. D) Fabrication of the mesh-silicone membrane. Silicone is poured into a mold. Mesh is embedded, laser cut, removed, and then the silicone is cured. A topcoat of silicone is added to fill voids and improve mesh adhesion. E) A tenstile testing specimen section with horizontal layers of mesh and LS / L silcione length fraction. F) Stress-strain measurements for specimens with multiple silicone length fractions. G) Relationship between the composite membrane strain and silicone length fraction taken from (F) at 50, 100, and 150 kPa membrane stress states. This information maps the mesh design in a silicone-mesh composite membrane to a target 3-D shape.

Controlling shape by turning on and off stiffness

Building from the shape transformable surfaces, we are incorporating electroadhesive clutches into inflated elastomer membranes to control the actuator shape. The clutches allow us to electrically turn on and off regions of high stiffness, and the inflated membrane responds by changes shape (Fig. 2, from Conference Paper [13]). We can also use these shape changes to move a ball through a workspace, launch the ball when a clutch is released under pressure, and tilt heavy objects (Fig. 3, from Conference Paper [13]).

Figure 2: A. Actuator expansion formed into three different shapes; shape chosen
by clutch activation. Top-view presented over side-view. B. Expansion
of soft, elastomeric actuator to 3.1 kPa under three different clutch activation
con gurations. C. Soft actuator manipulation of 3.7 g ball. D. Manipulation
of 820 g textbook. From Conference Paper [13]
Figure 3: Mode 1 Actuator Characterization. A. Plot of actuator workspace along 4 DoF, looking at the actuator from a top-down view. Each DoF is accessed
by activating a different set of clutches. Photos of corresponding positioning of 40 mm diameter ball. B. Plot that tracks position of ball once it has been
placed on the actuator. The actuator lifts it to the speci ed direction and then releases the inboard clutch to rapidly apply force to the ball in the desired
direction. Photos of corresponding projectile motion for 40 mm diameter ball. C. Labeled photo of actuator with directions and DoFs. From Conference Paper [13]

Materials with electroprogrammable stiffness

In order to achieve better mechanical control over soft robots, we would like to use electricity to quickly program the local stiffness of a material, and to be able to do this with low energy input and over large changes in stiffness. To do so, we need new advances in materials with electroprogrammable stiffness. We have organized a comprehensive review of the available materials that realize electroprogrammable stiffness in ref. [31], show that all current approaches can be categorized as using electrostatics or electrically-activated phase changes (Fig. 5), and summarizes the advantages, limitations, and applications of these materials. Finally, we identify state-of-the-art trends and an outlook of future opportunities for the development and use of materials with electroprogrammable stiffness.

Figure 4: Programmable stiffness in biology and engineering. Biological systems are capable of significant stiffness changes. Humans routinely increase the stiffness of their muscles to perform tasks. Sea cucumbers increase the stiffness of their dermal layer as a means of protection. Swifts change the stiffness of their wings to dynamically alter aerial efficiency, while elephants can use their trunk in its soft state to conform to delicate objects and stiffen it to carry large loads. Programmable stiffness materials are increasingly used in engineering fields, including robotics, biomedicine, air vehicles, and virtual reality. From ref. [31]
Figure 5: Many stimuli have been used to program stiffness changes in material systems, but materials with electroprogrammable stiffness are easily integrated and enhanced with electronics, and are either driven by electrostatics or an electrically-driven phase transitions. From ref. [31]
Figure 6. A quantitative summary of state-of-the-art materials with electroprogrammable stiffness. Individual data points show the activated and de-activated moduli of a given electroprogrammable material system as a function of their lowest applied voltage (left), shortest response time (right). We also show normalized power consumption per area for electrostatic materials and normalized power consumption per volume for electrically-activated  phase change materials in ref. [31].

Sensing soft and highly deformable surfaces

The most common sensing modalities found in a robot perception system are vision and touch, which together can provide global and highly localized data for manipulation. However, these sensing modalities often fail to adequately capture the behavior of target objects during the critical moments as they transition out of static, controlled contact with an end-effector to dynamic and uncontrolled motion. In recent work published in Conference Paper [11], we present a novel multimodal visuotactile sensor that provides simultaneous visuotactile and proximity depth data. The sensor integrates an RGB camera and air pressure sensor to sense touch with an infrared time-of-flight (ToF) camera to sense proximity by leveraging a selectively transmissive soft membrane to enable the dual sensing modalities. We present the mechanical design, fabrication techniques, algorithm implementations, and evaluation of the sensor’s tactile and proximity modalities. The sensor is demonstrated in three open-loop robotic tasks: approaching and contacting an object, catching, and throwing. The fusion of tactile and proximity data could be used to capture key information about a target object’s transition behavior for sensor-based control in dynamic manipulation.

Figure 7: Proximity depth map from internal depth camera. B. Image
from internal RGB camera for tactile data.. Conference Paper [11]

Publications:

Conference Paper [13] Gregory Campbell, Jessica Yin, Yuyang Song, Umesh Gandhi, Mark Yim, James Pikul, “Electroadhesive Clutches for Programmable Shape Morphing of Soft Actuators”, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, 2022.

Conference Paper [11] Jessica Yin, Gregory Campbell, James Pikul, Mark Yim, “Multimodal Proximity and Visuotactile Sensing With a Selectively Transmissive Soft Membrane”, IEEE-RAS International Conference on Soft Robotics – RoboSoft, Edinburgh, Scotland, 2022. [Best paper award]

[34] Yichao Shi, Zhimin Jiang, and James H. Pikul, “Harnessing Cascading Dominoes for Peristaltic Wave Motion”, IEEE Robotics and Automation Letters, 2022.

[31] David Levine, Kevin Turner, and James H. Pikul, “Materials with electroprogrammable stiffness”, Advanced Materials, 2007952, 2021.

[19] Cameron A. Aubin, Snehashis Choudhury, Rhiannon Jerch, Lynden A. Archer, James H. Pikul, Robert F. Shepherd, “Electrolytic vascular systems for energy-dense robots“, Nature, vol. 571, June 2019.

[17] Steven Ceron, Itai Cohen, Robert F. Shepherd, James H. Pikul, Cindy Harnett, “Fiber Embroidery of Self-Sensing Soft Actuators”, Biomimetics, vol. 3, September 2018.

[15] T.J. Wallin, James Pikul, Robert F. Shepherd, “3D printing of soft robotic systems“, Nature Reviews Materials, May 4, 2018.

[14] Dean Keithly, John Whitehead, Alexandra Voinea, Diego Horna, Samantha Hollenberg, Mason Peck, James Pikul, Robert F. Shepherd, “A cephalopod-inspired combustion powered hydro-jet engine using soft actuators“, Extreme Mechanics Letters, vol 20, April 2018.

[13] James H. Pikul, Shuo Li, Hedan Bai, Roger T. Hanlon, Itai Cohen, Robert F. Shepherd, “Stretchable surfaces with programmable 3-D texture morphing for synthetic camouflaging skins”, Science, vol. 358, pp. 210-214, October 18, 2017.

[12] T.J. Wallin, J.H. Pikul, S. Bodkhe, B.N. Peele, B.C. Mac Murray, D. Therriault, B.W. McEnerney, R.P. Dillon, E.P. Giannelis, R.F. Shepherd, “Click chemistry stereolithography for soft robots that self-heal”, Journal of Materials Chemistry B, vol. 5, pp. 6249-6255, 2017.