Skip to yearly menu bar Skip to main content


Poster
in
Workshop: Touch Processing: a new Sensing Modality for AI

Blind Robotic Grasp Stability Estimation Based on Tactile Measurements and Natural Language Prompts

Jan-Malte Giannikos · Oliver Kroemer · David Leins · Alexandra Moringen


Abstract:

We design and train a composition of neural network modules that predicts robotic grasp success based on tactile sensor measurements and natural language prompts identifying the object. We use a Franka Emika Panda robot arm equipped with two DIGIT sensors for grasping and language descriptions generated by chatGPT. Our short-term goal is to utilize this approach to improve the accuracy of a grasp stability estimator.The longer-term goal of this work is to enhance haptically driven robot control with language-based context, i.e. task-relevant information which might not be robustly inferred from vision.

Chat is not available.