Skip to content

DAVATaR: Depth-Aware Visually Augmented Tactile Learning Across the Air-Water Interface

Session: Not recorded. A secondary listing (hongsong-wang.github.io/CoRL2026) marks it as a Spotlight (unverified). Speaker(s): Paper: PDF on the project page (anonymized submission version, 16 pages) by Bowen Jiang*, Siqi Shang*, Rahul Ajay Iyer, Ashwin Prakash, Lillian Chin†, and Roberto Martín-Martín† (UT Austin; * equal contribution, † equal advising). The author list comes from the project page. No arXiv ID yet. The same secondary listing gives OpenReview hSRtOgnkTM (not opened). Project page: davatar-corl26.github.io, with a 3 min overview video. Code and data are not mentioned.

Paper summary (from the PDF text; figures not inspected)

  • Problem: Submerged manipulation means a robot in air reaches into water, for example washing dishes, bathing, or water sampling. This differs from fully underwater robots because the robot crosses the air–water interface over and over.
  • Vision breaks: Refraction shifts and rescales objects, reflection causes glare, and foam, bubbles, or turbidity occlude the scene.
  • Electronics and water don't mix: Conventional tactile sensors put electronics in the wet region.

Method

  • Gripper:
  • Each finger is a 305 mm aluminum H-beam with a 90 mm compliant fingertip printed in EPU40, so the electronics sit far from the water.
  • Contact force: Uses fluidic innervation. Each fingertip has 5 pairs of embedded L-shaped air channels, each pair read by a differential pressure transducer (DLHR-L05D) at the finger root. The differential layout cancels temperature and hydrostatic offsets.
  • Immersion depth: A tube open to the water and a sealed reference tube run along the back of the finger and feed one differential transducer. Hydrostatic pressure gives depth. They turn the "nuisance" of hydrostatic pressure into a signal.
  • In total the gripper gives 11 signals at 20 Hz: 10 force and 1 depth.
  • Submerged-domain augmentations (wrist RGB only, training time only, on top of translation and color jitter):
  • Reflection-like: Translucent bright patches covering about 6% of the image, with random brightness and scattering. Applied with p = 0.3.
  • Refraction-like: Random zoom in U(0.95, 1.05), cropped or padded back to size. Applied with p = 0.3.
  • Policy: Behavior-cloning transformer with an MDN head (5 components). It predicts operational-space end-effector deltas plus a gripper command.
  • Inputs are a history of H = 5 steps. The 256×256 wrist image goes through ResNet-18 with spatial softmax (32 keypoints) to make one token, and the 11-d tactile vector goes through an MLP to make one token. A learned readout token fuses them.
  • It is small: 64-d tokens, 4 layers, and modality dropout of 0.2 on tactile.
  • Trained from 40 SpaceMouse demos per task on a Franka Panda. Evaluation runs at 5 Hz.

Results

  • Sensor characterization:
  • Force RMSE overall is 0.37 N for grip and 0.66 N for push, over a 0–20 N range. Error grows with depth, from 0.11 N for grip in dry conditions to 0.85 N at 20 cm.
  • Depth is near-linear (R² = 0.9996), with 1.6 mm RMSE.
  • Policies: Four tasks with 18 rollouts per method and condition (3 seeds × 6). Demos come only from the normal setting, and the adversarial settings are held out. Each entry is the mean of the normal and adversarial results.
Method Water sampling Spoon pickup Sink plug removal Plate wiping Mean
VisionOnly 0.11 0.40 0.24 0.37 0.28
DAVATaR-T (no tactile) 0.22 0.52 0.41 0.48 0.41
DAVATaR-VA (no augmentation) 0.56 0.61 0.37 0.64 0.55
DAVATaR 0.67 0.75 0.61 0.81 0.71
  • Adversarial conditions are where it matters: Success averaged over the adversarial settings is 9% for VisionOnly, 25% without tactile, 37% without augmentation, and 61% for the full method. Sink plug removal under adversarial conditions goes from 0% (VisionOnly) to 50% (full).
  • Why tactile helps:
  • In water sampling, foam hides the surface, so vision can't tell how deep the syringe is.
  • In spoon pickup, refraction makes the gripper look closer than it is, so vision-only policies close early.
  • Attention rollout shows tactile attention rising during contact phases. The authors call this qualitative only.
  • Single-augmentation ablation (spoon pickup):
  • Reflection augmentation alone fixes glare: without tactile it goes from 0% to 54%, and with tactile from 46% to 71%.
  • Refraction augmentation alone barely helps without tactile (38% to 39%), but helps a lot with tactile (37% to 67%).
  • The authors' reading: reflection hurts visibility, while refraction corrupts geometry, which touch can fix.
  • Limitations (as the authors state them):
  • The compliant fingertips deform, and the channel sealant peels under repeated large forces.
  • The long fluidic lines drift with temperature (ideal gas law).
  • The augmentations model only zoom and patches. They miss water motion, dynamic lensing, and defocus.

Talk notes

  • The talk opened with the paper's Fig. 1: "Submerged tasks are difficult." Submerged manipulation suffers air–water interface distortions: refraction, reflection, and foam.