Documentation
The details behind the project page: platform, dataset, policies, protocol, and the full result tables. It documents completed work.
Thesis
Learning Contact Representations from Zero-Cost Proprioceptive Effort Signals for Contact-Rich Visuomotor Imitation Learning. Jaydeepkumar Ranpariya, master's thesis, Friedrich-Alexander-Universität Erlangen-Nürnberg, hosted by the Fraunhofer Institute for Integrated Circuits, 2026. Supervised by Prof. Michael Philippsen and Dr. Tobias Feigl (FAU) and Alexander Mattick (Fraunhofer IIS).
Overarching question
Can weak, uncalibrated motor-current signals already available on a low-cost robot provide a useful learned contact representation for contact-rich visuomotor imitation learning?
Contributions
- A reproducible characterisation of all six servo-current channels as candidate learning inputs.
- A synchronised 100-demonstration multimodal peg-in-hole dataset with human-reviewed phase marks on the analysed subset.
- Controlled ACT variants for raw concatenation, training-derived normalisation, and a learned joint-selective temporal representation.
- A Diffusion Policy baseline and a learned-current counterpart evaluated under one dataset and one execution timebase.
- A model-agnostic counterfactual diagnostic of how much a policy's actions depend on one modality.
- A verification framework that recomputes every analysis and figure from the retained records.
Platform
SO-101 leader and follower arms, six STS3215 bus servos each. Teleoperation at 30 Hz with one-to-one joint mapping and no live camera feed to the operator; follower current is not reflected back to the leader.
Current register
- 6.5 mA per count; pipelines keep raw counts. Observed task range 0–77 counts.
- All six registers read at 55.3 Hz (18.1 ms), comfortably inside the 33.3 ms policy period.
- At rest the register varies by less than half a count; across arm poses the shoulder-lift baseline moves from about 0.1 to 26 counts.
- Contact rises last roughly 100–300 ms, three to nine samples at 30 Hz. Firmware stall protection lowers current after about two seconds of sustained stall.
Task
- 16.45 mm printed peg into a 16.8 mm unchamfered bore: 0.35 mm diametral clearance, a clearance-to-diameter ratio of 0.021, four times tighter than the pilot fixture.
- Vertical peg grasped from the side, starting at a marked position about 7 cm from the hole.
Channel selection
The gripper load register discriminates contact strongly but is coarsely quantised, saturates, and correlates with current at 0.88, so it was dropped as redundant. Speed, voltage, and temperature are diagnostic only. The policy state is the six joint positions plus the six currents, with no dynamics model, torque calibration, or force reconstruction.
Dataset
Each frame stores the wrist image (and a side image for inspection), six follower joint positions, six follower currents, six temperatures for diagnostics, and the six leader joint positions as the action, all at 30 Hz.
Recorded sets
| Set | Episodes | Clearance | Role |
|---|---|---|---|
| Flat peg, chamfered hole | 50 | 1.40 mm | easy pilot |
| Flat peg, no chamfer | 50 | 0.35 mm | difficulty ablation |
| Vertical peg, no chamfer | 50 | 0.35 mm | first batch |
| Vertical peg, no chamfer | 50 | 0.35 mm | extension batch |
| Vertical peg, merged | 100 | 0.35 mm | training set for every matched comparison |
The training set behind every matched comparison holds 100 demonstrations and 99,831 frames from four sessions, all successful insertions, mean length 33 s.
Current distribution
- 44.9 % of all current samples are exactly zero; per channel 35–54 %.
- The shoulder-pan channel has skewness 11 and excess kurtosis near 400, so plain standardisation maps rare maxima to more than fifty standard deviations.
- The later variants clip each channel at its per-joint training 99.5th percentile before scaling. Caps and statistics come from training demonstrations only.
Recording
45 s episodes with 15 s resets and a temperature check every ten episodes. Of 32 human-reviewed phase rows, one non-monotonic row was excluded; the 31 remaining serve only as calibration metadata.
Policies
Vision-only policies receive the six joint positions; current-aware policies receive positions and currents. ACT runs at 30 Hz; the recovered Diffusion Policy runs at 10 Hz.
Action Chunking Transformer
| Image input | wrist RGB 640 × 480 at 30 Hz, ResNet-18 backbone |
| Action chunk | 50 predicted, 50 executed (about 1.7 s open loop) |
| Transformer | width 256, 8 heads, 4 encoder and 1 decoder layers |
| Training | 100,000 steps, batch 16, single seed |
| Learned current encoder | a short per-joint current history through a shared 1D convolution into six joint tokens |
Diffusion Policy
| Observation and horizon | 2 observation steps, horizon 16, 8 executed actions |
| Backbone | conditional U-Net, widths 512 / 1024 / 2048 |
| Training | DDPM with 100 steps; recovery 200,000 updates at 10 Hz, then a 40,000-update warm start |
| Inference | DDIM with 10 or 16 steps, gated on latency |
| Learned current route | eight preceding samples per joint through a small convolution, feeding a bounded residual modulation of every U-Net block; the base network stays frozen |
Two historical fusion variants (an early current encoder and a feature-modulation model) are kept for traceability but are implementation-limited and do not support architectural conclusions.
Evaluation protocol
- Both policies of a pair train on the same demonstrations with the same seed and are evaluated in the same session, alternating in blocks of six trials, five blocks each.
- Success: the policy grasps, transports, and seats the peg flush before the 45 s limit. Partial insertion is a failure. Failure phases are logged.
- Thermal gating: every servo at or below 50 °C before a block starts.
- Statistics: Wilson 95 % interval per proportion, two-sided Fisher exact test per pair, Holm correction when several pairs are reported, and a predeclared ten-point practical threshold.
- Comparisons from different sessions are never pooled.
Results
Matched comparisons
| Comparison | Vision only | Current-aware | Fisher p | Difference, 95 % |
|---|---|---|---|---|
| ACT, raw current | 19/30 | 8/30 | 0.0089 | [−56.0, −11.5] points |
| ACT, normalised current | 17/30 | 15/30 | 0.796 | [−29.9, 17.6] points |
| ACT, learned encoder | 7/30 | 12/30 | 0.267 | [−6.7, 37.8] points |
| Diffusion Policy | 9/30 | 12/30 | 0.589 | [−13.6, 32.1] points |
In the final Diffusion Policy session both policies declined across the five blocks, from eight combined successes in the first block to one in the last. The unchanged vision-only ACT checkpoint scored 17 of 30 in one session and 7 of 30 in another, which is why comparisons are never ranked across sessions.
Task difficulty, vision-only ACT
| Condition | Successes | Rate | Δ points | Fisher p |
|---|---|---|---|---|
| Flat, 1.4 mm clearance, chamfer | 25/30 | 83 % | n/a | n/a |
| Flat, 0.55 mm, chamfer | 23/30 | 77 % | −7 | 0.748 |
| Flat, 0.55 mm, no chamfer | 14/30 | 47 % | −37 | 0.006 |
| Flat, 0.35 mm, no chamfer, retrained | 19/30 | 63 % | −20 | 0.143 |
| Vertical, 0.35 mm, 50 demonstrations | 5/30 | 17 % | −67 | < 0.001 |
| Vertical, 0.35 mm, 100 demonstrations | 15/30 | 50 % | n/a | n/a |
| Vertical, 100 demonstrations, two cameras | 14/30 | 47 % | −3 | 1.00 |
Removing the chamfer, not the tighter clearance alone, caused the first significant drop. The vertical grasp and the side camera each change geometry and visibility together, so their effects cannot be split.
Historical fusion variants, 30 trials each
| Policy | Successes | Rate | Wilson 95 % |
|---|---|---|---|
| Vision only | 19/30 | 63.3 % | [45.5, 78.1] |
| Raw current, concatenated | 8/30 | 26.7 % | [14.2, 44.4] |
| Raw current, early encoder | 6/30 | 20.0 % | [9.5, 37.3] |
| Raw current, feature modulation (defective) | 0/30 | 0.0 % | [0.0, 11.4] |
Limitations
- One robot, one servo firmware, one task, one operator, one training seed, thirty trials per confirmatory row.
- Binary seating success does not measure insertion quality or contact force.
- The counterfactual diagnostic measures sensitivity, not causal importance; the imputed input may lie off the training distribution.
- Phase-specific claims need verified recording-based phase boundaries, which remain incomplete.
- Demonstrations are success-only, so failed grasps, dropped pegs, and rim-search recoveries are not covered.
- Motor current cannot identify slip; a 30 Hz effort stream may not distinguish slip from free motion.
Future work
- A same-session comparison of a learned encoder against plain concatenation.
- Shorter execution horizons or 100–200 Hz current histories, to test whether latency hides brief contacts.
- Repeats across seeds, tasks, and servo families, and a slip-capable modality such as acoustic or tactile sensing.