NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perception

A neural actuator model for low-cost servo-driven robot platforms, linking actuator telemetry to differentiable dynamics, sensorless force perception, and force-aware sim-to-real control.

Robotics: Science and Systems (RSS) 2026

Finalist for the Outstanding Student Paper and Outstanding Paper Awards.

Zhiyang Dou1 John U. Onyemelukwe1* Hangxing Zhang1* Heng Zhang1 Minghao Guo1 Yunsheng Tian1 Michal Piotr Lipiec1 Joshua Jacob1 Chao Liu1 Peter Yichen Chen1 Yuri Ivanov2,† Wojciech Matusik1

1 MIT 2 Amazon Robotics

* Research Assistant at MIT CDFG, equal contribution.
 The work of this author does not relate to their position at Amazon.

Abstract

Differentiable simulators have advanced policy learning and model-based control across diverse robotic tasks. To date, actuator dynamics remain underexplored and are a major source of sim-to-real error, especially on low-cost platforms where the linear current-torque model breaks down under commanded-target tracking due to friction, hysteresis, backlash, and thermal effects. Beyond forward dynamics, accurate actuator models also support force perception, which is crucial for jointly modeling force and position control in manipulation tasks. We present NeuralActuator, a neural actuator model that jointly predicts torques, external contact forces, force detection gates, and motor conditions. We introduce a twin-arm teleoperation system that collects motor states alongside ground-truth forces from interactions and known external forces, contributing a dataset named Neural Actuation Dataset (NAD). NeuralActuator is trained through differentiable simulation using only pose trajectories as supervision, eliminating the need for torque sensors. A Transformer-based architecture captures temporal dependencies while enabling efficient real-time inference. We validate NeuralActuator across three platforms spanning low-cost and industrial arms, and show that it enables accurate dynamics modeling, sensorless force estimation, motor condition estimation, and improved behavior-cloning control when used as a pretrained module.

Overview

Low-cost robot actuation as a path to dynamics, force perception, and real-robot control.

Force-gauge validation for NeuralActuator
Force-gauge validation. Real contact trials, simulated rollouts, and predicted force curves across contact onset, steady force, and release.
Downstream manipulation tasks with payloads
Downstream manipulation tasks. Payload configurations used for lift-and-hold and pick-and-place robot control experiments.

NeuralActuator targets the actuator-modeling gap that is especially visible on low-cost robot arms: affordable servos expose rich telemetry, but the conventional linear current-to-torque model breaks down under commanded-target tracking due to friction, hysteresis, backlash, and thermal effects. The model learns how commanded targets, measured states, current, temperature, and recent history translate into torque, then reuses the same temporal representation to estimate external contact forces and motor condition.

Low-cost actuator dynamics

Learns joint-level torque for servo-driven arms instead of relying on a fixed current-to-torque conversion, capturing friction, hysteresis, saturation, thermal effects, and degraded motor behavior.

Sensorless force perception

Estimates contact force and force gates from actuator-side signals, enabling contact onset, payload change, and release detection without deployment-time force sensors.

Method

Problem formulation, NAD, and Transformer actuator model.

Problem Formulation

NeuralActuator models each servo-driven joint as a history-dependent mapping from commanded pose, current, measured state, thermal state, and recent telemetry to real torque. The same input stream is also used to estimate end-effector contact force and motor condition, enabling dynamics modeling, sensorless force perception, and actuator health monitoring from actuator-side signals.

Neural Actuation Dataset (NAD)

NAD is collected with a twin-arm teleoperation setup that records commanded pose, measured pose, current, voltage, temperature, and force supervision from force sensors, force gauges, and known payloads.

NeuralActuator data collection setup
NAD data collection. Leader-follower teleoperation records synchronized robot states, actuator telemetry, and external force labels.
Force data verification
Data verification. Time-synchronized video and trajectory logs with visualized external force.

Dataset composition. NAD includes free-motion, force-labeled, and motor-conditioned trajectories for actuator dynamics, force prediction, and condition monitoring.

Component Description Duration
Free motionNo external force~34.15 min
Force labeledKnown weights or force sensing~46.24 min
Motor conditionDegraded actuation~14.13 min
Total-~94.52 min

NeuralActuator

NeuralActuator formulates actuation as a history-dependent mapping from commanded targets, proprioception, and actuator telemetry to torque, external force, contact gate, and motor condition. The torque head is trained through differentiable simulation from pose trajectories, while the force, gate, and condition heads use direct supervision when labels are available.

NeuralActuator pipeline and transformer architecture
NeuralActuator pipeline. A Transformer predicts torque, external force, and motor condition inside a differentiable simulation loop.
01 Torque-Label-Free Differentiable Actuator Learning

Learns joint torque from real pose trajectories by backpropagating through differentiable simulation, avoiding torque labels and reliable current-to-torque calibration on low-cost platforms.

θ = arg minθ t Lpose ( qt+1sim , qt+1real ) qt+1sim = πq [ DiffSim ( st , τθ (Ht) ) ]
02 History-Dependent Nonlinear Actuator Modeling

Uses a Transformer over commands, proprioception, and actuator telemetry to model nonlinear and time-varying torque behavior associated with friction, backlash, saturation, hysteresis, and thermal drift.

03 Unified Actuation and Proprioceptive Force Perception

Jointly predicts actuator torque (τ), the 3-axis end-effector external force (fext), contact gate (g), and motor condition (c), supporting sensorless force perception, motor-health monitoring, and force-aware downstream control.

M(q) q¨ + C(q,q˙) q˙ + ggrav(q) = τ + Jv(q) fext τext = Jv(q) fext

Experiments

Rollout accuracy, force estimation, baselines, motor condition, and runtime.

Rollout and Force Estimation

Force sensor measurements and results
External force estimation. Estimated end-effector forces across directional pushes and payload manipulation tasks.

Accuracy summary. Units are shown in the table header: joint errors in degrees, gripper errors in millimeters, and force errors in Newtons.

Benchmark Horizon (steps) J1 (deg) J2 (deg) J3 (deg) J4 (deg) Grip (mm) Force (N)
No-load rollout600 steps3.12.83.23.10.2-
Force-sensor test500 steps1.783.312.011.580.650.23
Weight-based test600 steps2.974.063.513.770.500.11

Force-estimation baselines. All entries are force MAE in Newtons (N). NeuralActuator predicts external force from simulated rollout states, while classical baselines consume ground-truth states at each step.

Method Go Up 200g (N) Go Up 300g (N) Go Up 400g (N) Pick 200g (N) Pick 300g (N) Pick 400g (N) Pick 500g (N) Avg. (N)
ID-Linear1.371.812.300.720.951.221.471.41
ID-Friction1.061.592.150.620.821.101.311.23
GMO0.580.661.230.330.470.630.750.66
NeuralActuator0.120.200.070.240.000.190.050.12

Motor Condition and Runtime

Motor condition prediction
Motor condition estimation. Abnormal resistance produces higher current under similar position trajectories.
Degraded motor data collection inset
Motor degradation setup. Rubber-band-constrained joint used to collect motor-conditioned trajectories.

Motor condition detection. A learned condition head separates normal operation from mechanically degraded trajectories.

Method Accuracy Precision Recall AUC-ROC
Threshold58.6%0.0%0.0%0.45
SVM59.9%52.6%31.7%0.62
Random Forest67.1%62.3%52.4%0.72
NeuralActuator91.0%84.5%96.2%0.95

Runtime performance. The model is lightweight enough for simulation and real-time control, with sub-millisecond GPU inference latency.

Metric Value Unit Metric Value Unit
Parameters1.42M-Mean time0.25ms
FLOPs (forward)5.46M-P95 time0.31ms
Parameter memory5.43MBThroughput (batch=1)4,019Hz
---Throughput (batch=32)10,992Hz

Sim-to-Real Robot Control

Frozen actuator model for payload-aware behavior cloning.

High-level robot tasks
High-level pick-and-place. Time-lapse snapshots under heavier and shape-varied payloads.
Pick and place robot experiment
Pick-and-place with payloads. Real executions and simulated rollouts with estimated external forces.
Lift and hold robot experiment
Lift-and-hold with payloads. Real and simulated rollouts with visualized forces from 200g to 500g.

Behavior cloning success rates. Results are averaged over 40 real-robot trials and compare position-only control against a force-aware policy using the frozen NeuralActuator module.

Task Without NeuralActuator With NeuralActuator
Pick-and-place80%92.5%
Go up-and-stay85%95%

Supporting Results

Cross-platform validation, force generalization, visual supervision, and dataset details.

These results expand the main experiments along four directions: transfer across robot platforms, robustness to unseen payloads, differentiable visual supervision, and the data collection details needed to reproduce NAD.

Cross-Platform Dynamics and Learned Torque

NeuralActuator is built around low-cost servo-driven platforms, where actuator nonlinearity is a dominant sim-to-real bottleneck. The same formulation is also evaluated on an industrial manipulator to test generality, and the learned torque traces expose the current-to-torque variation that motivates replacing a fixed linear actuator model.

Robot platforms used for NeuralActuator
Cross-platform validation. Low-cost and industrial robot arms across multiple actuator families.
Current torque relationship
Learned torque values. The effective current-to-torque mapping varies across joints, phases, saturation, and noise.

Force Generalization

Force prediction is evaluated beyond the exact training objects with two unseen payload geometries and surface conditions. During the still-holding phase, the pretrained model predicts 2.80 N for a 261 g object with 2.56 N ground truth, and 2.40 N for a 226 g object with 2.21 N ground truth.

Novel contact geometry weights
Novel contact geometries. Predicted vs. ground-truth forces for unseen objects: 2.80 N vs. 2.56 N (261 g) and 2.40 N vs. 2.21 N (226 g).

Visual Supervision

Because the actuator model sits inside a differentiable simulator, it can be combined with differentiable rendering. The visual-supervision example refines camera and robot state by aligning rendered robot silhouettes with image masks.

Differentiable rendering appendix figure
Visual supervision. Differentiable rendering aligns robot silhouettes during hand-eye calibration.

Dataset and Robot Details

NAD is built from a leader-follower collection rig, controlled payloads, force-sensor interactions, and motor-condition interventions. These setup figures document how the three supervision sources are produced and how the robot gripper is modeled in simulation.

Data collection hardware
Data collection hardware. Twin-arm setup, force/torque sensor, force gauge, and payload set.
Free motion data collection
Free-motion data. Contact-free trajectories for nominal actuator dynamics.
Weight-labeled data collection
Weight-labeled data. Payload tasks for repeatable gravity-induced force labels.
Motor restricted data collection
Motor-restricted data. Pick-and-place trajectories under constrained actuator conditions.
Force sensor data collection
Force-sensor data. Directional interaction trials with an external force/torque fixture.
OpenManipulator-X motor
Robotic arm actuator. Dynamixel motor used in the OpenManipulator-X leader-follower system.
Robot gripper schematic
Robot gripper schematic. Top and side views used to derive the gripper aperture from motor rotation.

Gauge Pushing

A separate gauge-pushing test isolates contact transitions from the payload tasks above. The predicted force tracks the measured force through contact onset, sustained pushing, and release.

Gauge pushing qualitative force prediction
Gauge pushing test. Predicted force tracks measured force across contact onset, magnitude, and release.

Architecture and Adaptation

The final checks compare temporal model choices, show rapid online calibration from a small batch of new trajectories, and verify that NeuralActuator can be inserted into another differentiable physics backend.

Architecture ablation. Transformer history modeling improves force prediction while remaining competitive on rollout accuracy.

Model J1 J2 J3 J4 Grip Force
MLP4.555.813.482.160.910.47
GRU1.832.081.681.700.650.49
LSTM2.917.663.213.080.710.41
NeuralActuator1.783.312.011.580.650.23
Online learning progress plot
Online adaptation. Rapid fine-tuning from a small batch of newly collected trajectories.
Warp simulation results
Warp simulation. NeuralActuator integrated with another differentiable physics backend.

Quantitative Results

Rollout accuracy, force prediction, adaptation, and cross-platform tests.

Dynamics and Force Prediction

Gradient behavior across rollout horizons.

HTorque Grad.||grad L||Cos.
642.73e-28.770.96
1282.59e-217.331.00
2561.72e-222.350.99
3201.45e-222.910.99
5009.37e-320.290.98

Simulation accuracy on the test set. J1-J4: joint-angle MAE (deg). Grip: gripper-aperture MAE (mm).

Task@100@300@600
J1J2J3J4GripJ1J2J3J4GripJ1J2J3J4Grip
backward_forward2.44.34.44.202.35.93.94.203.14.84.94.50
circular_ccw1.75.22.22.601.53.12.03.402.52.32.12.80
circular_cw3.63.21.42.203.11.91.22.602.61.72.02.80
go_up_stay_still2.51.82.31.203.12.13.12.503.11.53.52.70
joint_sweep_12.32.51.81.401.91.02.61.802.91.92.01.60
joint_sweep_22.14.34.61.201.86.78.41.102.44.65.21.70
joint_sweep_33.74.53.11.903.42.62.82.803.12.62.53.40
joint_sweep_42.64.82.53.402.93.32.43.202.42.92.24.60
joint_sweep_51.73.01.73.10.42.63.92.32.70.55.53.35.13.00.7
pick_place_empty2.62.52.23.51.11.93.62.73.21.13.12.72.03.91.0
Average2.53.62.62.50.22.53.43.12.80.23.12.83.23.10.2

Simulation and force prediction accuracy on the force-sensor test set. F: force MAE (N).

Task@100@300@500
J1J2J3J4GripFJ1J2J3J4GripFJ1J2J3J4GripF
force_X+0.92.30.91.61.00.361.11.82.41.31.00.460.91.92.71.71.00.50
force_X-1.41.10.81.11.00.391.62.31.32.31.00.422.02.91.53.01.00.44
force_Y+2.35.72.80.60.10.362.14.52.60.90.20.382.74.82.10.90.30.44
force_Y-1.41.63.10.50.10.381.53.03.11.30.10.361.43.63.11.50.20.38
force_Z+3.36.23.30.80.10.392.35.62.51.90.20.432.54.42.22.30.30.46
force_Z-1.05.90.60.61.00.362.96.62.82.51.00.643.86.43.52.01.00.57
ref_X+0.62.01.51.21.00.010.81.81.01.11.00.021.01.61.11.11.00.01
ref_X-2.02.50.51.11.00.001.42.30.51.01.00.001.52.20.81.31.00.00
ref_Y+2.01.02.00.60.10.002.13.32.11.10.20.001.93.71.71.10.40.00
ref_Y-1.01.81.21.30.10.001.64.52.02.20.10.001.24.02.41.60.30.00
ref_Z+1.21.72.10.50.10.001.63.82.21.20.20.001.52.92.01.30.30.00
ref_Z-0.62.30.90.91.00.020.71.61.11.11.00.010.91.31.01.11.00.01
Avg1.482.841.640.900.550.191.643.431.971.490.580.231.783.312.011.580.650.23

Simulation and force prediction accuracy on the weight-based test set.

TaskWeight@100@300@600
J1J2J3J4GripFJ1J2J3J4GripFJ1J2J3J4GripF
go up and stay200g1.35.02.62.40.10.123.43.94.22.30.10.103.13.34.32.90.10.16
go up and stay300g2.65.35.73.70.10.201.83.45.75.40.00.173.52.53.44.60.00.20
go up and stay400g2.24.11.45.00.10.072.92.12.16.10.10.122.01.43.07.10.20.11
pick and place200g0.73.14.33.41.10.242.73.94.04.01.10.082.95.12.93.01.00.11
pick and place300g0.54.53.70.90.30.001.25.14.81.50.20.002.77.84.93.20.30.09
pick and place400g0.82.12.82.91.10.192.81.43.43.61.10.062.42.03.83.31.00.03
pick and place500g2.71.81.73.31.10.054.82.81.72.31.10.024.26.32.32.30.90.04
Avg1.543.703.173.090.560.122.803.233.703.600.530.082.974.063.513.770.500.11

Dataset and Adaptation

Task composition of the dataset.

CategoryTaskVariant#FramesDuration (s)
Free motionCircular trajectoryClockwise8615147.06
Free motionCircular trajectoryCounterclockwise8428143.90
Free motionJoint sweepMotor 122935392.04
Free motionJoint sweepMotor 28688148.47
Free motionJoint sweepMotor 311357193.95
Free motionJoint sweepMotor 415012256.57
Free motionJoint sweepMotor 57338125.21
Free motionLean back and extend forward12261209.35
Free motionPick & place (empty)15288261.08
Free motionGo up and stay still10011171.14
Force-labeledGo up and stay still200g10606181.26
Force-labeledGo up and stay still300g11259192.40
Force-labeledGo up and stay still400g11291192.89
Force-labeledPick and place200g13245226.15
Force-labeledPick and place300g13129224.31
Force-labeledPick and place400g15124258.54
Force-labeledPick and place500g13957238.49
Force-labeledForce sensor+X500585.17
Force-labeledForce sensor+X w/o interaction500585.21
Force-labeledForce sensor-X516987.94
Force-labeledForce sensor-X w/o interaction516988.00
Force-labeledForce sensor+Y7444126.74
Force-labeledForce sensor+Y w/o interaction7444126.78
Force-labeledForce sensor-Y6921117.88
Force-labeledForce sensor-Y w/o interaction6921117.91
Force-labeledForce sensor+Z6906117.59
Force-labeledForce sensor+Z w/o interaction6906117.63
Force-labeledForce sensor-Z556594.69
Force-labeledForce sensor-Z w/o interaction556594.73
Motor conditionPick & place w/ weightMotor restricted10113168.26
Motor conditionPick & place w/ weightMotor in good condition13245226.15
Motor conditionPick & place w/o weightMotor restricted11252192.41
Motor conditionPick & place w/o weightMotor in good condition15288261.08

Visual-supervision and architecture settings.

GroupMetric / ModelValue 1Value 2Value 3Value 4Value 5
Silhouette IoUInitializationMean 0.2589Std 0.0441
Silhouette IoURobot refinementMean 0.2974Std 0.0754
Silhouette IoUJoint refinementMean 0.8515Std 0.0065
RuntimeJoint refinement1845 iterations180.23 s
ArchitectureMLPHidden 416Latent 2081.14M paramsLR 3e-5Clip 0.3
ArchitectureGRUHidden 325Latent 1621.43M paramsLR 1e-4Clip 1.0
ArchitectureLSTMHidden 275Latent 1371.44M paramsLR 1e-4Clip 1.0
ArchitectureNeuralActuatorHidden 192Latent 961.44M paramsLR 1e-4Clip 1.0

Online adaptation before (B) and after (A). J1-J4: joint-angle MAE (deg). Grip: gripper-aperture MAE (mm). F: force MAE (N).

Task@100@300@500
J1 BJ1 AJ2 BJ2 AJ3 BJ3 AJ4 BJ4 AGrip BGrip AF BF AJ1 BJ1 AJ2 BJ2 AJ3 BJ3 AJ4 BJ4 AGrip BGrip AF BF AJ1 BJ1 AJ2 BJ2 AJ3 BJ3 AJ4 BJ4 AGrip BGrip AF BF A
force_X+0.90.32.10.71.00.80.80.81.01.00.360.372.11.22.31.00.71.50.90.81.01.00.460.412.71.64.01.30.61.90.90.81.01.00.520.41
force_X-0.20.51.11.70.91.20.71.31.01.00.390.391.70.71.61.61.40.70.81.21.01.00.410.402.50.91.91.62.00.80.91.21.01.00.420.41
force_Y+2.11.85.43.40.81.11.21.30.00.10.370.372.31.54.53.21.11.11.61.00.10.10.390.373.01.23.72.32.51.23.11.70.30.30.450.39
force_Y-1.11.20.61.31.11.81.81.80.10.10.380.381.61.03.12.61.21.42.12.00.10.10.360.361.71.02.32.02.51.12.11.60.20.20.380.37
force_Z+2.94.05.23.61.33.10.60.50.10.10.390.392.72.14.02.71.41.50.70.90.20.10.440.425.01.83.22.01.71.51.41.50.30.20.460.45
force_Z-0.70.96.23.52.00.61.00.71.01.00.360.362.00.77.12.31.80.83.31.21.01.00.610.382.61.07.21.62.00.83.01.11.01.00.540.40
ref_X+1.40.41.60.41.01.00.80.81.01.00.010.081.50.51.30.60.80.81.01.21.01.00.020.081.30.51.10.60.61.01.41.91.01.00.010.05
ref_X-0.40.73.20.62.40.50.71.01.01.00.000.000.70.62.41.21.90.50.90.91.01.00.000.040.80.62.21.02.20.71.10.81.01.00.000.05
ref_Y+1.41.81.02.81.41.41.31.90.10.10.000.002.21.73.33.31.01.21.81.40.20.20.000.012.21.24.42.41.01.11.31.30.40.30.000.01
ref_Y-2.01.22.21.90.80.42.71.30.00.10.000.012.51.54.82.70.70.83.21.80.10.10.000.012.01.24.92.10.71.32.72.00.30.20.000.01
ref_Z+0.60.91.30.80.41.31.41.30.10.10.000.001.31.23.41.60.41.01.21.00.20.20.000.001.01.13.41.50.51.20.91.30.30.30.000.00
ref_Z-1.80.51.90.81.10.90.50.71.01.00.030.141.70.60.90.71.10.80.80.81.01.00.020.111.60.50.80.71.00.70.81.11.01.00.010.08
Avg1.281.192.651.811.191.171.131.120.550.550.190.211.861.093.221.961.111.021.531.180.590.580.230.222.201.043.251.581.441.101.641.350.660.650.230.22

Cross-Platform Evaluation

SO-101 arm evaluation. Joint errors are in degrees and force errors are in Newtons.

PlatformTaskLift & Hold / @500Pick & Place / @500
J1J2J3J4J5FJ1J2J3J4J5F
SO-101300g2.278.206.405.402.370.643.735.634.176.172.470.73
SO-101400g2.839.309.634.005.730.573.536.832.435.235.130.63
SO-101500g2.277.639.777.132.670.473.805.601.536.733.330.54

7-DoF industrial arm per-joint, force, and torque MAE.

Task@100@500
J1J2J3J4J5J6J7FTorqueJ1J2J3J4J5J6J7FTorque
200g0.61.13.71.70.70.41.50.420.500.81.23.71.91.42.31.20.310.50
300g0.41.22.82.00.80.51.10.350.680.61.02.71.71.11.70.90.280.59
400g1.01.21.91.80.70.50.80.360.791.00.81.31.40.91.50.60.270.69
500g1.41.10.91.70.80.60.40.270.871.70.70.71.40.81.30.50.260.79
600g2.01.31.01.60.70.90.40.300.982.40.71.41.40.71.10.50.280.90
Avg1.11.22.01.80.70.60.80.340.761.30.92.01.61.01.60.80.280.69

Gauge pushing test set.

Task@100@300@600
J1J2J3J4GripFJ1J2J3J4GripFJ1J2J3J4GripF
high_push_front0.911.681.180.620.000.071.621.290.530.320.000.081.481.161.441.090.000.09
high_push_top0.731.010.781.610.010.101.381.351.971.920.010.111.621.572.722.010.010.12
mid_push_front0.660.520.960.610.300.060.850.510.600.990.310.070.950.560.480.980.310.08
mid_push_top0.490.540.870.740.300.090.640.430.731.070.310.100.890.520.750.780.310.11
low_push_front0.610.751.830.230.320.080.870.561.650.330.320.091.290.881.120.980.330.10
low_push_top1.000.361.050.290.310.081.170.842.110.740.320.091.350.762.271.040.320.10
Avg0.730.811.110.680.210.081.090.831.270.900.210.091.260.911.461.150.210.10

Future Directions

Scaling actuator learning across data, robot morphologies, and force-aware control.

01 Synthetic-to-Real Actuator Pretraining

Leverages large-scale synthetic actuator data for pretraining, followed by real-world fine-tuning, to reduce costly hardware data collection.

02 Cross-Morphology Actuation Learning

Extends NeuralActuator across robot morphologies and actuator families for more generalizable actuation modeling.

03 Scalable Whole-Body Force Perception

Moves beyond single-arm end-effector force estimation toward multi-joint and whole-body force-aware robot control.