NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perception

Dou, Zhiyang; Onyemelukwe, John U.; Zhang, Hangxing; Zhang, Heng; Guo, Minghao; Tian, Yunsheng; Lipiec, Michal Piotr; Jacob, Joshua; Liu, Chao; Chen, Peter Yichen; Ivanov, Yuri; Matusik, Wojciech

NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perception

A differentiable neural actuator model for low-cost robots that supports torque-surrogate prediction, sensorless force perception, and force-aware real-robot control.

Robotics: Science and Systems (RSS) 2026.

Outstanding Systems Paper Award at RSS 2026.

Zhiyang Dou¹ John U. Onyemelukwe^1* Hangxing Zhang^1* Heng Zhang¹ Minghao Guo¹ Yunsheng Tian¹ Michal Piotr Lipiec¹ Joshua Jacob¹ Chao Liu¹ Peter Yichen Chen¹ Yuri Ivanov^2,† Wojciech Matusik¹

¹ MIT ² Amazon Robotics

^* Research Assistant at MIT CDFG, equal contribution.
^† The work of this author does not relate to their position at Amazon.

Abstract

Differentiable simulators have advanced policy learning and model-based control across diverse robotic tasks. To date, actuator dynamics remain underexplored and can be a major source of sim-to-real error, especially on low-cost platforms where the linear current-to-joint-torque approximation τ = K_tI becomes unreliable under commanded-target tracking because of friction, hysteresis, backlash, and thermal effects. Beyond forward dynamics, accurate actuator models also support force perception and integrated force/position control in manipulation tasks. We present NeuralActuator, a neural actuator model that jointly predicts (i) a torque surrogate for trajectory propagation on low-cost servo platforms, (ii) external forces together with a contact-probability gate for sensorless force perception, and (iii) a motor-condition score for the supervised joint, distinguishing normal from mechanically restricted operation. We introduce a twin-arm teleoperation system that collects robot states and actuator telemetry alongside external-force labels, yielding the Neural Actuation Dataset (NAD). The torque-surrogate head is trained through differentiable simulation from pose trajectories without ground-truth joint-torque measurements. A Transformer-based architecture captures temporal dependencies while enabling efficient real-time inference. We validate NeuralActuator across three platforms: a 5-DoF OpenManipulator-X, a 6-DoF SO-101 from LeRobot, and a 7-DoF Franka Emika Panda, spanning three actuator families and costs from approximately $500 to more than $30,000. The low-cost platforms support physically plausible dynamics and enable force evaluation, while the offline Franka experiment provides an additional payload-force-estimation benchmark. We also demonstrate motor-condition estimation and improved behavior-cloning performance when NeuralActuator is used as a pretrained module. We release the dataset, code, and hardware configurations on this project page.

Overview

Low-cost robot actuation as a path to dynamics, force perception, and real-robot control.

Force-gauge validation for NeuralActuator — **Force-gauge validation.** Real contact trials, simulated rollouts, and predicted force curves across contact onset, steady force, and release.

Downstream manipulation tasks with payloads — **Downstream manipulation tasks.** Payload configurations used for lift-and-hold and pick-and-place robot control experiments.

NeuralActuator addresses actuator dynamics on low-cost robot arms, where affordable servos expose rich telemetry but the conventional linear current-to-joint-torque approximation becomes unreliable under commanded-target tracking. From command, state, tracking-error, and telemetry histories, the model predicts a torque surrogate for trajectory propagation together with external force, contact probability, and motor-condition scores. Under the implicit force coupling used in the paper, the torque surrogate alone drives the simulated body and may absorb interaction effects needed to reproduce the observed motion; it should therefore not be interpreted as an identifiable estimate of physical actuator torque.

Low-cost actuator dynamics

Learns a bounded torque surrogate for trajectory propagation from real pose trajectories, without ground-truth joint-torque measurements or a fixed current-to-joint-torque approximation.

Sensorless force perception

Estimates external force and contact probability from commands, proprioception, and actuator telemetry without deployment-time force sensors.

Method

Problem formulation, NAD, and Transformer actuator model.

Problem Formulation

NeuralActuator maps a nine-frame history of commanded targets, proprioceptive state, tracking error, and actuator telemetry to four outputs: a pre-clipping torque surrogate, a raw 3D external-force estimate, a contact-probability gate, and per-motor condition scores. Under implicit coupling, the clipped torque surrogate alone drives the differentiable simulator; the force estimate is directly supervised but is not applied to the state update. Under explicit coupling, the predicted force is additionally applied at the force-reference point and contributes J_v(q)^Tf_ext alongside the torque surrogate, so the rollout loss also backpropagates through the applied force. The paper uses implicit coupling, which is more accurate on both low-cost platforms; explicit coupling is more sensitive to calibration mismatch in the force-reference frame, kinematics, and inertial parameters. In the OpenManipulator-X condition benchmark, only the Joint 3 score is supervised and evaluated, distinguishing normal from mechanically restricted operation.

Neural Actuation Dataset (NAD)

NAD is collected with a twin-arm teleoperation setup that records commanded and measured states, current, voltage, temperature, and external-force labels from known payloads or a fixture-mounted six-axis force/torque sensor. The force gauge is reserved for an independent evaluation benchmark and is not a source of NAD training supervision.

NeuralActuator data collection setup — **NAD data collection.** Leader-follower teleoperation records synchronized robot states, actuator telemetry, and external force labels.

Force data verification — **Data verification.** Time-synchronized video and trajectory logs with visualized external force.

OpenManipulator-X model-development and evaluation subset. The 94.52 minutes are task-assignment durations used in the experiments; nominal trajectories reused for the condition comparison also appear in the free-motion and force-labeled categories.

Component	Description	Duration
Free motion	No external force	~34.15 min
Force labeled	Known weights or force sensing	~46.24 min
Motor condition	Mechanically restricted Joint 3	~14.13 min
Total	-	~94.52 min

NeuralActuator

NeuralActuator formulates actuation as a history-dependent mapping from commanded targets, proprioception, and actuator telemetry to a torque surrogate, external force, contact probability, and motor-condition scores. The torque-surrogate head is trained through differentiable simulation from pose trajectories without ground-truth joint-torque measurements, while the force, gate, and condition heads use direct supervision when labels are available.

01 Torque-Surrogate Learning without Torque Labels

Learns a torque surrogate from real pose trajectories by backpropagating through differentiable simulation, without ground-truth joint-torque measurements or a reliable current-to-joint-torque calibration.

\begin{matrix} θ^{⋆} = \underset{θ}{arg min} \sum_{t} L_{pose} (q_{t + 1}^{sim}, q_{t + 1}^{real}) \\ q_{t + 1}^{sim} = π_{q} [DiffSim (s_{t}, {\bar{τ}}_{t})] \\ τ_{t}^{pred} = g_{θ} (X_{t}), {\bar{τ}}_{t} = clip (τ_{t}^{pred}) \end{matrix}

02 History-Dependent Nonlinear Actuator Modeling

Uses a Transformer over commands, proprioception, and actuator telemetry to predict the torque surrogate required for trajectory propagation, capturing nonlinear and history-dependent behavior without interpreting the surrogate as calibrated physical actuator torque.

03 Unified Actuation and Proprioceptive Force Perception

Separate heads predict the torque surrogate (τ^pred), 3D end-effector force (f_ext), contact probability (g), and per-motor condition scores (c). The manipulator equation below provides physical context; under the implicit coupling used in the paper, it is not imposed as an identifiable decomposition of actuator and external torques.

\begin{matrix} τ_{ID} = M (q) \ddot{q} + C (q, \dot{q}) \dot{q} + g_{grav} (q) \\ τ_{ID} = τ_{act} + τ_{ext}, τ_{ext} = {J_{v} (q)}^{⊤} f_{ext} \end{matrix}

Experiments

Rollout accuracy, force estimation, baselines, motor condition, and runtime.

Rollout and Force Estimation

Force sensor measurements and results — **External force estimation.** Estimated end-effector forces across directional pushes and payload manipulation tasks.

Accuracy summary. Joint errors are in degrees, Grip is single-finger slide-coordinate MAE in millimeters, and force errors are in Newtons.

Benchmark	Horizon (steps)	J1 (deg)	J2 (deg)	J3 (deg)	J4 (deg)	Grip (mm)	Force (N)
No-load rollout	600 steps	3.1	2.8	3.2	3.1	0.2	-
Force-sensor test	500 steps	1.78	3.31	2.01	1.58	0.65	0.23
Weight-based test	600 steps	2.97	4.06	3.51	3.77	0.50	0.11

Force-estimation baselines. All entries are force MAE in Newtons (N). NeuralActuator predicts external force from simulated rollout states, while classical baselines consume ground-truth states at each step.

Method	Go Up 200g (N)	Go Up 300g (N)	Go Up 400g (N)	Pick 200g (N)	Pick 300g (N)	Pick 400g (N)	Pick 500g (N)	Avg. (N)
ID-Linear	1.37	1.81	2.30	0.72	0.95	1.22	1.47	1.41
ID-Friction	1.06	1.59	2.15	0.62	0.82	1.10	1.31	1.23
GMO	0.58	0.66	1.23	0.33	0.47	0.63	0.75	0.66
NeuralActuator	0.12	0.20	0.07	0.24	0.00	0.19	0.05	0.12

Joint 3 Condition and Runtime

Joint 3 condition prediction under controlled mechanical restriction — **Joint 3 condition estimation.** Mechanically restricted operation draws higher current under matched position commands while following a similar trajectory.

Mechanically restricted Joint 3 data collection inset — **Mechanical restriction setup.** Rubber bands constrain Joint 3 to create the controlled restricted-operation condition.

Joint 3 condition classification. The supervised score separates unrestricted from mechanically restricted operation in this controlled experiment; it does not diagnose motor damage or assess general motor health.

Method	Accuracy	Precision	Recall	AUC-ROC
Threshold	58.6%	0.0%	0.0%	0.45
SVM	59.9%	52.6%	31.7%	0.62
Random Forest	67.1%	62.3%	52.4%	0.72
NeuralActuator	91.0%	84.5%	96.2%	0.95

Runtime performance. The model is lightweight enough for simulation and real-time control, with sub-millisecond GPU inference latency.

Metric	Value	Unit	Metric	Value	Unit
Parameters	1.44M	-	Mean time	0.25	ms
FLOPs (forward)	5.46M	-	P95 time	0.31	ms
FP32 parameter memory	5.50	MiB	Throughput (batch=1)	4,019	Hz
-	-	-	Throughput (batch=32)	10,992	Hz

Implicit vs. Explicit Force Coupling

We ablate how the predicted external force is coupled to the differentiable rollout on the combined payload benchmarks. The OpenManipulator-X benchmark combines pick-and-place tasks with 0-500 g payloads and lift-and-hold tasks with 0-400 g payloads; SO-101 follows the six-task payload protocol used in the paper. Implicit coupling drives the simulated body with the torque surrogate alone, whereas explicit coupling additionally applies the predicted force at the end effector. The latter closes the force path inside the simulator but introduces an additional generalized load that the torque-surrogate head must counteract. The animations below verify the explicit injection path using known forces at the grasp point: no applied force on the left, a downward force in the middle, and an upward force on the right.

Explicit external-force injection in the OpenManipulator-X differentiable simulation — **OpenManipulator-X force injection.** Applying 3.9 N downward and upward forces at the grasp point moves the end effector in the corresponding directions.

Explicit external-force injection in the SO-101 differentiable simulation — **SO-101 force injection.** The same no-force, downward-force, and upward-force test verifies the explicit coupling path on a second low-cost platform.

Force-coupling ablation. Joint MAE is the worst task-joint cell and force MAE is the all-task mean. For OpenManipulator-X, joint MAE reports the three-seed mean and range, while force MAE reports the three-seed mean. For SO-101, implicit uses the released checkpoint and explicit summarizes three from-scratch seeds.

Platform	Coupling	Worst per-joint MAE (deg)	Force MAE (N)
OpenManipulator-X	Implicit	0.88 (0.84–0.90)	0.06
OpenManipulator-X	Explicit	1.33 (1.00–1.53)	0.08
SO-101	Implicit	2.50	0.22
SO-101	Explicit	6.9 (6.35–7.34)	0.33

Implicit coupling is more accurate on both platforms. The difference is modest on OpenManipulator-X and substantially larger on SO-101, particularly during sustained payload holds. Explicit coupling is more sensitive to mismatch in the force-reference frame, kinematics, and inertial parameters; the resulting uncompensated load can degrade rollout accuracy and force prediction. We therefore use implicit coupling in the paper.

Force-Aware Imitation Learning for Real-Robot Control

Frozen force-perception module for payload-aware behavior cloning.

High-level robot tasks — **Real-robot manipulation.** Hardware evaluation of behavior-cloning policies under the reported payload conditions.

Pick and place robot experiment — **Pick-and-place with payloads.** Real executions and corresponding model rollouts with estimated external forces.

Lift and hold robot experiment — **Lift-and-hold with payloads.** Real and simulated rollouts with visualized forces from 200g to 400g.

Behavior cloning success rates. Both policies are trained from real-robot teleoperation demonstrations and evaluated on hardware. Results are averaged over 40 trials and compare position-only control with a force-aware policy using a frozen pretrained NeuralActuator module.

Task	Without NeuralActuator	With NeuralActuator
Pick-and-place	80%	92.5%
Go up-and-stay	85%	95%

Supporting Results

Cross-platform evaluation, unseen contact geometries, visual supervision, and dataset details.

These results expand the main experiments with platform-specific evaluation, a restricted test on unseen contact geometries, differentiable visual supervision, and the collection details needed to reproduce NAD.

Cross-Platform Evaluation and Surrogate Diagnostics

OpenManipulator-X and SO-101 provide rollout and force evaluation on low-cost servo platforms. Franka contributes only a future-state-conditioned offline payload-force-estimation benchmark, not torque or online dynamics validation. On OpenManipulator-X, raw pre-clipping torque-surrogate outputs are compared with measured currents as multivariate, history-conditioned diagnostics rather than as a calibrated current-to-torque relation.

Robot platforms used for NeuralActuator — **Cross-platform validation.** Low-cost and industrial robot arms across multiple actuator families.

Raw torque-surrogate outputs and measured motor currents — **Torque-surrogate diagnostics.** Raw, pre-clipping surrogate outputs and measured motor currents vary across joints and motion phases; the curves are not direct motor-torque measurements or a calibrated current-to-torque map.

Force Estimation on Unseen Contact Geometries

The pretrained force head is evaluated on two objects whose shapes and surface properties are not represented in training. During stationary holding under this restricted setting, it predicts 2.80 N for a 261 g object with 2.56 N ground truth, and 2.40 N for a 226 g object with 2.21 N ground truth. This test probes limited transfer to unseen contact geometries rather than robustness to arbitrary unseen payloads.

Novel contact geometry weights — **Unseen contact geometries.** During stationary holding, predicted vs. ground-truth forces are 2.80 N vs. 2.56 N (261 g) and 2.40 N vs. 2.21 N (226 g).

Visual Supervision

Because the actuator model sits inside a differentiable simulator, it can be combined with differentiable rendering. The visual-supervision example refines camera and robot state by aligning rendered robot silhouettes with image masks.

Differentiable rendering appendix figure — **Visual supervision.** Differentiable rendering aligns robot silhouettes during hand-eye calibration.

Dataset and Robot Details

NAD combines leader-follower trajectories, controlled payloads, fixture-mounted force/torque-sensor interactions, and unrestricted or mechanically restricted Joint 3 operation. The force gauge shown with the hardware is used only for the separate gauge-pushing evaluation, not to supervise NAD training samples.

**Data collection hardware.** Twin-arm setup, force/torque sensor, independent evaluation gauge, and 100-500 g payload set.

Free motion data collection — **Free-motion data.** Contact-free trajectories for nominal actuator dynamics.

Weight-labeled data collection — **Weight-labeled data.** Payload tasks for repeatable gravity-induced force labels.

Mechanically restricted Joint 3 data collection — **Mechanically restricted data.** Pick-and-place trajectories with Joint 3 under controlled rubber-band resistance.

Force sensor data collection — **Force-sensor data.** Directional interaction trials with an external force/torque fixture.

OpenManipulator-X motor — **Robotic arm actuator.** Dynamixel motor used in the OpenManipulator-X leader-follower system.

**Robot gripper schematic.** Top and side views map motor rotation to the single-finger slide coordinate used by the simulator.

Gauge Pushing

This independent evaluation benchmark is separate from NAD training supervision. It isolates contact transitions from the payload tasks above, and the predicted force tracks the gauge measurement through contact onset, sustained pushing, and release.

Architecture and Adaptation

The final checks compare temporal model choices, show rapid online calibration from a small batch of new trajectories, and verify that NeuralActuator can be inserted into another differentiable physics backend.

Architecture ablation. Transformer history modeling improves force prediction while remaining competitive on rollout accuracy. Joint errors are in degrees, Grip is single-finger slide-coordinate MAE in millimeters, and Force is in Newtons.

Model	J1	J2	J3	J4	Grip	Force
MLP	4.55	5.81	3.48	2.16	0.91	0.47
GRU	1.83	2.08	1.68	1.70	0.65	0.49
LSTM	2.91	7.66	3.21	3.08	0.71	0.41
NeuralActuator	1.78	3.31	2.01	1.58	0.65	0.23

Online learning progress plot — **Online adaptation.** Rapid fine-tuning from a small batch of newly collected trajectories.

Warp simulation results — **Warp simulation.** NeuralActuator integrated with another differentiable physics backend.

Quantitative Results

Rollout accuracy, force prediction, adaptation, and cross-platform tests.

Dynamics and Force Prediction

Gradient behavior across rollout horizons. Input-grad is the norm of the loss gradient at the simulator's bounded generalized-effort input; cosine similarity is computed against the parameter gradient at H=128.

H	Input-grad	\|\|grad_θ L\|\|	Cos.
64	2.73e-2	8.77	0.96
128	2.59e-2	17.33	1.00
256	1.72e-2	22.35	0.99
320	1.45e-2	22.91	0.99
500	9.37e-3	20.29	0.98

Simulation accuracy on the test set. J1-J4: joint-angle MAE (deg). Grip: single-finger slide-coordinate MAE (mm).

Task	@100					@300					@600
	J1	J2	J3	J4	Grip	J1	J2	J3	J4	Grip	J1	J2	J3	J4	Grip
backward_forward	2.4	4.3	4.4	4.2	0	2.3	5.9	3.9	4.2	0	3.1	4.8	4.9	4.5	0
circular_ccw	1.7	5.2	2.2	2.6	0	1.5	3.1	2.0	3.4	0	2.5	2.3	2.1	2.8	0
circular_cw	3.6	3.2	1.4	2.2	0	3.1	1.9	1.2	2.6	0	2.6	1.7	2.0	2.8	0
go_up_stay_still	2.5	1.8	2.3	1.2	0	3.1	2.1	3.1	2.5	0	3.1	1.5	3.5	2.7	0
joint_sweep_1	2.3	2.5	1.8	1.4	0	1.9	1.0	2.6	1.8	0	2.9	1.9	2.0	1.6	0
joint_sweep_2	2.1	4.3	4.6	1.2	0	1.8	6.7	8.4	1.1	0	2.4	4.6	5.2	1.7	0
joint_sweep_3	3.7	4.5	3.1	1.9	0	3.4	2.6	2.8	2.8	0	3.1	2.6	2.5	3.4	0
joint_sweep_4	2.6	4.8	2.5	3.4	0	2.9	3.3	2.4	3.2	0	2.4	2.9	2.2	4.6	0
joint_sweep_5	1.7	3.0	1.7	3.1	0.4	2.6	3.9	2.3	2.7	0.5	5.5	3.3	5.1	3.0	0.7
pick_place_empty	2.6	2.5	2.2	3.5	1.1	1.9	3.6	2.7	3.2	1.1	3.1	2.7	2.0	3.9	1.0
Average	2.5	3.6	2.6	2.5	0.2	2.5	3.4	3.1	2.8	0.2	3.1	2.8	3.2	3.1	0.2

Simulation and force prediction accuracy on the force-sensor test set. F: force MAE (N).

Task	@100						@300						@500
	J1	J2	J3	J4	Grip	F	J1	J2	J3	J4	Grip	F	J1	J2	J3	J4	Grip	F
force_X+	0.9	2.3	0.9	1.6	1.0	0.36	1.1	1.8	2.4	1.3	1.0	0.46	0.9	1.9	2.7	1.7	1.0	0.50
force_X-	1.4	1.1	0.8	1.1	1.0	0.39	1.6	2.3	1.3	2.3	1.0	0.42	2.0	2.9	1.5	3.0	1.0	0.44
force_Y+	2.3	5.7	2.8	0.6	0.1	0.36	2.1	4.5	2.6	0.9	0.2	0.38	2.7	4.8	2.1	0.9	0.3	0.44
force_Y-	1.4	1.6	3.1	0.5	0.1	0.38	1.5	3.0	3.1	1.3	0.1	0.36	1.4	3.6	3.1	1.5	0.2	0.38
force_Z+	3.3	6.2	3.3	0.8	0.1	0.39	2.3	5.6	2.5	1.9	0.2	0.43	2.5	4.4	2.2	2.3	0.3	0.46
force_Z-	1.0	5.9	0.6	0.6	1.0	0.36	2.9	6.6	2.8	2.5	1.0	0.64	3.8	6.4	3.5	2.0	1.0	0.57
ref_X+	0.6	2.0	1.5	1.2	1.0	0.01	0.8	1.8	1.0	1.1	1.0	0.02	1.0	1.6	1.1	1.1	1.0	0.01
ref_X-	2.0	2.5	0.5	1.1	1.0	0.00	1.4	2.3	0.5	1.0	1.0	0.00	1.5	2.2	0.8	1.3	1.0	0.00
ref_Y+	2.0	1.0	2.0	0.6	0.1	0.00	2.1	3.3	2.1	1.1	0.2	0.00	1.9	3.7	1.7	1.1	0.4	0.00
ref_Y-	1.0	1.8	1.2	1.3	0.1	0.00	1.6	4.5	2.0	2.2	0.1	0.00	1.2	4.0	2.4	1.6	0.3	0.00
ref_Z+	1.2	1.7	2.1	0.5	0.1	0.00	1.6	3.8	2.2	1.2	0.2	0.00	1.5	2.9	2.0	1.3	0.3	0.00
ref_Z-	0.6	2.3	0.9	0.9	1.0	0.02	0.7	1.6	1.1	1.1	1.0	0.01	0.9	1.3	1.0	1.1	1.0	0.01
Avg	1.48	2.84	1.64	0.90	0.55	0.19	1.64	3.43	1.97	1.49	0.58	0.23	1.78	3.31	2.01	1.58	0.65	0.23

Simulation and force prediction accuracy on the weight-based test set.

Task	Weight	@100						@300						@600
		J1	J2	J3	J4	Grip	F	J1	J2	J3	J4	Grip	F	J1	J2	J3	J4	Grip	F
go up and stay	200g	1.3	5.0	2.6	2.4	0.1	0.12	3.4	3.9	4.2	2.3	0.1	0.10	3.1	3.3	4.3	2.9	0.1	0.16
go up and stay	300g	2.6	5.3	5.7	3.7	0.1	0.20	1.8	3.4	5.7	5.4	0.0	0.17	3.5	2.5	3.4	4.6	0.0	0.20
go up and stay	400g	2.2	4.1	1.4	5.0	0.1	0.07	2.9	2.1	2.1	6.1	0.1	0.12	2.0	1.4	3.0	7.1	0.2	0.11
pick and place	200g	0.7	3.1	4.3	3.4	1.1	0.24	2.7	3.9	4.0	4.0	1.1	0.08	2.9	5.1	2.9	3.0	1.0	0.11
pick and place	300g	0.5	4.5	3.7	0.9	0.3	0.00	1.2	5.1	4.8	1.5	0.2	0.00	2.7	7.8	4.9	3.2	0.3	0.09
pick and place	400g	0.8	2.1	2.8	2.9	1.1	0.19	2.8	1.4	3.4	3.6	1.1	0.06	2.4	2.0	3.8	3.3	1.0	0.03
pick and place	500g	2.7	1.8	1.7	3.3	1.1	0.05	4.8	2.8	1.7	2.3	1.1	0.02	4.2	6.3	2.3	2.3	0.9	0.04
Avg		1.54	3.70	3.17	3.09	0.56	0.12	2.80	3.23	3.70	3.60	0.53	0.08	2.97	4.06	3.51	3.77	0.50	0.11

Dataset and Adaptation

Full NAD trajectory accounting. NAD contains 350 OpenManipulator-X task assignments and 100 SO-101 task assignments, for 450 assignments in total. Twenty nominal OpenManipulator-X condition assignments reuse trajectories listed in other categories, leaving 430 distinct trajectories. The OpenManipulator-X table below includes the full 100-500 g payload range; the earlier 94.52-minute summary covers only the model-development and evaluation subset used in the reported experiments.

Category	Task	Variant	#Frames	Duration (s)
Free motion	Circular trajectory	Clockwise	8615	147.06
Free motion	Circular trajectory	Counterclockwise	8428	143.90
Free motion	Joint sweep	Motor 1	22935	392.04
Free motion	Joint sweep	Motor 2	8688	148.47
Free motion	Joint sweep	Motor 3	11357	193.95
Free motion	Joint sweep	Motor 4	15012	256.57
Free motion	Joint sweep	Motor 5	7338	125.21
Free motion	Lean back and extend forward		12261	209.35
Free motion	Pick & place (empty)		15288	261.08
Free motion	Go up and stay still		10011	171.14
Force-labeled	Go up and stay still	100g	10976	187.61
Force-labeled	Go up and stay still	200g	10606	181.26
Force-labeled	Go up and stay still	300g	11259	192.40
Force-labeled	Go up and stay still	400g	11291	192.89
Force-labeled	Pick and place	100g	12483	213.35
Force-labeled	Pick and place	200g	13245	226.15
Force-labeled	Pick and place	300g	13129	224.31
Force-labeled	Pick and place	400g	15124	258.54
Force-labeled	Pick and place	500g	13957	238.49
Force-labeled	Force sensor	+X	5005	85.17
Force-labeled	Force sensor	+X w/o interaction	5005	85.21
Force-labeled	Force sensor	-X	5169	87.94
Force-labeled	Force sensor	-X w/o interaction	5169	88.00
Force-labeled	Force sensor	+Y	7444	126.74
Force-labeled	Force sensor	+Y w/o interaction	7444	126.78
Force-labeled	Force sensor	-Y	6921	117.88
Force-labeled	Force sensor	-Y w/o interaction	6921	117.91
Force-labeled	Force sensor	+Z	6906	117.59
Force-labeled	Force sensor	+Z w/o interaction	6906	117.63
Force-labeled	Force sensor	-Z	5565	94.69
Force-labeled	Force sensor	-Z w/o interaction	5565	94.73
Motor condition	Pick & place w/ weight	Mechanically restricted	10113	168.26
Motor condition	Pick & place w/ weight	Unrestricted operation	13245	226.15
Motor condition	Pick & place w/o weight	Mechanically restricted	11252	192.41
Motor condition	Pick & place w/o weight	Unrestricted operation	15288	261.08

Visual-supervision and architecture settings.

Group	Metric / Model	Value 1	Value 2	Value 3	Value 4	Value 5
Silhouette IoU	Initialization	Mean 0.2589	Std 0.0441
Silhouette IoU	Robot refinement	Mean 0.2974	Std 0.0754
Silhouette IoU	Joint refinement	Mean 0.8515	Std 0.0065
Runtime	Joint refinement	1845 iterations	180.23 s
Architecture	MLP	Hidden 416	Latent 208	1.14M params	LR 3e-5	Clip 0.3
Architecture	GRU	Hidden 325	Latent 162	1.43M params	LR 1e-4	Clip 1.0
Architecture	LSTM	Hidden 275	Latent 137	1.44M params	LR 1e-4	Clip 1.0
Architecture	NeuralActuator	Hidden 192	Latent 96	1.44M params	LR 1e-4	Clip 1.0

Online adaptation before (B) and after (A). J1-J4: joint-angle MAE (deg). Grip: single-finger slide-coordinate MAE (mm). F: force MAE (N).

Task	@100												@300												@500
	J1 B	J1 A	J2 B	J2 A	J3 B	J3 A	J4 B	J4 A	Grip B	Grip A	F B	F A	J1 B	J1 A	J2 B	J2 A	J3 B	J3 A	J4 B	J4 A	Grip B	Grip A	F B	F A	J1 B	J1 A	J2 B	J2 A	J3 B	J3 A	J4 B	J4 A	Grip B	Grip A	F B	F A
force_X+	0.9	0.3	2.1	0.7	1.0	0.8	0.8	0.8	1.0	1.0	0.36	0.37	2.1	1.2	2.3	1.0	0.7	1.5	0.9	0.8	1.0	1.0	0.46	0.41	2.7	1.6	4.0	1.3	0.6	1.9	0.9	0.8	1.0	1.0	0.52	0.41
force_X-	0.2	0.5	1.1	1.7	0.9	1.2	0.7	1.3	1.0	1.0	0.39	0.39	1.7	0.7	1.6	1.6	1.4	0.7	0.8	1.2	1.0	1.0	0.41	0.40	2.5	0.9	1.9	1.6	2.0	0.8	0.9	1.2	1.0	1.0	0.42	0.41
force_Y+	2.1	1.8	5.4	3.4	0.8	1.1	1.2	1.3	0.0	0.1	0.37	0.37	2.3	1.5	4.5	3.2	1.1	1.1	1.6	1.0	0.1	0.1	0.39	0.37	3.0	1.2	3.7	2.3	2.5	1.2	3.1	1.7	0.3	0.3	0.45	0.39
force_Y-	1.1	1.2	0.6	1.3	1.1	1.8	1.8	1.8	0.1	0.1	0.38	0.38	1.6	1.0	3.1	2.6	1.2	1.4	2.1	2.0	0.1	0.1	0.36	0.36	1.7	1.0	2.3	2.0	2.5	1.1	2.1	1.6	0.2	0.2	0.38	0.37
force_Z+	2.9	4.0	5.2	3.6	1.3	3.1	0.6	0.5	0.1	0.1	0.39	0.39	2.7	2.1	4.0	2.7	1.4	1.5	0.7	0.9	0.2	0.1	0.44	0.42	5.0	1.8	3.2	2.0	1.7	1.5	1.4	1.5	0.3	0.2	0.46	0.45
force_Z-	0.7	0.9	6.2	3.5	2.0	0.6	1.0	0.7	1.0	1.0	0.36	0.36	2.0	0.7	7.1	2.3	1.8	0.8	3.3	1.2	1.0	1.0	0.61	0.38	2.6	1.0	7.2	1.6	2.0	0.8	3.0	1.1	1.0	1.0	0.54	0.40
ref_X+	1.4	0.4	1.6	0.4	1.0	1.0	0.8	0.8	1.0	1.0	0.01	0.08	1.5	0.5	1.3	0.6	0.8	0.8	1.0	1.2	1.0	1.0	0.02	0.08	1.3	0.5	1.1	0.6	0.6	1.0	1.4	1.9	1.0	1.0	0.01	0.05
ref_X-	0.4	0.7	3.2	0.6	2.4	0.5	0.7	1.0	1.0	1.0	0.00	0.00	0.7	0.6	2.4	1.2	1.9	0.5	0.9	0.9	1.0	1.0	0.00	0.04	0.8	0.6	2.2	1.0	2.2	0.7	1.1	0.8	1.0	1.0	0.00	0.05
ref_Y+	1.4	1.8	1.0	2.8	1.4	1.4	1.3	1.9	0.1	0.1	0.00	0.00	2.2	1.7	3.3	3.3	1.0	1.2	1.8	1.4	0.2	0.2	0.00	0.01	2.2	1.2	4.4	2.4	1.0	1.1	1.3	1.3	0.4	0.3	0.00	0.01
ref_Y-	2.0	1.2	2.2	1.9	0.8	0.4	2.7	1.3	0.0	0.1	0.00	0.01	2.5	1.5	4.8	2.7	0.7	0.8	3.2	1.8	0.1	0.1	0.00	0.01	2.0	1.2	4.9	2.1	0.7	1.3	2.7	2.0	0.3	0.2	0.00	0.01
ref_Z+	0.6	0.9	1.3	0.8	0.4	1.3	1.4	1.3	0.1	0.1	0.00	0.00	1.3	1.2	3.4	1.6	0.4	1.0	1.2	1.0	0.2	0.2	0.00	0.00	1.0	1.1	3.4	1.5	0.5	1.2	0.9	1.3	0.3	0.3	0.00	0.00
ref_Z-	1.8	0.5	1.9	0.8	1.1	0.9	0.5	0.7	1.0	1.0	0.03	0.14	1.7	0.6	0.9	0.7	1.1	0.8	0.8	0.8	1.0	1.0	0.02	0.11	1.6	0.5	0.8	0.7	1.0	0.7	0.8	1.1	1.0	1.0	0.01	0.08
Avg	1.28	1.19	2.65	1.81	1.19	1.17	1.13	1.12	0.55	0.55	0.19	0.21	1.86	1.09	3.22	1.96	1.11	1.02	1.53	1.18	0.59	0.58	0.23	0.22	2.20	1.04	3.25	1.58	1.44	1.10	1.64	1.35	0.66	0.65	0.23	0.22

Cross-Platform Evaluation

SO-101 arm evaluation. Joint errors are in degrees and force errors are in Newtons.

Platform	Task	Lift & Hold / @500						Pick & Place / @500
		J1	J2	J3	J4	J5	F	J1	J2	J3	J4	J5	F
SO-101	300g	2.27	8.20	6.40	5.40	2.37	0.64	3.73	5.63	4.17	6.17	2.47	0.73
SO-101	400g	2.83	9.30	9.63	4.00	5.73	0.57	3.53	6.83	2.43	5.23	5.13	0.63
SO-101	500g	2.27	7.63	9.77	7.13	2.67	0.47	3.80	5.60	1.53	6.73	3.33	0.54

Franka Panda external-force output MAE. Values are in Newtons for 100- and 500-step future-state-conditioned offline rollouts. Only f_z is supervised; the three-component metric uses zero lateral references and the nominal payload reference [0, 0, -mg]^T. This is not torque or online dynamics validation.

Payload	F @100 (N)	F @500 (N)
200g	0.42	0.31
300g	0.35	0.28
400g	0.36	0.27
500g	0.27	0.26
600g	0.30	0.28
Avg	0.34	0.28

Gauge pushing test set.

Task	@100						@300						@600
	J1	J2	J3	J4	Grip	F	J1	J2	J3	J4	Grip	F	J1	J2	J3	J4	Grip	F
high_push_front	0.91	1.68	1.18	0.62	0.00	0.07	1.62	1.29	0.53	0.32	0.00	0.08	1.48	1.16	1.44	1.09	0.00	0.09
high_push_top	0.73	1.01	0.78	1.61	0.01	0.10	1.38	1.35	1.97	1.92	0.01	0.11	1.62	1.57	2.72	2.01	0.01	0.12
mid_push_front	0.66	0.52	0.96	0.61	0.30	0.06	0.85	0.51	0.60	0.99	0.31	0.07	0.95	0.56	0.48	0.98	0.31	0.08
mid_push_top	0.49	0.54	0.87	0.74	0.30	0.09	0.64	0.43	0.73	1.07	0.31	0.10	0.89	0.52	0.75	0.78	0.31	0.11
low_push_front	0.61	0.75	1.83	0.23	0.32	0.08	0.87	0.56	1.65	0.33	0.32	0.09	1.29	0.88	1.12	0.98	0.33	0.10
low_push_top	1.00	0.36	1.05	0.29	0.31	0.08	1.17	0.84	2.11	0.74	0.32	0.09	1.35	0.76	2.27	1.04	0.32	0.10
Avg	0.73	0.81	1.11	0.68	0.21	0.08	1.09	0.83	1.27	0.90	0.21	0.09	1.26	0.91	1.46	1.15	0.21	0.10

Future Directions

Scaling actuator learning across data, robot morphologies, and force-aware control.

01 Synthetic-to-Real Actuator Pretraining

Leverages large-scale synthetic actuator data for pretraining, followed by real-world fine-tuning, to reduce costly hardware data collection.

02 Cross-Morphology Actuation Learning

Extends NeuralActuator across robot morphologies and actuator families for more generalizable actuation modeling.

03 Scalable Whole-Body Force Perception

Moves beyond single-arm end-effector force estimation toward multi-joint and whole-body force-aware robot control.

Citation

@INPROCEEDINGS{DouZ-RSS-26,
    AUTHOR    = {Zhiyang Dou AND John U. Onyemelukwe AND Hangxing Zhang
                 AND Heng Zhang AND Minghao Guo AND Yunsheng Tian
                 AND Michal Piotr Lipiec AND Joshua Jacob AND Chao Liu
                 AND Peter Yichen Chen AND Yuri Ivanov AND Wojciech Matusik},
    TITLE     = {{NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perception}},
    BOOKTITLE = {Proceedings of Robotics: Science and Systems},
    YEAR      = {2026},
    ADDRESS   = {Sydney, Australia},
    MONTH     = {July}
}