ANALYSIS Robotics

Robot Hand Starts Writing After an 18-Second Physical Calibration

An ETH Zurich controller learned how finger commands moved a grasped pen without simulation training or demonstrations, achieving a reported 0.64-millimeter mean in-plane error in laboratory tests.

By Maso Intelligence
AI INTELLIGENCE DESK

Published September 14, 2026 at 12:08 AM EDT

Updated September 14, 2026 at 12:08 AM EDT

6 min read

A tendon-driven ORCA robot hand holding a sleeved pen and tracing letters on paper while a webcam-based system tracks the pen tip.

Writing tests more than pen placement

Putting a pen into a robot hand is comparatively easy. Moving that pen accurately with the fingers while preserving a stable grasp is harder. Each finger changes several contacts at once, tendons introduce compliance, and small shifts in the grip can alter how a joint command moves the pen tip. An exact mathematical model of that relationship is difficult to build and may become outdated as the object slips.

Researchers at ETH Zurich have demonstrated a different approach. Their controller estimates the relationship directly from movement observed on the physical robot. After an approximately 18-second initialization, a tendon-driven ORCA hand traced shapes and letters in the air and on paper using the thumb, index finger and middle finger. The wrist and arm did not generate the individual strokes.

The first-version preprint was submitted to arXiv at 16:29 UTC on September 10. Its evidence comes from one physical hand and two simulated hand models. The work has not been peer reviewed or independently replicated. Within that boundary, it presents a useful case for learning a compact control relationship on the machine instead of relying on a detailed contact model or a large training set.

The controller learns a local motion map

The central concept is a task Jacobian. A Jacobian is a mathematical map describing how small changes in the robot's joints affect motion in the task being controlled. Here, the task was the two-dimensional position of the pen tip across the plane of the paper. The physical system controlled ten finger joints while leaving seven joints in the wrist, ring finger and little finger fixed.

The relationship was initially unknown. A webcam observed an ArUco marker attached to the pen and three markers defining the paper's coordinate frame. From those images, the system estimated the pen-tip position, rejected obvious tracking outliers and updated its Jacobian as the fingers moved. The controller then inverted the estimate to calculate joint commands that should carry the tip along the requested path.

This is online system identification rather than reinforcement or imitation learning. The method did not train a policy in simulation, collect demonstrations from a person or require an analytic model of the hand, pen and contacts. It continually revised a small numerical description of how recent commands affected the pen. Computation ran on a laptop CPU, and the camera-limited control loop operated at approximately 15 hertz.

Eighteen seconds provide a starting estimate

Beginning from a blank Jacobian creates a practical problem: the controller does not yet know which finger motions move the pen horizontally or vertically. The researchers addressed that with a predetermined initialization sequence. The hand passed through six manually chosen grip poses along a smooth path for about 15 seconds, followed by roughly three seconds for settling.

Those poses were selected once to move the pen across its available range without releasing it. Observed joint commands and pen motion supplied an initial estimate. Afterward, recursive least squares continued updating the map during writing. A forgetting factor prevented old observations from permanently dominating, allowing the estimate to follow gradual changes in the grasp.

A second mechanism helped the fingers retain the pen. Because the hand had ten controlled joints but the writing task used only two coordinates, multiple finger configurations could produce similar pen-tip motion. The controller used that extra freedom to pull the hand toward its starting grip while minimizing interference with the commanded stroke. Commands were also damped, clipped to joint-speed limits and smoothed before reaching the hardware.

Thirty-eight runs produced submillimeter in-plane errors

The authors evaluated 22 fully configured runs in the air and 16 on paper. Mean in-plane error was approximately 0.62 millimeters in air and 0.67 millimeters on paper. Pooling all 38 runs produced a reported mean of 0.64 millimeters with a run-to-run standard deviation of 0.10 millimeters. The 95th-percentile error was about 1.4 millimeters in both categories.

The paper experiments included eight runs on individual shapes or letters and eight in which the system wrote the word hello letter by letter. A programmed arm motion moved the hand between letters, but the fingers generated each stroke. This distinction matters because the method did not coordinate the arm and hand as one learned system. It solved local pen motion within a fixed grasp, then relied on separate positioning between characters.

A continuous air-writing run lasted about 31 minutes and passed through all 26 Latin letters. Mean error across six time intervals remained between 0.59 and 0.69 millimeters. During the letter K, a sudden finger movement produced an approximately 20-millimeter excursion, but the controller returned to the path without restarting its calibration. One long run supports sustained adaptation in that setup, not a general reliability rate.

Ablations show why continued adaptation matters

Tests with components removed clarified the result. When the Jacobian was frozen immediately after initialization, two of three runs diverged from the path and the remaining run degraded to about one millimeter of mean error. Freezing it after approximately 12 seconds of writing also produced one divergence and one degraded run. Four runs frozen after about 30 seconds completed with a mean near 0.62 millimeters, suggesting that interaction after initialization substantially improved the estimate.

Grip stabilization was also important. With the posture term disabled, four of ten runs failed, two more degraded severely and only four finished near the fully configured baseline. These experiments were small and performed on the same platform, but they support the authors' claim that both adaptation and grip management contributed to the demonstration.

The same estimator and controller structure was tested in MuJoCo with the Shadow Hand and Wuji Hand 2. Those simulations controlled the pen in three dimensions and produced reported root-mean-square errors of 0.17 and 1.48 millimeters. They used noise-free simulator states and implementation changes described in the public repository. They therefore offer early portability evidence, not confirmation on two additional physical hands.

The uncontrolled dimension defines the limit

The headline accuracy covers motion parallel to the paper. Vertical pen-tip movement was not controlled and was excluded from the reported in-plane metric. The physical runs showed approximately two to three millimeters of vertical drift on paper. To maintain contact, the researchers folded or raised the paper by roughly one centimeter so it could flex beneath the pen.

That accommodation means the current system cannot write on an ordinary rigid surface under the same conditions. It also cannot release and regrasp the pen because its learned map assumes continuous contact. Disconnected characters, such as a lowercase i, require separate movement or arm assistance. The pen itself wore a compliant sleeve that increased its effective diameter to about four times that of the bare pen, making the grasp more secure.

Speed presents another boundary. The reference motion was only 0.8 millimeters per second. Even doubling that speed produced inconsistent outcomes across two trials, while progressive speed increases caused large transients or divergence. Accuracy was measured through the same marker-and-camera pipeline used for control rather than against an independent measurement of deposited ink, so calibration error and marker jitter carry into the reported values.

Writing is valuable here because it makes small tracking failures visible, not because handwriting is necessarily the controller's final application. A compact method that adapts on physical hardware could eventually help robots manipulate tools or assistive devices when contact models and demonstrations are scarce. This preprint does not establish those uses. It shows a narrower form of progress: a complex hand can learn enough about one continuous grasp in seconds to perform precise, measurable motion with modest computing resources.

Evidence status

MEDIUM

Primary documents reviewed. Company claims and independent evidence are identified in the article.

Key claims

The physical ORCA hand began controlled pen motion after an approximately 18-second initialization sequence.
The controller estimated a task Jacobian online from finger commands and camera-observed pen motion without simulation training, prerecorded demonstrations or an analytic hand-contact model.
Across 22 in-air and 16 on-paper runs, the authors report a pooled mean in-plane pen-tip error of approximately 0.64 millimeters.
A single approximately 31-minute air-writing run traced all 26 Latin letters without an upward error trend, while recovering from one large transient during the letter K.
Freezing the Jacobian early or removing grip stabilization increased degradation and failures in small ablation tests.
The physical accuracy metric excluded approximately two to three millimeters of vertical drift, which flexible raised paper absorbed.
The work is a first-version preprint evaluated on one physical hand, with two additional hand models tested only in simulation.

Primary sources

Inspect the evidence

PRIMARY · Preprint abstract and submission history

Rapid in-hand writing submission record

PRIMARY · Full research preprint

Complete rapid in-hand writing manuscript and Figure 1 provenance

PRIMARY · Author project page and demonstration media

Rapid dexterous writing project page

PRIMARY · Author code repository and simulation documentation

Jacobian dexterity implementation

PRIMARY · Research figure asset

Figure 1 image asset

PRIMARY · Official license terms

Creative Commons Attribution 4.0 legal code

Event timeline

2026-09-10

The first version of the rapid in-hand pen-writing preprint was submitted to arXiv at 16:29 UTC.

Related event

2026-09-10

Researchers Post Rapid In-Hand Robot Writing Preprint

An ETH Zurich team submitted a first-version preprint reporting an online Jacobian-estimation controller that began physical pen writing after an approximately 18-second calibration.

Open source

Corrections

This version was last updated September 14, 2026 at 12:08 AM EDT. The newsroom retains the article’s revision history.

Disclosure

AI-written reporting by Maso Intelligence for The Expectancy, based on cited public primary sources. Results and performance measurements are attributed to the study authors; the work is a preprint without independent replication.