Audience: This article is for ML engineers, firmware leads, and OEM product managers evaluating the on-device handwriting recognition (HTR) engine shipping on the L5, Q8 and T2 reading pens. We assume you understand the difference between online and offline recognition, you know what a CTC decoder does, and you want a concrete engineering reference for the model architecture, accuracy, latency, and memory footprint inside the 11-model lineup.
A reading pen that recognizes printed text uses OCR. A reading pen that recognizes handwriting uses HTR — a fundamentally different problem because the input is a sequence of (x, y, t) stroke samples, not a static image. The L5, Q8 and T2 include an HTR engine as a premium feature, primarily used for math input, blank-fill exercises, and short-answer validation in workbooks. This article covers the engine, the on-device model architecture, and the numbers you need to write a defensible OEM specification.
1. Why OCR and HTR Are Different Problems
The OCR engine consumes a static binarized image and produces a character sequence. The HTR engine consumes a sequence of pen-tip coordinates over time and produces a character sequence. The differences cascade through the entire pipeline:
- Input modality: 2D image (OCR) vs 1D temporal stream (HTR).
- Input size: 640 x 480 x 1 byte per line (OCR) vs 200 to 500 samples per stroke (HTR).
- Pre-processing: binarization and connected-component (OCR) vs resampling and smoothing (HTR).
- Model architecture: 2D CNN + RNN + CTC (OCR) vs 1D temporal CNN + BiGRU + CTC (HTR).
- Output: full word recognition (OCR) vs incremental character-by-character (HTR).
The two engines share some code (the CTC decoder, the dictionary lookup, the language model) but the front-end and middle layers are independent. We do not run HTR on the B1, B2, F8, P2, K7, KF1, Q6 and Y6 because those models lack the 2D touch sensor and the CPU budget for a temporal model.
2. Hardware Boundary and Which Models Run HTR
HTR runs only on the matrix-touch models with sufficient compute: L5, Q8 and T2. Each of the three uses a different SoC, so the HTR model is compiled per-platform.
| Model | SoC | CPU cores | DSP/NPU | RAM available to HTR | Flash for model |
|---|---|---|---|---|---|
| L5 | Cortex-A53 x4 @ 1.2 GHz | 4 | none (CPU only) | 96 MB | 8 MB |
| Q8 | Cortex-A53 x4 @ 1.4 GHz | 4 | 0.5 TOPS NPU | 192 MB | 12 MB |
| T2 | Cortex-A55 x4 @ 1.8 GHz | 4 | 1.2 TOPS NPU | 384 MB | 18 MB |
The L5 runs the HTR model on the CPU cores because it has no NPU. The Q8 and T2 can run on either the CPU or the NPU; we ship the CPU build by default and the NPU build as an OEM option for buyers who need lower latency.
3. The Stroke Capture Pipeline
The HTR engine begins with the touch controller reporting (x, y, t) samples. The samples are already timestamped by the touch IC. The host MCU routes the samples into a ring buffer of 1,024 entries, which is enough for 8 seconds of continuous writing at 120 Hz.
Pre-processing steps run in real time as the samples arrive:
- Smoothing: a 3-tap moving average on x and y. Removes sensor jitter.
- Dehooking: removes the initial "hook" stroke that occurs when the stylus first touches the surface. A hook is a fast 30 mm to 50 mm arc within the first 30 ms.
- Resampling: resample the stroke to a fixed 80 points regardless of original speed. This is what makes the model size-independent of writing speed.
- Normalization: translate centroid to origin and scale to fit a 20 mm by 20 mm bounding box.
- Feature extraction: compute dx, dy, dx², dy² at each resampled point, plus pen-up flag. Feature vector is 5 values per point.
The 80-point x 5-feature vector is fed to the model. Pre-processing adds 4 ms latency on the L5.
4. The Model Architecture
The HTR engine uses a 1D temporal CNN followed by a 2-layer bidirectional GRU, followed by a CTC decoder. The architecture is identical across L5, Q8 and T2; only the per-platform weight quantization differs.
The temporal CNN has 3 layers:
- Conv1D(5 → 32, kernel 5, stride 1) + batch norm + ReLU.
- Conv1D(32 → 64, kernel 5, stride 2) + batch norm + ReLU.
- Conv1D(64 → 128, kernel 3, stride 2) + batch norm + ReLU.
Output of the CNN is a 20-step sequence with 128 channels per step (80 input points downsampled by 4).
The 2-layer bidirectional GRU has 128 hidden units per direction, giving 256 units per step. The GRU runs in both directions and the outputs are concatenated.
The CTC decoder is a single linear layer of 256 → 96 units (94 printable ASCII characters plus blank plus pad). It outputs a probability distribution over 96 symbols at each of the 20 time steps. The CTC loss function aligns the 20-step output to the target character sequence during training.
Total parameters: 412,000 weights for the CNN + 296,000 for the GRU + 24,700 for the CTC head = 732,700 parameters.
On disk, the int8 quantized model is 1.84 MB. On the L5 it expands to 1.92 MB at runtime (with batch-norm folded). On the Q8 and T2 with NPU acceleration, the int8 model runs as a single 1.84 MB blob.
5. Training Data and Accuracy
The model is trained on a corpus of 2.4 million handwriting samples collected from 4,800 writers across 14 countries. Writers included left-handed and right-handed, ages 5 to 70. Each writer produced 500 to 800 samples of single characters, short words, and short numeric expressions. The dataset is balanced across English, Spanish, French, German, Italian, Portuguese, Arabic, Chinese, Japanese, Korean, Russian, Hindi, Thai, and Vietnamese.
Character-level accuracy on a held-out test set of 80,000 samples (not in the training set):
| Script | L5 accuracy | Q8 accuracy | T2 accuracy |
|---|---|---|---|
| English (Latin) | 96.4% | 97.8% | 98.4% |
| Spanish | 96.1% | 97.6% | 98.2% |
| French | 95.7% | 97.3% | 98.0% |
| German | 95.4% | 97.0% | 97.7% |
| Arabic | 92.1% | 94.8% | 95.9% |
| Chinese | 93.8% | 95.9% | 96.7% |
| Japanese (kana) | 94.0% | 96.0% | 96.9% |
| Korean (Hangul) | 93.6% | 95.7% | 96.5% |
| Hindi (Devanagari) | 92.8% | 95.1% | 96.1% |
| Thai | 91.9% | 94.4% | 95.5% |
| Vietnamese | 95.2% | 97.0% | 97.8% |
Word-level accuracy on a separate 10,000-word test set (English) is 92.4% on the L5, 94.8% on Q8, and 95.7% on T2. Word accuracy is always lower than character accuracy because a single mis-recognized character can invalidate a word.
6. The CTC Decoder and Language Model
The CTC decoder outputs a sequence of (symbol, probability) pairs. We collapse repeats and remove blanks to get a candidate character sequence. Then a 4-gram language model rescores the top 5 candidates and picks the highest-scoring one.
The language model is a compact KenLM model with 32 million n-grams, occupying 18 MB on disk and 24 MB in RAM at runtime. The LM is English-only on the L5; the Q8 and T2 include separate LMs for Spanish, French, German, Italian, Portuguese, Arabic, Chinese, Japanese, Korean, Russian, Hindi, Thai, and Vietnamese, totaling 187 MB across all 13 languages.
For OEM buyers who want a custom LM (for example, a publisher-specific vocabulary), we can train a domain LM from a customer-supplied corpus of as few as 50,000 sentences. The training cycle takes 6 hours on a single A100 GPU. The custom LM replaces the general LM in firmware and adds 4 to 8 MB depending on vocabulary size.
7. Model Quantization and Platform Variants
We ship three quantization variants:
- float32: 2.93 MB on disk, 3.04 MB runtime, 32 ms CNN latency on L5. Used only for debugging.
- int8 per-channel: 1.84 MB on disk, 1.92 MB runtime, 32 ms CNN latency on L5. Default for L5 and the CPU build for Q8/T2.
- int8 per-tensor (NPU-friendly): 1.81 MB on disk, 1.81 MB runtime, 10 ms CNN latency on Q8 NPU, 6 ms on T2 NPU.
The NPU-friendly variant rounds each tensor with a single scale factor instead of per-channel scales, which costs 0.3 percentage points of accuracy but enables the 3x to 5x speedup on the NPU. The accuracy drop is recovered by a small calibration step where we run 4,000 samples through the int8 model and adjust the scales.
8. Memory Footprint and Power Consumption
On the L5 with the int8 per-channel model, the HTR engine consumes:
- 1.92 MB model in flash, loaded into 1.92 MB RAM at recognition time.
- 24 MB LM in RAM (English only).
- 1.1 MB ring buffer and pre-processing scratch.
- Total working set: 27 MB RAM, leaving 69 MB headroom on the 96 MB available.
Average power consumption during a recognition operation: 92 mW for 70 ms = 6.4 mJ per word. At 250 mW average scan power, the HTR engine adds about 2.5% to the total system power budget. On the Q8 and T2 the per-recognition energy is 3.4 mJ and 2.1 mJ respectively.
9. AQL on the HTR Calibration Line
Each L5, Q8 and T2 unit runs a 12-second HTR self-test during calibration. The test presents a fixed 32-character string via the touch simulator and checks the recognized output against a known-good reference. Acceptance criteria:
- Exact string match within 70 ms: pass.
- Exact string match within 90 ms: pass with note.
- 1-character error: rework (reflash model, recalibrate).
- 2+ character error or no output within 100 ms: scrap.
Across 8,000 L5 units calibrated in Q1 2026, the first-pass yield was 95.6%, rework rate 3.1%, scrap rate 1.3%. The dominant scrap cause is a mis-flashed int8 model from a corrupted OTA transfer, which we mitigate by a checksum verify on every boot.
10. Specifying HTR in an OEM Contract
A defensible HTR spec includes:
- Model architecture and quantization (we ship int8 per-channel by default).
- Per-platform character-level accuracy on a held-out test set (English ≥ 96%, Chinese ≥ 93%).
- Recognition latency on the target platform (L5 ≤ 80 ms, Q8 ≤ 35 ms, T2 ≤ 25 ms).
- LM size and supported languages.
- Custom LM training SLA (10 business days for orders above 5,000 units).
- AQL pass criteria (exact match within 90 ms on the calibration string).
For more on how the HTR engine integrates with the dictionary lookup, see Reading Pen Dictionary File Format and Edition. For how the OCR engine that recognizes printed text is structured, see Reading Pen OCR Engine vs Deep Learning OCR. For how the touch samples are preprocessed, see Reading Pen Touch Accuracy and Jitter Test. For the OEM spec language we recommend for handwriting acceptance, see Reading Pen OEM and ODM Specification Template. To request a 5-unit sample with a custom LM trained on your corpus, see /request-a-quote.
Related Reading
- Reading Pen OCR Engine vs Deep Learning OCR
- Reading Pen Text Detection and Segmentation
- Reading Pen Touch Accuracy and Jitter Test
- Reading Pen Dictionary File Format and Edition
- Reading Pen Gesture Recognition Algorithm
- Reading Pen Firmware Architecture Explained
Supplementary Reading: HTR for Math and Equation Input
The HTR engine is also used in a math-input mode on the L5, Q8 and T2. In math mode, the model recognizes individual digits and 18 operator symbols, plus 14 Greek letters and 9 geometric shapes. The math-mode vocabulary is smaller (41 symbols vs 96 in the standard HTR vocabulary) so the language model can be a simple bigram instead of a 4-gram, and the LM size drops to 1.4 MB.
Math-mode accuracy on a 5,000-expression test set is 98.7% symbol-level on the T2 and 96.9% on the L5. We use a grammar-aware decoder that constrains the output to valid expression trees, which reduces errors on ambiguous inputs like "x × 2" vs "x · 2" vs "x ÷ 2."
For school math programs targeting K-6, the HTR math mode has been the most-requested feature of the last 18 months. We have shipped HTR math mode to 7 OEM customers across 5 countries, totaling 14,500 T2 units. The average math-mode usage is 8.4 minutes per student per day, which is well within the T2's 14-hour battery life.
Math mode is also the path that consumes most of the 1.2 TOPS NPU on the T2. Sustained math-input at 30 expressions per minute drives the T2 NPU to 78% utilization, with the SoC package temperature rising from 28 °C at idle to 41 °C after 5 minutes of continuous input. Both numbers are well within the 0 °C to 70 °C commercial operating window.