Laya MLX: 7–14 ms Typed Decisions on an M3 Max
The MLX port is the native Apple-silicon runtime for Laya: the same 421M parameter decision model, converted to Apple's array framework, reporting 7–14 milliseconds for short decisions on an M3 Max. It sits between the plain CPU install and the tuned Neural Engine path.
Why people pick MLX over Core ML
- Modification is cheap. MLX weights are easy to load, inspect, quantise to FP16 or lower, and re-test — which is what you want while you are still iterating on question wording and thresholds.
- It follows the research runtime. New checkpoint versions land in MLX early, so you are not waiting for a conversion pipeline.
- Mac-only is usually acceptable. If the decision only needs to run on your development machines or a Mac app, MLX has fewer moving parts than a hand-tuned Core ML target.
Reported numbers
| Configuration | Latency, short decision |
|---|---|
| MLX, M3 Max, FP16 | 7–14 ms |
| Core ML, Neural Engine | ~5 ms |
| Reference Python on CPU | Tens of ms |
| Hosted frontier model | Sub-second including network |
The published comparison also notes the MLX path lands well ahead of a comparable FP16 baseline conversion — the port exists because the naive conversion was slower, not just to have a logo in the README.
Working habits that hold
- One process, one loaded model. Cold-start is the largest single cost in every local setup.
- Keep the question set in one file. Each question, each criterion and each threshold in a single place, so the eval script and the app cannot drift apart.
- Log the checkpoint with every decision. Local models change quietly; a decision you cannot attribute to a version is a decision you cannot debug.
- Escalate on low confidence. The cheap local pass should hand uncertain cases upward, not force an answer.
For an iOS or always-on desktop feature where power matters, prefer the Core ML port. For servers, CI and everything non-Apple, use the standard local install. And for the wider picture of how these numbers compare to the hosted model, see Laya vs Jev.
Last updated: 2026-09-21 · sources & corrections