Laya-MLX

The MLX port that makes Laya a local runtime: typed decisions in milliseconds on a Mac, with no PyTorch, no Transformers runtime and no cloud API in the loop.

13.4 msmedian, one short English decision (Laya 421M)

7.4 msmedian with the multilingual 322M checkpoint

146.8 / 395 q/s50-question throughput (Laya / multilingual)

0output tokens per decision — typed answers only

943 MiBpeak MLX allocation for one short question

Open source Apple Silicon runtime github.com/mizorewww/laya-mlx

What Laya-MLX is

Laya-MLX is the Apple Silicon runtime for open-weight Laya: the same typed-decision model, loaded and executed through MLX instead of PyTorch, so inference stays on the machine. The published benchmark is an M3 Max (40 GPU cores, 128 GiB) with the full pipeline timed — prompt preparation, tokenization, tensors, synchronized inference, calibration and result formatting — and model loading excluded.

The two checkpoints the project ships behave differently enough to matter when you size a machine:

FP16, end-to-end	Laya 421M	Multilingual 322M
One short question, P50	13.42 ms	7.39 ms
One short question, P95	13.92 ms	7.79 ms
50-question throughput	146.8 q/s	395.0 q/s
Peak MLX allocation	943.6 MiB	687.6 MiB

Throughput is measured with batch_size=64 (the API default is 16). Latency depends on question length and count, so treat the table as a machine reference, not a constant.

Run it

pip install laya-mlx

import laya_mlx as laya

agent = laya.load("aac6fef/laya-mlx")
result = agent.predict(
    "I was billed twice. Please refund the duplicate.",
    {"department": {"type": "choice",
                    "instructions": "Who should handle this?",
                    "criteria": ["billing", "technical", "sales"]}},
)
print(result["answers"]["department"])

Apple Silicon, Python 3.11+ and macOS 14+. The first load downloads the checkpoint from Hugging Face; after that inference is fully local, which is the point — no request leaves the laptop and the marginal cost per decision is zero.

The Snake demo is the interesting part

The repository's terminal demo plays Snake with the model in the loop: every move is a typed call, and a visible safety layer can correct an unsafe proposal. Two numbers are worth separating. The demo's frame budget comes from a three-question loop, while the 13.4 ms figure above is the separate one-question API benchmark — mixing them flatters the game.

With the tested compilation and prefix-reuse path (laya-snake --optimize --max-speed) the run reached 75.40 moves/s across 2,400 moves with zero deaths and two visible safety interventions, about 6.5% faster than its same-run eager control on an M3 Max. That is the shape most local decision-model deployments end up with: a fast model plus a cheap deterministic guard, because a wrong label in a game loop is cheap to catch and expensive to obey.

Why it matters for a Jev-style workload

Zero output tokens. A decision returns a label, a score or a calibrated yes/no, so there is nothing to parse and nothing to hallucinate.
Local by construction. Regulated or offline environments can run the same typed questions without shipping data to an API.
Batch economics. At 146–395 questions per second on a laptop, the cost per judgement is electricity rather than per-call pricing.

The project is honest about its limits: figures were measured on one machine (macOS 27.2, Python 3.12.13, MLX 0.32.2), older supported macOS versions were not tested there, and the demo's assisted scores measure the combined model-plus-guard system rather than model skill alone.

Source

Repository: mizorewww/laya-mlx · weights aac6fef/laya-mlx · full method in the repo's BENCHMARKS.md. Licensing and attribution follow the upstream Laya project.

Other repositories in this series

Laya (official package) — Python · 100+ languages
Laya vs Jev (T-Rex arena) — Head-to-head benchmark

Keep browsing: all open-source repos · Jev cases · what Jev is · the Laya guide

Last updated: 2026-09-23 · sources & corrections · every card links to its author's original post