A lab studying small language models
Clayley Labs is an independent research lab. We work on what we're curious about, write about what we find, and publish what doesn't work alongside what does.
§ 01 Research
Currently studying.
Things we've been working on. Each one has a paper or technical note attached.
§ 02 Writing
Notes from the lab. Long form when something works, longer form when it doesn't.
What we're trying. What's failing. The math we've stopped pretending to ignore.
A small lab. A long horizon.
Independent. Long horizon. We publish what works, what doesn't, and the reasoning behind both. The target is a language model that runs anywhere.
§ 03 Approach
↳ One
We prefer slow understanding over fast claims.
A clean explanation of why something doesn't work is worth more to us than a benchmark number that won't survive the next architecture.
↳ Two
Failures are the part of the research we publish first.
The dead ends are the part most papers leave out and most engineers need most. We write them down while the frustration is still fresh.
↳ Three
A new mechanism is the only kind of progress that lasts.
Numbers improve and then get beaten next quarter. Mechanisms reframe the problem and stay reframed.
↳ Four
The most important language models will run on the device in your hand.
Cloud-only intelligence is a halfway state. Eventually the model has to come to where the user is, and the methods that make that possible don't exist yet.
Clayley Labs
Independent research on small language models that run on any device.
© 2026 Clayley Labs · All rights reserved
Independent · Made by hand
A lab studying small language models
Clayley Labs is an independent research lab. We work on what we're curious about, write about what we find, and publish what doesn't work alongside what does.
§ 01 Research
Currently studying.
Things we've been working on. Each one has a paper or technical note attached.
01
Polar weight encoding
A reparameterization of neural network weights into magnitude and direction, where directions are quantized to a single bit. Underway.
§ 02 Writing
Notes from the lab. Long form when something works, longer form when it doesn't.
What we're trying. What's failing. The math we've stopped pretending to ignore.
April 2026
Field Report
Sixty models, three scales, two datasets. The straight-through estimator costs 12%. Post-training quantization to 1-bit produces perplexity 795,000. Binary models need 70% of the steps.
§ 03 Approach
A small lab. A long horizon.
Independent. Long horizon. We publish what works, what doesn't, and the reasoning behind both. The target is a language model that runs anywhere.
↳ One
We prefer slow understanding over fast claims.
A clean explanation of why something doesn't work is worth more to us than a benchmark number that won't survive the next architecture.
↳ Two
Failures are the part of the research we publish first.
The dead ends are the part most papers leave out and most engineers need most. We write them down while the frustration is still fresh.
↳ Three
A new mechanism is the only kind of progress that lasts.
Numbers improve and then get beaten next quarter. Mechanisms reframe the problem and stay reframed.
↳ Four
The most important language models will run on the device in your hand.
Cloud-only intelligence is a halfway state. Eventually the model has to come to where the user is, and the methods that make that possible don't exist yet.
Clayley Labs
Independent research on small language models that run on any device.
© 2026 Clayley Labs