Project Ceylon Development Log — September 24, 2026: 55 Out of 55 Passed. Then We Found the Test Was Wrong.
Yesterday I wrote that "worked once" is no longer one of our engineering states.
Apparently Ceylon took that personally.
Today we got NuGUI through 55 out of 55 requirements.
The compositor was producing reproducible pixels.
Client crash containment worked.
Bad handles were rejected.
The negative tests passed.
The independent review looked good.
For a little while, it looked like we had reached the point where the new NuGUI work could be promoted into the main Ceylon line.
Then we did the reconciliation work.
And storage disappeared.
Not "a disk test was flaky."
Not "one read failed."
Both drives were being rejected.
The really irritating part?
One of the smoke tests had already told us storage passed.
It was wrong.
That, more than anything else, is what today's work was about.
First, the good news: NuGUI really did make a lot of progress
I do not want the later failure to erase what actually worked.
NuGUI's current D2 work is a substantial step forward.
The compositor is no longer just an architectural sketch with a few pieces hanging off it.
We now have bounded per-client resources.
We have shared ring behavior with explicit ordering.
We have damage-carrying presentation.
We have real client-fault containment.
We have stale-handle rejection.
We have visible target output.
And we have been deliberately testing what happens when applications misbehave instead of assuming polite applications forever.
One of today's more satisfying tests involved deliberately crashing a client with a real invalid-instruction fault.
Not once.
Three times.
The compositor survived.
Another client kept running.
The stale client handles were rejected afterward.
That is exactly what I want from the architecture.
If one application falls over, the desktop should not throw itself down the stairs in sympathy.
The test suite also reached 55/55 requirements for the D2 scope.
That is a real milestone.
But it turns out there is a difference between:
"This component passed its requirements."
and:
"This entire set of changes is safe to promote into the operating system."
Today was mostly about that difference.
Promotion is where things got interesting
The D2 work is not a tiny patch.
It spans 29 commits and more than a hundred files.
Along the way, pieces of the low-level development environment changed too: SDK pieces, kernel-facing dependencies, Rust support, launch configuration and some older diagnostic paths.
That does not automatically make the work wrong.
But it does mean promotion cannot be treated like copying one compositor binary into a directory and calling it done.
The question became:
What exactly are we promoting?
And more importantly:
What assumptions changed underneath the part we were actually testing?
That is where storage raised its hand.
The smoke test said storage passed
During reconciliation, both storage devices were rejected.
That should have been impossible according to the smoke result.
So we did what you do when the machine and the test disagree:
we assumed the test was guilty until proven innocent.
It was.
The smoke check was not actually proving that usable volumes had been published.
It was counting the presence of expected-looking log activity.
Those are not the same thing.
A system can print a line that looks encouraging while still failing the actual thing the line was supposed to represent.
That sounds painfully obvious when written down.
It is also exactly the kind of mistake automated testing can hide beautifully because the output is green and everybody wants to move on.
The corrected question is not:
Did storage print the right message?
It is:
Did the expected volumes actually appear and remain usable?
That is what the test should have been asking in the first place.
So today we got one of those wonderful development milestones that does not look good on a feature list:
we found a test that was capable of lying to us.
Honestly, I would rather find that now than after somebody else downloads A1.
Why this matters more than one storage bug
This is not really a story about storage.
It is a story about qualification.
As Ceylon gets larger, it is becoming possible for one subsystem to be completely correct inside its own test boundary while the larger integrated machine is still wrong.
NuGUI can pass its compositor tests.
Storage can have passed yesterday.
Networking can work.
USB can work.
And a change in the platform underneath all of them can still create a machine that is not the same machine we thought we qualified.
That is why I keep coming back to the complete QEMU configuration as an artifact.
The kernel matters.
The SDK matters.
The Rust bindings matter.
The loader matters.
The firmware matters.
The launch command matters.
The virtual devices matter.
The order of those devices matters.
Apparently, even a test script's definition of the word "pass" matters.
Operating systems are very good at punishing vague nouns.
So we stopped promotion
This is the part where it would have been very easy to say:
"NuGUI passed 55/55. Ship it."
We did not.
The D2 work remains valuable.
The qualification evidence remains valuable.
But promotion into the canonical Ceylon tree is being held until the platform ancestry and storage behavior are reconciled properly.
That also means preserving the exact artifacts that produced the qualified results.
If a test result depends on a particular build environment, kernel revision, SDK revision or launcher, then that environment is part of the evidence.
I do not want a future version of me staring at a directory six months from now wondering which mystery toolchain created the binary that everybody once agreed was good.
Future Mike already has enough reasons to be irritated with Current Mike.
No need to manufacture more.
The weirdly good news: the Rust side may be cleaner than expected
While sorting out the platform mismatch, we ran another experiment.
The concern was that different parts of Ceylon might have quietly become dependent on incompatible Rust/platform combinations.
If that were true, cleaning up the toolchain could turn into a major source-code migration.
So we tested the obvious question:
Can the applicable Ceylon components build against one homogeneous platform tuple?
Then we tried the other candidate tuple.
The result was encouraging.
All 30 applicable MicroKit protection domains and 40 host/no_std libraries built on either single tuple with zero source changes.
That does not yet prove that either tuple is the final answer.
Build success is not runtime qualification.
We have learned that lesson enough times now.
But it strongly suggests that the source tree itself is not hopelessly split between two incompatible worlds.
The next step is boot and runtime qualification, including proving that real volumes publish, not merely that storage says something pleasant in the log.
Independent adversarial review is also still part of the closure work.
So the result tonight is:
promising, but not promoted.
There is a phrase I seem to be using a lot lately.
This is what the project looks like now
One thing I have noticed over the last couple of weeks is that the nature of the failures is changing.
Early Ceylon failures were things like:
"Nothing happened."
"Why are there no pixels?"
"Why did the controller never start?"
"Why is this pointer zero?"
Those are hard problems, but they are usually fairly local.
The newer failures are more annoying because the individual components often look healthy.
The compositor works.
The storage driver works.
The build works.
The machine boots.
The test is green.
And then some assumption between those things turns out not to mean what we thought it meant.
That is a sign of a larger system.
Not necessarily a mature one yet.
But definitely a larger one.
The job now is increasingly about making sure the pieces agree with each other.
And because apparently we needed another project...
While the platform work was being sorted out, I also spent some time on the A1 audio side.
The basic synth requirements are becoming more concrete.
One decision that came out of that work is that I want the A1 synth architecture to be able to support VST3-style plugins rather than building a closed little island that becomes obsolete the moment somebody wants to extend it.
That is not the same thing as saying every random desktop plugin will magically run on Ceylon tomorrow.
There is a lot of architecture between those two statements.
The current direction is to isolate third-party plugin execution, keep real-time audio on bounded low-latency paths, and make plugin failure somebody else's problem instead of the audio engine's last words.
There will be more to say about that later.
Tonight I am mostly mentioning it because apparently, while debugging an operating system, I needed something relaxing to do.
So naturally I started designing plugin isolation.
Where we ended the night
NuGUI D2 has real qualification behind it.
The compositor work is substantially stronger than it was yesterday.
Client crashes can be contained.
Visible pixels are being produced.
The D2 requirement set reached 55/55.
Then promotion testing exposed a storage regression.
That exposed a weak smoke test.
That exposed a platform-ancestry question.
And that, in turn, led to an experiment showing that the Rust source may be considerably easier to normalize than we feared.
That is not a bad day's work.
It is not the kind of progress that produces a dramatic screenshot.
But it does make the eventual screenshot mean more.
Because when I finally show a Ceylon desktop and say:
This works.
I do not want that sentence to mean:
"The log looked about right."
I want it to mean we actually checked.
And, increasingly, we're getting better at checking the checker.
Project Ceylon is an independent retro-futurist operating-system project exploring what a modern creative workstation might look like if some of the best ideas of earlier personal computers had continued evolving instead of being discarded.