Ceylon Development Log — September 19, 2026: The Controller Was Ready. It Still Wouldn’t Run.
Work continued through approximately 5:00 AM Central, September 20.
Today I submitted Project Ceylon to the Hackaday Retrocomputing Contest.
And then, almost immediately, Ceylon reminded me why I am writing this development journal instead of waiting until everything is polished and pretending the path was obvious.
A controller that should have started did not.
Not crashed.
Not faulted.
Not obviously misconfigured.
It just sat there.
That turned the rest of the day — and quite a bit of the night — into one of those debugging sessions where the useful result is not a dramatic fix.
It is eliminating one wrong explanation after another until the problem finally becomes small enough to understand.
First, the public side
The Hackaday project page got a fairly substantial cleanup today.
I uploaded the current architecture and progress graphics, clarified the staged alpha plan, and added an older framebuffer capture as historical proof that Ceylon has already produced native graphics under QEMU.
That image is the project's earlier First Light milestone.
First Light did not happen today.
The important distinction is that the Hackaday page now has more evidence of what already exists instead of relying mainly on descriptions of where the project is going.
The alpha plan also became clearer.
The first A-series release will go to a small curated group of technical testers. The things they break, misunderstand or expose will be addressed before a broader public QEMU evaluation.
That is slower than throwing an image onto the Internet and calling it an alpha.
I think it is also more useful.
Then the controller refused to start
The main engineering problem today was a controller Protection Domain that never produced its expected startup marker.
The first obvious question was whether I had simply failed to start it.
That would have been convenient.
The generated configuration said otherwise.
The controller had the correct resume flag.
The system initializer could see it.
It was included in the generated startup collections.
Its executable was loaded.
Its scheduling context was present.
The initializer attempted to start it.
The child identifier was valid.
So the easy explanation disappeared.
The machine thought this thing should exist.
The question became:
What does the MicroKernel think the controller is actually doing?
Looking inside the running system
A diagnostic observation finally gave us the controller's kernel-visible state.
Its instruction pointer was sitting exactly at its generated entry point:
0x2025d8
Its stack pointer looked correct.
Its priority was 199.
It was assigned to CPU 0.
Its scheduling context was bound.
Most importantly, its thread state was Restart — a runnable/startable state.
That was a significant result.
The controller was not inactive.
It was not blocked waiting on a message.
It was not sitting in a fault state.
And it had not wandered halfway through startup before failing.
It appeared to have never moved from the front door.
A bad observation can be worse than no observation
The next scheduler snapshot initially looked suspicious.
The controller was still waiting while a higher-priority supervisor was running.
Except the supervisor was still inside its own initialization routine when the snapshot was taken.
That means the observation itself was too early.
A priority-250 thread running while a priority-199 thread waits is not evidence of a scheduler bug.
It is the scheduler doing exactly what it was told to do.
So that result was classified as an observation-timing problem, not a system failure.
That may sound like splitting hairs.
It is not.
One of the easiest ways to manufacture a fake operating-system bug is to instrument the system in a way that changes the thing you are trying to observe.
So we stopped and changed the observation method rather than changing the system.
After initialization, the mystery remained
A valid post-initialization observation came next.
The controller was still sitting at:
PC = 0x2025d8
Still in Restart.
Still ready to go.
But the existing debug interfaces did not expose enough information to answer the next question.
Was the controller actually in the ready queue?
Was the ready bitmap set?
Was its scheduling context really eligible at that moment?
Those details live inside the scheduler.
So I built a temporary diagnostic kernel whose only job was to expose those internal states without changing them.
That gave us the strongest result of the night.
The controller was:
queued YES
present in ready queue YES
ready bitmap present YES
runnable YES
schedulable YES
SC refill ready YES
SC refill sufficient YES
release queued NO
CPU/domain match YES
In other words:
the controller itself looked healthy.
Then we found what was ahead of it
The highest ready thread on CPU 0 was the PCI service.
Its priority is 248.
The controller's is 199.
And the PCI service was observed running.
That explained the scheduler decision at that instant.
It did not, however, prove that PCI was the bug.
That distinction became important almost immediately.
The obvious temptation would have been to lower the PCI service's priority, raise the controller's priority, force a resume, or otherwise make the symptom disappear.
I did none of those things.
Changing a priority until something starts running is not debugging.
It is negotiating with the evidence.
PCI looked guilty. Then it didn't.
The next pass audited the PCI service itself.
Was it spinning?
Was it sitting in an accidental infinite loop?
Was it polling forever?
No.
Its initialization path was bounded.
Its notification handler did bounded work.
It returned to the normal blocking runtime path afterward.
Lower-priority services were observed getting CPU time after PCI initialization.
Later PCI wakeups also had legitimate event-driven explanations.
So the first theory —
the high-priority PCI service is simply spinning forever and starving the controller
— did not survive inspection.
That is where the controller investigation stood when I stopped that branch for the night.
We had proven a great deal about what wasn't wrong.
We had not yet proven what was.
That is still progress.
A provenance problem surfaced too
Late in the investigation another issue became impossible to ignore.
We are in the middle of validating the move to the newer 2.3 platform stack, but some of the retained build tooling and validation paths still trace back to the older 2.2 environment.
That means there is a dangerous question sitting underneath all of the scheduler evidence:
Are we actually debugging the exact software stack we think we are debugging?
That needs to be answered before I start accusing the MicroKernel scheduler of doing something impossible.
So the next stage of this work will include a full provenance check of the build environment: exact source revisions, exact tools, exact generated artifacts, and which copy of what actually produced the test image.
This sounds like bookkeeping until the day it isn't.
Reproducibility only matters when two things that are supposedly the same suddenly are not.
Then it matters a lot.
NuBASIC got another round of scrutiny
The controller work was not the only thing that made it into the overnight session.
I also spent time reviewing the proposed NuBASIC Level 1 specification.
The goal remains to have a complete Level 1 language available for A1 rather than shipping a tiny novelty BASIC that exists only to print HELLO.
But writing a plausible language specification is surprisingly good at exposing places where hand-waving is not good enough.
The current review surfaced issues around things such as:
- grammar consistency
- STOP/CONTINUE lifetime
- procedure and call-frame behavior
- STATIC data reset behavior
- stack exhaustion
- error scope
- platform-binding completeness
- built-in function accounting
- deterministic behavior
None of that is implementation work yet.
That is deliberate.
It is much cheaper to discover contradictions in a specification than in a language runtime people have already started using.
And the applications are starting to shape the machine
Near the end of the night I also went back to something I keep returning to with Ceylon:
I don't want the operating system to become technically impressive and unpleasant to actually create things with.
That led into more design research around future 2D/3D tools.
One principle is becoming firm:
CPU-only capability stays first-class.
A GPU can eventually make rendering dramatically faster.
It should not be the admission ticket for rendering at all.
If a user wants to let a CPU grind on a final render for a long time, the system should still be capable of producing the result.
The same philosophy is starting to shape CAD and design-tool thinking.
Precision needs to exist.
Exact dimensions need to exist.
Constraints need to exist.
But I do not want every creative action to begin with somebody typing measurements into a dialog box.
There should also be room to sketch, push, pull, shape and experiment — and then have the system help normalize that work into something dimensionally valid and, eventually, fabricatable.
That is research, not an implementation commitment yet.
But it is exactly the kind of application pressure that tells us what the platform underneath needs to become.
What today actually accomplished
If I reduced today to a release-note list, it would look unimpressive.
The controller still wasn't running when this workday ended.
But that misses what happened.
We proved that the controller is generated correctly.
We proved that the initializer starts it.
We proved that the MicroKernel sees it.
We proved that it is runnable.
We proved that it is schedulable.
We proved that its scheduling context is usable.
We proved that it is actually present in the ready queue.
We found a higher-priority contender.
Then we proved that our first explanation for that contender was incomplete.
And instead of forcing the machine into the answer we wanted, we stopped.
That is what development looks like sometimes.
The project also became public in a new way today.
It is now entered in the Hackaday Retrocomputing Contest.
There are architecture diagrams on the page.
There is an old First Light image showing that the thing really has put pixels on a screen.
There is a plan for how outside testers will eventually get their hands on it.
And behind all of that, there is still a stubborn controller sitting at 0x2025d8 waiting for us to explain precisely why it has not been given its turn.
That's tomorrow's problem.
Tonight, the useful result is that the list of possible answers is a whole lot shorter than it was this morning.