issues/10-055-source-browser-rendering-and-navigation.md

10-055: Source Browser — Rendering Fidelity and Navigation

Status

Phase: 10 (Developer Tooling)
Priority: Medium
Type: Feature (builds on Issue 10-052, the source browser itself)
Status: In progress

Implemented so far

A — code folds: vimfold regions render as no-JS <details> blocks

(fold_marker_kind + the fold-aware render_text_page, one block per line so
they nest validly). Default open.

B — markdown rendering: .md / .info.md render as formatted HTML via the

new libs/markdown.lua (render_markdown_page); unit-tested in
libs/markdown-test.lua.

E — extensionless files: classify_file renders extensionless prose (e.g.

notes/vision) as markdown, and the tree is built only from rendered files, so
the table of contents can no longer point at a missing page.

H — back-to-site link: every page links to /similar-different/wordcloud.html.
D — issue-number links: build_issue_index + linkify_issues turn

"Issue NNNN" mentions in comments into links to that issue's page.

G — output as live links: a single output/ sidebar entry deep-links to the

live poem index (wordcloud.html#poem-index, anchor added there) instead of
listing ~23k pages.

F — saved-webpage mirror: per the author's call, a saved page (.html +

_files/) is mirrored as the real article (find_mirror_pages, strip_scripts,
copy_raw) rather than shown as source or linked out; scripts (incl. an
analytics tracker) are stripped for the no-JS rule and privacy.

Remaining

C — the dash-aligned, description-bearing table of contents: each entry

links to the file's .info.md page (which gains a "view source" button at top),
with descriptions drawn from docs/table-of-contents.md. Needs missing
descriptions and some missing .info.md files created along the way (most
src//libs/ files lack one today). This is the largest remaining piece.

Visual note

The code folds and rendered markdown should be eyeballed in a browser after the
next regeneration; the markup is well-formed and the markdown is unit-tested, but
the exact look (fold triangles, spacing) may want small CSS tweaks in
write_style_file.

Background / Why This Exists

Issue 10-052 built the "Machine Codex" — a link-only static website of the
project's own code, issues, and docs, generated by src/generate-source-browser.lua
into output/source/. It works, but it renders everything as flat numbered
text and its navigation has gaps. This issue is the second pass: make the
browser render each file as what it is (code that folds, markdown that
formats), and make every path in it actually lead somewhere useful.

Hard Constraint (applies to every feature below)

No JavaScript. The deployment platform (neocities, the link-shared site)
forbids it, and the browser was deliberately built JS-free. Every interactive
behavior here must be pure HTML/CSS. The good news: the sidebar already proves
the pattern — it is a tree of <details>/<summary> elements, which collapse
and expand on click with no script at all. The fold feature reuses exactly that.

Relevant Files / Functions

src/generate-source-browser.lua — the whole generator. Key parts:
list_published_files() — git-tracked + allowlisted file discovery.
ext() — maps a path to (extension, language); the source of the

extensionless-file bug.

highlight_line() — per-line tokenizer; where comment text is emitted, so

where issue-number links would be injected.

render_text_page() — wraps lines in numbered <pre>; where folds and

markdown rendering branch in.

render_sidebar() — the nested <details> tree; the table of contents.
page_shell() — the page frame and nav bar; where a "back to the site"

link belongs.

main() — per-file classification (text / image / skipped) and the index.
The project's vimfold convention (-- {{{ name … -- }}}) — the fold regions

are already authored in every source file.

_style.css (written by write_style_file()) — the shared theme; new

elements get styled here, not inline.

Feature A — Mouse-driven collapsible code folds

Current Behavior

render_text_page() emits every line into one flat <pre>. The vimfold markers
that bracket each function (-- {{{ name … -- }}}) render as ordinary comment
text. There is no way to collapse a region.

Intended Behavior

Each vimfold region renders as a collapsible block: the {{{ name line becomes
a clickable <summary> (with a little triangle/marker, same as the sidebar
folds), and the region's body is the collapsible <details> content. Clicking
the marker opens/closes the region. Folds nest. Line numbers and #L<n> anchors
must survive folding (a deep link to a line inside a closed fold should still
resolve — <details> content is in the DOM even when collapsed, and CSS can be
asked to open a fold that contains the :target).

Implementation Notes

Parse the marker pairs while rendering lines; the language's line comment

prefix plus {{{ / }}} identify open/close. Markers are balanced per file.

Wrap each region in <details open><summary>…</summary>…</details> (default

open so the page reads top-to-bottom unless the reader collapses).

Reuse the sidebar's summary::before triangle styling so folds look native.
Files with no markers (json, css, most md) render exactly as today.
Decide: fold by vimfold markers only, or also offer a "collapse all" via a

CSS-only control. Marker-based is the foundational version; ship that first.

Feature B — Render `.md` files as Markdown, not plain text

Current Behavior

.md (and .info.md) files are in TEXT_EXTS as language md, but LANGS.md
is empty, so they render as numbered plain-text lines. Tables are ASCII soup,
headings are not headings, bold/links/lists are literal characters.

Intended Behavior

Markdown files render as formatted HTML: headings, bold/italic, lists, tables,
fenced code blocks (with the existing tokenizer for known languages), block
quotes, and links. The result reads like documentation, not source.

Implementation Notes

A small server-side Markdown→HTML pass in Lua (no JS, no client highlighter).

Cover the subset the repo actually uses: ATX headings, **/*, -/1.
lists, GitHub pipe tables, fenced code, blockquotes, inline/reference links,
inline code.

Branch in main() / render_text_page(): when language is md, run the

markdown renderer instead of the line-numbered tokenizer.

.info.md files render as markdown too (they describe a source file's public

functions — Feature D should cross-link them to that file).

Keep a "view raw" affordance is optional; the rendered view is the default.

Feature C — Table of contents: linked paths, dash-aligned descriptions

Current Behavior

render_sidebar() lists bare filenames as links inside the <details> tree.
There are no descriptions, and nothing is aligned.

Intended Behavior

The table of contents shows each file's path as a link, followed by a dash
leader that aligns the descriptions into a column, like:

/path/to/file.md ------------ text explanation
/short/path.md -------------- other explanation
/longer/than/others/path.md - text and emoji explanation

Each path links to that file's .info.md page (the description node), and
that .info.md page carries a "view source" button at the top that jumps to
the actual file. Alignment resets per section (per directory grouping), so each
section computes its own longest-path width.

Settled design (author's call)

Descriptions come from docs/table-of-contents.md — they already live

there (format: ` - /path - description under ###` section headers). The
ToC parses that file, so descriptions have one home and do not rot. Entries get
added there whenever a new file that belongs in the ToC is created; missing
ones are filled in.

Each entry links to the file's .info.md page, not straight to the source.

The .info.md page gets a "view source / view file" button at the top linking
to the rendered source page. A file with no .info.md falls back to linking the
source page directly (until its .info.md is created).

Implementation Notes

The dash leader width = (section's longest path length) − (this path length) +

a fixed gap. Compute per section after collecting that section's entries.

Monospace makes character-count alignment exact; the browser is already mono.
This likely lives in the main content area (a dedicated ToC page) rather than

the cramped sidebar, since the dash columns need width. The sidebar tree stays
as the always-present quick nav.

Folded in: backfill the missing `.info.md` files

Most src//libs/ files have no .info.md today (~5 of ~60), but this feature
routes the ToC through them — so the gap must be closed. Plan:

Build the tool, then refine (per the project's "create the tool that

creates things" rule): a small generator that reads each source file's header
comment + its public (M.-exported) functions and their .info.md-style
descriptions, and emits a starter <file>.info.md. This makes ~55 consistent
stubs in one pass instead of 55 hand-written files.

Then refine each stub into a real black-box description (purpose + how it

operates, inputs/outputs), the way the existing .info.md files read.

Add the corresponding docs/table-of-contents.md entry where the file belongs

in the ToC (source files don't all belong there, but docs/notes do; use
judgment per the existing ToC scope).

Going forward: a new file that belongs in the ToC gets its .info.md and its

ToC line when it is created (the existing convention, now enforced by this
feature actually using them).

Feature D — Issue-number references become links

Current Behavior

Comments everywhere say things like "Issue 10-036" or "(Issue 8-043c)". They
render as plain comment text. A reader cannot jump to the referenced issue.

Intended Behavior

Any issue reference in a rendered file becomes a link to that issue's page in the
browser. Clicking "10-036" opens issues/.../10-036-….md.html.

Implementation Notes

Build an index once: scan list_published_files() for issues/**/NNNN-….md

and map the issue number (PHASE-ID, e.g. 10-036) → its published relpath.
The file may be in issues/ or issues/completed/ or a phase subdir, so the
path is looked up, never hardcoded.

Apply during rendering: after highlight_line() emits a comment token (or in a

post-pass over emitted comment spans), replace issue-number matches with links.
Match carefully so line numbers, version strings, and dates are not mistaken
for issue numbers (anchor on the word "Issue" or the NNNN- filename shape).

A reference to an issue that is not published (held back by the allowlist)

stays plain text — log/skip, don't emit a dead link.

Feature E — Fix broken links for extensionless files (the `vision` bug)

Current Behavior

notes/vision is tracked and lives under an allowlisted directory, so
list_published_files() keeps it and render_sidebar() emits
href="notes/vision.html". But ext("notes/vision") returns an empty
extension, so main() classifies it as neither text nor image and skips it —
no page is written. Result: the sidebar links a page that does not exist (404).
This hits any extensionless tracked file in an allowlisted dir.

Intended Behavior

notes/vision (the project's foundational document) renders as a readable page,
and no table-of-contents entry ever points at a missing page.

Implementation Notes

Treat extensionless tracked files in allowlisted dirs as text (they are prose

here). Optionally sniff for binary (NUL bytes) and only then skip.

vision is prose — render it as markdown (Feature B) for nice formatting.
Defense in depth: the sidebar should only link files that actually produced a

page. Build the published-page set first, then render links from it, so a skip
can never become a dead link again.

Feature F — Saved webpages link out, not in

Current Behavior

A saved webpage — an .html with a sibling <name>_files/ assets directory — is
not hosted at all. Its table-of-contents entry is an external link (new tab,
rel="noopener", a ↗ marker) to the original article; no page is written for it
and its _files/ assets are not published. The platform is no-JS and hosting a
copy of a stranger's article (scripts, trackers, CSS) was the wrong thing to ship.
docs/Your URL Is Your State.html now links to
https://alfy.blog/2025/10/31/your-url-is-your-state.html.

(An earlier pass mirrored the page locally — wrote the .html with <script>
tags stripped and copied its CSS/image assets. That still hosted a copy, so it was
replaced by the link-out behavior above.)

Intended Behavior

Clicking that file in the table of contents opens the **actual article on the
web**, not a source dump and not a hosted copy.

Implementation Notes

Detect saved webpages: an .html file with a sibling <name>_files/ directory

is a saved page, not project source.

The original URL is read from the saved file's own <link rel="canonical"> /

<meta property="og:url"> — self-describing, so it never drifts out of date.
A page lacking those tags can be mapped explicitly in MIRROR_URL_OVERRIDES;
one with neither is held back and named in the build report (never guessed,
never dumped as source).

The table-of-contents entry is an external link to the real URL; no source page

is generated, and the _files/ assets are not published.

The hosted copies left by the earlier mirror/source-dump passes were a one-time

legacy in output/source/ (git-ignored, per working tree); they were deleted
once by hand rather than carried as permanent cleanup code, since the generator
no longer produces them.

Feature G — Show `output/` as links to the live site

Current Behavior

output/ is git-ignored, so git ls-files never lists it and it does not
appear in the browser at all.

Intended Behavior

The browser shows the output/ directory in its tree, but each entry links to
where that file is displayed on the actual website (its production URL under
/similar-different/…), not to a rendered copy of the output HTML. The point is
to let a reader cross from "the code" to "the thing the code makes," live.

Implementation Notes

output/ cannot come from git ls-files; enumerate it directly (a separate,

clearly-labeled listing), and map each path to its production URL using the
same base the deploy converter uses (/similar-different, Issue 9-005b).

These are external links to the deployed site — no source pages are generated

for output files (that would be huge and pointless).

Consider listing representative entry points (the menus, an index per

directory) rather than all ~23k generated pages, to keep the tree usable.
Note in the page whatever is summarized so nothing reads as "complete" when it
was trimmed.

Feature H — A link back to the live site (wordcloud / menu)

Current Behavior

page_shell()'s nav has only the "Machine Codex" home link (to the browser's own
index). There is no way back to the actual poetry site from inside the browser.

Intended Behavior

Every browser page has a link back to the live site's menu (wordcloud.html),
so a reader can return from "the source" to "the work."

Implementation Notes

Add the link to the nav in page_shell(), pointing at the production menu URL

(/similar-different/wordcloud.html).

Style it as a sibling of the home/brand link in _style.css.

Suggested Build Order

Foundational first, so later features build on them:

E (extensionless fix) and H (back link) — small, unblock navigation.
B (markdown renderer) — reused by E (vision) and improves all docs/issues.
A (code folds) — independent, high visual value.
D (issue links) — needs the published-page index; pairs with B for issues.
C (dash-aligned linked ToC) — needs a description source decided.
F (saved-webpage out-links) and G (output → live links) — need the

external-URL map / production-base wiring.

issues/10-055-source-browser-rendering-and-navigation.md

10-055: Source Browser — Rendering Fidelity and Navigation

Status

Implemented so far

Remaining

Visual note

Background / Why This Exists

Hard Constraint (applies to every feature below)

Relevant Files / Functions

Feature A — Mouse-driven collapsible code folds

Current Behavior

Intended Behavior

Implementation Notes

Feature B — Render .md files as Markdown, not plain text

Current Behavior

Intended Behavior

Implementation Notes

Feature C — Table of contents: linked paths, dash-aligned descriptions

Current Behavior

Intended Behavior

Settled design (author's call)

Implementation Notes

Folded in: backfill the missing .info.md files

Feature D — Issue-number references become links

Current Behavior

Intended Behavior

Implementation Notes

Feature E — Fix broken links for extensionless files (the vision bug)

Current Behavior

Intended Behavior

Implementation Notes

Feature F — Saved webpages link out, not in

Current Behavior

Intended Behavior

Implementation Notes

Feature G — Show output/ as links to the live site

Current Behavior

Intended Behavior

Implementation Notes

Feature H — A link back to the live site (wordcloud / menu)

Current Behavior

Intended Behavior

Implementation Notes

Suggested Build Order

Related Documents / Issues

Feature B — Render `.md` files as Markdown, not plain text

Folded in: backfill the missing `.info.md` files

Feature E — Fix broken links for extensionless files (the `vision` bug)

Feature G — Show `output/` as links to the live site