The notes pipeline

How NicAI turns "write a note about X" into a live, signed page on this site in about 15 seconds: the write-note skill, the publish hook and the 5 publish phases.

Table of Contents

This notebook is where NicAI's work is most visible. When I say "write a note about X", an agent researches it, writes it, saves it, and the page is live about 15 seconds after the file lands. Nobody clicks publish.

The pipeline has 2 halves:

  • The write-note skill, which turns a topic into a Markdown file.
  • The publish pipeline, which a hook starts as soon as that file exists.

The site itself is older than NicAI, and its history is in Building this site: notes.nicolasdeville.com. This note covers the path from request to live page.

From request to Markdown

The write-note skill is one SKILL.md file of about 670 lines. It walks the agent through these steps.

1. Classify the note

I can name the folder (apps: some-tool.com) or leave it out. The skill then picks one of 20 folders with a short set of rules: a GitHub repository goes to github, a Python library to python, an AI tool to AI, anything about NicAI itself to NicAI. random is the last resort. Each folder has its own structure: a book gets a details table, a helper note stays a short command reference.

2. Research

The agent uses web search and page fetches for facts, prices and official URLs. A YouTube link switches it to a transcript workflow: the full video summarised on its own, or its insights folded into a wider topic. Either way, the player is embedded.

Before any of this, the agent searches for an existing note on the subject. If one exists, it updates it, bumps the date, and datestamps the old and the new content so a reader can tell the layers apart.

3. Images

Images come only from the subject's official site, never from news or review pages, because the site re-hosts them on my own CDN. A helper script stages ranked candidates in a temp folder. The agent looks at each one, keeps 1 to 4, and copies them into the note's image folder under descriptive names. The first image becomes the thumbnail in search results, so its alt text is a real sentence.

Link blocks to external resources carry a small logo, fetched from the logo.dev API if it isn't on disk yet. Every brand or product named in a note links to its official site, opening in a new tab.

5. The signature

Every note NicAI writes ends with the same signature block: "Written by NicAI... Verify anything you plan to act on." It is verbatim, styled by the theme, and excluded from search so it doesn't match every query. A helper, sign_notes.py, backfills or audits signatures across the site. It keeps an exclusion list, because a NicAI tag marks the topic, not the author. A note I wrote myself about my own setup stays unsigned.

6. The single Write

The agent writes the .md once, as its last step, after every image and logo is on disk. That order matters for the next half.

From Markdown to live page

The hook

A Claude Code PostToolUse hook on the Write tool runs a small zsh script, publish_on_note_hook.zsh, after every file the agent writes. The script reads the hook's JSON from stdin and exits at once unless the path is a .md under the articles folder. Every other write is a no-op.

For a note, it takes a lock and runs publish.sh. macOS has no flock, so the lock is an atomic mkdir. That matters when several agents write notes in parallel: the publishes queue instead of colliding. A lock older than 5 minutes counts as stale and gets taken over. The hook always exits 0, so a failed publish never blocks the agent, and everything goes to a log.

I added the hook on 12th June 2026. The log shows 106 hook-triggered publishes since then.

The phases

publish.sh runs 5 phases, with parallel work where 2 steps don't touch the same files.

Phase What it does
0. Verify logos Normalises logo references to lowercase and warns on missing files. The server is case-sensitive, my Mac isn't
1. Build Pelican turns about 1,300 Markdown notes into HTML
1b. Redirects and prune Writes 301 redirects for moved notes, then deletes pages the build no longer produces
2. Post-process Uploads new images to ImageKit, rewrites image links to the CDN, expands !folder/slug links
3. Search and sitemap Pagefind indexes every page; a sitemap is built in parallel
4. Deploy rsync --delete --checksum mirrors the output folder to the server

A few details that took more than one attempt:

  • Redirects. A plain text file lists old and new paths. write_redirects.py writes a 301 .htaccess and a noindex meta-refresh page at each old path, as a fallback. It never overwrites a real page and skips a redirect whose target doesn't exist.
  • Prune. Pelican never deletes output it stops generating, and rsync --delete mirrors whatever is local. Without a prune, a renamed note stays online at its old URL. A small Pelican plugin now records every file the build writes. prune_stale.py deletes anything not in that manifest or the redirects, with 2 guards: the manifest must be less than 1 hour old, and above 25 stale files it deletes nothing and warns. A broken build can't wipe the site.
  • Images. ImageKit serves every image. The upload script keeps a cache of about 5,100 uploaded paths and only sends new ones.
  • Internal links. !folder/slug expands to a link with the target note's title, from a cache of about 1,400 titles. This runs on the HTML after the build, not inside Pelican.
  • Search. The Pagefind index is deleted and rebuilt on every run. Pagefind appends fragments, so without the wipe, files from old builds pile up.

Timing

The run on the morning of 23rd September 2026 took 19 seconds. Pagefind indexed 1,288 pages in 1.2 seconds, the sitemap listed 1,309 URLs, and the deploy sent 4.5 MB of a 47.2 MB site in 6 seconds. Across the 106 logged runs, almost all finished in 12 to 19 seconds. 2 took just over 1 minute. The hook times out at 180 seconds.

Gotchas

  • Edit doesn't publish. The hook fires on Write only. An agent that fixes a typo with Edit must run publish.sh by hand.
  • Never overwrite an image. ImageKit caches by path, and my upload cache never re-sends a path it has seen. An overwritten image stays stale on the CDN. New versions get a new filename.
  • Never stage images in the content folder. The upload walks that tree, so a rejected candidate uploaded once stays uploaded.
  • ! links need whitespace. The pattern matches !folder/slug only after a space or at the start of a paragraph. Directly after a bracket or at the start of a list item it stays literal text.
  • Tables need a header row. Python-Markdown only parses a table when a header sits above the | - | separator. A key/value table gets an empty header, | | |.

Why it works

The agent does the writing. Deterministic scripts do everything after that. No model is involved once the file exists, so the publish is fast, repeatable and cheap. The signature keeps it honest: more than 100 of the 1,288 notes on this site came from NicAI, and each one says so.

NicAI
Written by NicAI, Nic's AI assistant, for his personal knowledge base. Researched and drafted by the model, not hand-written by Nic. Verify anything you plan to act on.