feat(seo): SEO & AEO workspace — metadata model, JSON-LD, robots/sitemap, AI suggestions#53
Draft
DavidBabinec wants to merge 28 commits into
Draft
feat(seo): SEO & AEO workspace — metadata model, JSON-LD, robots/sitemap, AI suggestions#53DavidBabinec wants to merge 28 commits into
DavidBabinec wants to merge 28 commits into
Conversation
New @core/seo engine module: persisted schemas (SeoMetadata, SiteSeoSettings), two-stage title resolver shared by publisher/admin/server, JSON-LD builders (WebSite/Organization/Article/BreadcrumbList) with safe serialization, robots.txt generation with AI-crawler group controls, health indicators, and pixel-width length meters. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…field One built-in seoMetadata field (id: seo) on page and postType tables, stored as a structured SeoMetadata object in cells_json.seo. Page schema and site settings gain seo objects; content panel title/description merge into the structured cell; migrations, data grid, binding catalog, AI content tools, and tests updated. Pre-release: no compat path retained. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Publisher head is now driven by the resolved SEO payload (new src/core/publisher/seoHead.ts): title/description/canonical/noindex, OG (incl. og:locale, og:url, og:site_name, article times), X cards, and JSON-LD scripts. Server renderer resolves page/row SEO with the configured public origin and entry-template patterns; previews share the same resolver via publishPage's fallback. Legacy site metaTitle/metaDescription removed in favour of site.settings.seo. First-party GET /robots.txt (with AI-crawler group toggles) and GET /sitemap.xml (publishVersion-cached) dispatch before public rendering. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ities New /admin/tools/seo workspace (Meta / Robots.txt / Sitemap tabs): target index with search, filters, issue chips, health dots, and keyboard nav; sticky preview editor with Search/OG/X/Schema platform views, inherited values as placeholders, pixel length meters, media-library image pickers, and a dirty-switch guard dialog. Site defaults editor covers title patterns, social defaults, X handle, and Organization JSON-LD fields. Robots tab pairs indexing + AI-crawler toggles with a byte-identical generated preview; Sitemap tab manages enable/exclusions with counts. Plugin admin pages move under a new Tools nav dropdown alongside SEO. Server: /admin/api/cms/seo route group (targets index, target writes with ownership gates, site settings) and seo.read/seo.manage capabilities. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
POST /admin/api/cms/seo/generate: one tool-less driver call through the existing server/ai stack (content/site scope default picks the provider), returns three suggestions parsed from a JSON-array response. Gated by seo.manage + ai.chat. The editor's sparkle action renders suggestions as tappable bubbles with More options (exclude-aware regenerate) and Reject. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New docs/features/seo.md covering the model, resolver, JSON-LD, robots/sitemap endpoints, workspace UX, AI suggestions, and permissions. Updated publisher/editor/content-storage/data-workspace/plugin-system/ site-shell/capabilities docs + the plugin template example for the structured seo field. Lint: compiler-safe dirty-guard effects, redundant setState removal, regex escape. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Complete Meta-tab redesign: a sticky left rail stacks the live 1:1 platform previews (Google SERP, Open Graph card, X card — real platform dark-theme palettes via new --seo-* tokens — plus a collapsible JSON-LD block), next to a sectioned metadata form (Search appearance / Social card / X card) capped at sane widths. Site defaults preview the homepage with the draft defaults applied live. Target index regrouped under Pages/Templates/Posts headers with a pinned Site defaults card, mono routes, and breathing room. Header tabs match the AiPage Button-row pattern (§T.6); robots/sitemap/schema code surfaces render through a read-only CodeMirror viewer (new readOnly mode on CodeMirrorEditor); switches ride the FormField primitive. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…adii Target index refactored onto bare-button list rows (§8.8 — the Button primitive's fixed heights crushed the two-line layout): pinned Site defaults card with globe icon, 'Pages · N' group headers with counts, clean two-line rows (title + mono route / sans descriptor) and a clickable issues line that jumps to the Issues filter. Google snippet now renders the configured site favicon. All preview cards share --panel-radius. Length meters gain a green→amber→red gradient clipped by fill (true progression) plus a budget tick; preview rail unsticks on narrow viewports so mobile scrolling isn't trapped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
'Saved' alone read as 'live on the site' — it isn't: SEO follows the publish lifecycle (Layer A bakes heads at publish time). The save status now states it, matching the Robots/Sitemap tab subheadings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Save/publish moves out of the editor headers into the workspace toolbar, matching the Site and Content workspaces: the active editor (target, site defaults, robots, sitemap) registers on a save bridge and SeoToolbar renders the shared PublishActionGroup (status dot, Save & publish split button, Save draft / Open live URL menu). Posts publish through the incremental row endpoint; everything else runs the step-up-gated full site publish — the same machinery as the Site toolbar. The Meta tab becomes a control center driven by a new core report engine (computeSeoReport): weighted per-target checks with a 0-100 score, a site-wide scoreboard with the LiquidProgressRing (promoted to a shared @ui primitive with amber/danger tones) and coverage tiles, tiered score pills in the target index, a live score chip, and a clickable improvements list that focuses the field each issue describes. Length meters get an ideal band: green is reserved for the 30-60 / 70-160 character bands instead of lighting up from the first character, and inherited placeholder values render muted with an Inherited/Missing tag instead of a bogus character count. The editor form switches to a two-column grid (muted labels left, controls right, large section headings). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The OG/X image fields hand-rolled a URL input + Browse button while the property panel already had the Library/URL segmented toggle over the shared MediaPickerField tile (thumbnail, Change/Clear, fullscreen MediaPickerModal). SeoImageField now uses the same pieces: Library mode shows the picked tile with asset metadata, URL mode keeps external images with an inline preview and validation, and a purely inherited image renders through the tile's fallback state with an "Inherited — pick one to override" hint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![github-advanced-security[bot]](https://cdn.statically.io/img/avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| {value !== '' && !urlInvalid && ( | ||
| <span className={styles.urlPreview}> | ||
| <img | ||
| src={value} |
…n the label cell The grid layout left AiSuggestionSparkle's whole output (trigger + error + suggestion bubbles) inside the 140px label column, so a provider-not- configured error wrapped one word per line. The state machine moves to hooks/useAiSuggestions.ts; the sparkle trigger stays in the label cell while AiSuggestionResults (error line + bubbles) renders in the control column under the input, via the new per-field MetaField component. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ent bar Target index rows render title and route/descriptor in a single baseline-aligned line with a 6px gap instead of two stacked lines, and the selected row drops the inset left accent shadow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every form row in the SEO workspace now renders through one component: SeoFormRow (muted label + control-stack grid) with a SeoSwitchRow boolean variant. The Meta editor's MetaField, the noindex and select rows, the site-defaults form, and SeoImageField all migrate onto it, deleting the per-file copies of the grid CSS. The Robots.txt and Sitemap tabs drop their single stacked column for the same workbench split the Meta tab uses: settings card (large heading, switch rows, counts) on the left, a sticky code preview on the right. SeoCodeViewer becomes a proper surface card — CM6's own editor/gutter backgrounds are flattened to the card surface and the read-only active-line stripe is removed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Switch primitive's on-state was surface-3 on surface-2 — nearly invisible. Globally: on = white (accent) track with a dark thumb, off = raised surface-4 track with a light thumb, plus a real disabled treatment. The unused hitArea variant is deleted. SEO switch rows move from the sm switch to the full-size one. Form rows step up for contrast: labels 13px --editor-text, hints 12px weight-500 --editor-text-secondary; card subheadings match. Settings cards gain padding (24px) and row spacing (22px); the Meta form breathes a little more too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ile band The full-width scoreboard with coverage tiles compressed the editor and duplicated what the index already shows. The liquid-progress score ring plus its explainer now sits at the top of the right sidebar (SeoScoreSummary), directly over the target search; the editor starts at the top of the page. The tiles and their Review-issues action are gone — the index's own issues line covers the jump — so the kind filter moves back into SeoTargetIndex as local state. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Index sidebar (score + targets) moves to the left, the editor form sits in the middle, and the platform-preview rail goes right — and the nested-grid structure goes away: SeoPreviewEditor/SiteDefaultsEditor render form + rail as fragment siblings, so MetaTab owns a single display: grid with three tracks instead of a two-column grid wrapping another two-column workbench. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
--editor-text-subtle is nearly invisible on the index's surface-2 rows. Row routes/descriptors, group counts, the Site defaults chevron, and the length meter's Inherited/Missing tag now use --editor-text-muted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tables Two template bugs in the target index: - `everywhere` layout templates appeared as SEO targets labelled "Entry template". They have no route and no content of their own — the pages they wrap own the metadata — so the targets endpoint now drops them; only entry templates (postTypes targets, token-pattern sources for their posts) are listed. - Entry templates rendered a bare "Entry template" descriptor because the index read `tableSlug` (post rows only) instead of `templateTableSlugs`. The index rows and the editor header now show "Entry template · posts". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Tools nav trigger now reads as a dropdown: a chevron-down rides the label, the menu opens on hover (with a 160ms close-intent timer bridging the trigger→menu gap, click still toggles), and when the current section lives inside the menu (SEO, plugin pages) the trigger carries the same bold current-section text the sibling nav items use. Also makes the robots-preview test deterministic: SeoCodeViewer mirrors its value in a data-code attribute, so assertions stop racing the lazy CM6 chunk (which paints zero measured-visible lines in test DOMs — the source of the admin+architecture combo flake). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ifier SITE_DEFAULTS_ID is only used within MetaTab — make it a private const. The design spec now lives on main (landed via #52), so drop the "(local)" qualifier in the feature doc. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…protection Three professional-grade additions to the robots.txt feature, all driven by the same pure generator the endpoint and the live preview share. #1 Path-level custom rules. SeoRobotsSettings gains a `rules` array (per-user-agent allow/disallow paths) and a `disallowSystemPaths` default (on) that blocks `/admin` and `/_instatic/`. The generator becomes a UA→group merge, so system paths, AI-crawler toggles, and custom rules compose without ever repeating a `User-agent:` header. #2 Escape hatch + validation. A raw `extraDirectives` field appends verbatim (Clean-param, Host, a brand-new bot). New robotsAnalysis.ts adds `lintRobotsTxt` (unknown directives, rules before any User-agent, malformed values) and `matchRobots` (Google longest-match, Allow tie-break, `*`/`$`), surfaced in the tab as a lint panel and a live "test a URL" checker. #3 Environment protection. `requestHostIsCanonical` compares the trusted request Host against PUBLIC_ORIGINS; a non-canonical host (preview/staging) gets a blanket `Disallow: /` from robots.txt plus `X-Robots-Tag: noindex` on pages, static assets, and uploads — so non-production deploys can't be indexed even via a direct asset URL. Null when unconfigured, so local dev is unaffected. The Robots tab grows three settings cards (crawling toggles incl. the new system-path block · custom-rule editor · Advanced raw directives) beside the preview, lint panel, and URL tester. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tant rail
The structured form (toggles + rule editor + extra-directives field) was
more complicated to use than the file it generated. Flip it: the robots.txt
body IS the artifact now — `seo.robots.content`, edited in a CodeMirror
surface that is the main column. The left rail becomes an assistant rather
than a parallel form:
- contextual recommendations (block AI crawlers / block system paths /
"all crawlers blocked" warning) that are themselves the one-click edits,
- quick-insert shortcuts (recommended defaults, block all AI, block all),
- a lint issue list (lintRobotsTxt) and a live URL tester (matchRobots).
generateRobotsTxt now serves the stored body (or DEFAULT_ROBOTS_TEMPLATE)
and appends the Sitemap line unless one is present; the structured
SeoRobotsSettings collapses to `{ content }`. Shortcuts compose bodies from
the maintained AI-crawler lists + SYSTEM_DISALLOW_PATHS, so nothing the old
toggles did is lost — it just edits the canonical text.
Also drops the "no public origin configured" notices from the Robots and
Sitemap tabs: production always sets PUBLIC_ORIGINS, so the warning was dev-
only noise. Env protection (#3) is unchanged server-side.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The recommendation cards used tinted fills + colored left-border accent bars under four stacked ALL-CAPS sections — the noisy pattern the rest of the app avoids (and the same left-accent bar already removed from index rows). Rebuild the assistant rail on the app's grouped-row pattern: recommendations and lint render as quiet surface-3 rows on a 1px-gap parent, the only color a single leading state dot; the URL tester result drops its colored fill for a dot + muted text. Two-layer color model restored — nothing decorative carries color. RobotsTab joins the §8.11 advice-row allowlist (full-width wrapping dot+text rows Button's fixed-height layout can't host). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Same assistant-rail + main-column shape: the left rail holds the heading, a compact Generate-sitemap toggle (label + switch on one line, hint full-width below — the wide SeoFormRow grid wrapped badly in the narrow rail), and the inclusion count; the per-target include/exclude list is the main column, rendered as the app's quiet grouped rows (title + route left, include switch right; noindex targets disabled with the reason). Off shows a quiet disabled state. Drops the XML entry-format code sample (reference fluff — the list already shows exactly which URLs ship) and the dead two-column workbench / preview-column CSS. Adds Sitemap-tab test coverage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts: # src/core/publisher/render.ts # src/ui/components/LiquidProgressRing/LiquidProgressRing.tsx
![github-code-quality[bot]](https://cdn.statically.io/img/avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
# Conflicts: # docs/features/content-storage.md # docs/features/data-workspace.md # server/db/migrations-pg.ts # server/db/migrations-sqlite.ts # src/admin/pages/content/components/ContentSettingsPanel/ContentSettingsPanel.tsx # src/ui/components/LiquidProgressRing/LiquidProgressRing.module.css # src/ui/components/Switch/Switch.module.css
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
SEO & AEO as a core publishing capability, end to end:
Core model (
src/core/seo/, new barrel-gated engine module)SeoMetadatastored incells_json.seo(one built-inseoMetadatafield onpage/postTypetables — replaces the flatseoTitle/seoDescriptionfields, no compat path; pre-release).SiteSeoSettingsundersite.settings.seo(title pattern, description, default social image, X handle/card, Organization, robots, sitemap) — replaces the legacymetaTitle/metaDescriptionsettings.{source.field}token pattern via the existing token engine → base title → site name), used identically by the publisher, the admin previews, and the health indicators.Published output
src/core/publisher/seoHead.ts: title, description, canonical,noindex(never a silentnofollow), Open Graph (incl.og:locale,og:site_name,article:*_time), X cards, and schema.org JSON-LD (WebSite,Organization,Article,BreadcrumbList) with safe serialization.PUBLIC_ORIGINS— baked HTML never guesses a host; dynamic endpoints fall back to the request origin.Crawl files
GET /robots.txtandGET /sitemap.xml(server/publish/seoEndpoints.ts), dispatched before static assets/public rendering, generated from the published snapshot and cached perpublishVersion.Admin workspace (
/admin/tools/seo)POST /seo/generate(one tool-less call through the existingserver/aidriver stack) → three tappable bubbles with More options / Reject.API & permissions
/admin/api/cms/seo/*route group; newseo.read/seo.managecapabilities; target writes additionally enforce the owning persona (pages.edit/ content edit); generate additionally requiresai.chat.Why
SEO/AEO correctness belongs in the CMS core: metadata emission, structured data, crawl files, and answer-engine controls must work without plugins. Spec was reviewed and hardened first (fallback-chain correctness, publish-lifecycle integration, origin handling, AEO scope).
Impact
rm -rf .tmp/dev.dbor freshbun run dev) — the seeded built-in field set changed.site.settings.metaTitle/metaDescriptionare gone; site-wide SEO copy lives in the SEO workspace.seo.*automatically.Verification
🤖 Generated with Claude Code