Search and AI discovery

H.2 Crawlable pagination (A-2)

Each list page needs its own URL and real links (links in the HTML that search engines follow). Example markup for The Stack hub:

<nav aria-label="Pagination">
  <a href="/the-stack?page=1" aria-current="page">1</a>
  <a href="/the-stack?page=2">2</a>
  <a href="/the-stack?page=3">3</a>
  <a href="/the-stack?page=2" rel="next">Next</a>
</nav>

• The server renders the matching list for ?page=N, so the server HTML of page N holds the article links for page N.
• Each page N gets a self-referencing canonical, which names the preferred URL of that page (/the-stack?page=2 points to itself, not to page 1).
• Keep the button behavior for users if needed, but update the URL with each click.
• Apply the same change to research /reports and /market-commentary and to the blog hub.

H.4 Redirects (A-4, A-17)

1. Add the 45 usable rows of the redirect map as single-hop 301s (permanent redirects). Of the other 15, 9 are marked TBD, 2 describe the target in words, and 4 point at placeholder URLs that return 404 (page not found).
2. Map the 270 dead historic page paths (old addresses from the Internet Archive; dead-urls-to-redirect.csv) to the closest live page, or to the section hub.
3. Map /globalassets/ PDF paths to the current document URL for the same fund and document type.
4. Send apex and trailing-slash variants straight to the final URL, in one hop.
5. Change the 11 Institute 302 redirects (temporary) to 301 (permanent).

Example rules for the Vercel config:

{
  "redirects": [
    { "source": "/blog/general-updates/assets-under-consideration-current-products", "destination": "<current AUC post>", "permanent": true },
    { "source": "/crypto-products/grayscale-bitcoin-trust", "destination": "https://etfs.grayscale.com/gbtc", "permanent": true },
    { "source": "/products/grayscale-bitcoin-trust", "destination": "https://etfs.grayscale.com/gbtc", "permanent": true }
  ]
}

H.3 Canonical tags (A-6)

• Print <link rel="canonical" href="https://<host><path>"> on every page, built from the path without the query string, except the paginated hub pages (H.2).
• On fund pages, the ?private and ?public views point to the clean fund URL.
• Write canonical paths without a trailing slash on www, to match its 308 redirects.

H.5 Sitemaps (A-8)

• Generate each sitemap from the published page list at build time.
• Include only URLs that answer 200 with no redirect and no noindex.
• Add all Stack articles to the www sitemap. Remove the 64 Institute Stack URLs that redirect to www (2 of them end at a 404).
• Add GDLC, ETCO, BPI and BTCC to the etfs sitemap. Remove /zcsh, because it redirects off the domain. Replace /mnrs and /gsol-prospectus with their final URLs.
• Give the ZCSH site its own sitemap with its own URLs.
• Set lastmod (the sitemap's last-change date) from the content's last edit date, not the build time.
• Remove /education and the duplicate glossary row from the www sitemap.

H.6 Soft 404s (pages that say not found but return 200) on research (A-7)

In the research routes for /reports/[slug] and /market-commentary/[slug], return the framework's not-found response (status 404) when no published entry matches the slug (the last part of the address).

H.7 Invisible zero-width characters (A-10)

Turn off content source map encoding in production builds of the content management system (CMS) client (the visual-editing "stega" option, if offered), or strip U+200B, U+200C, U+200D and U+FEFF from rendered text. Test: count those characters in the server HTML of /the-stack. Expected result: 0.

H.8 Other template fixes

• Footer: fix hthttps://etfs.grayscale.com/documents/ethe (A-16). The typo is hthttps, which should read https.
• Internal links: remove campaign tags (utm_*) and partnerref parameters from links between Grayscale hosts (A-15).
• Stack article template: one H1 (the main page heading), a title from the article headline, and a description from the summary (A-12, A-13).
• Research template: an H1 with the report title (A-12).
• Structured data: Organization and WebSite on every host (on Institute, correct the Organization data, A-26); Article with author, datePublished and dateModified on articles and research; BreadcrumbList (A-14).
• HTTP headers (settings the server sends with each page) on www, etfs, research and the ZCSH site: add X-Content-Type-Options: nosniff, Referrer-Policy: strict-origin-when-cross-origin and a Permissions-Policy (A-23).
• Institute robots.txt (the crawl rules file): publish a clean file without draft notes (A-22).

D.5 Speed by template