url-to-markdown: raw URL normalization corrupts escaped quote in YAML frontmatter

Author: dengdi30Created Aug 17, 2026Updated Aug 17, 2026

Summary

baoyu-fetch can emit invalid YAML frontmatter when a double-quoted metadata value contains an HTTP(S) URL immediately followed by an escaped quote.

This is reproducible on baoyu-url-to-markdown 1.61.0 and the current main runtime files.

Reproduction

The public X page below has a page title containing a quoted phrase that ends in a t.co URL:

https://x.com/andrewyng/status/2088302050706686198?s=46

After generic/Defuddle extraction, frontmatter serialization initially produces valid YAML:

yaml
title: "Andrew Ng on X: \"New: https://t.co/VVkn1Dqp1N\""

A network-free minimal reproduction is to pass the same valid string through normalizeMarkdownMediaLinks():

typescript
const before = [
  "---",
  "title: \"Andrew Ng on X: \\\"New: https://t.co/VVkn1Dqp1N\\\"\"",
  "---",
  "Body",
].join("\n");

console.log(normalizeMarkdownMediaLinks(before));

Actual result

yaml
title: "Andrew Ng on X: \"New: https://t.co/VVkn1Dqp1N/""

The closing embedded quote is no longer escaped, so YAML parsers reject the generated Markdown.

Expected result

Frontmatter remains valid and the escaped quote is preserved:

yaml
title: "Andrew Ng on X: \"New: https://t.co/VVkn1Dqp1N\""

Root cause

  1. renderFrontmatterValue() correctly serializes strings with JSON.stringify().
  2. cleanMarkdown() then calls normalizeMarkdownMediaLinks() on the complete Markdown string, including YAML frontmatter.
  3. RAW_URL_RE does not exclude backslashes, so it matches the URL plus the \ used to escape the following quote.
  4. normalizeMediaUrl() calls new URL(rawUrl).href, which converts the trailing backslash to /.
  5. Replacing the match therefore removes the YAML escape and corrupts the frontmatter.

In the minimal case, the regex match and normalization are:

matched:    "https://t.co/VVkn1Dqp1N\\"
normalized: https://t.co/VVkn1Dqp1N/

Suggested fix

Prefer keeping YAML frontmatter outside the raw-URL normalization pass, so normalization only processes the Markdown body. A smaller defensive change would be to exclude \ from RAW_URL_RE.

Please add a regression test that renders a document whose title contains a URL immediately followed by an embedded quote, then verifies that the resulting frontmatter can still be parsed as YAML.