Add extractor for Xiaohongshu (rednote) pages
Feature: Add Xiaohongshu (rednote) extractor
I'd like to implement this feature and submit a PR. Below is my analysis of the problem — would you please be open to this?
Background
Xiaohongshu (https://www.xiaohongshu.com/) is one of the largest content platforms in China where users publish notes (Image/Video Notes) covering lifestyle, travel, food, tech, and more. Defuddle currently has no site-specific extractor for it, and the generic pipeline produces nearly unusable results.
Problems
1. Video content is completely missing
When a Xiaohongshu note contains a video, Defuddle's generic extractor fails to capture it. The resulting clipped note has no indication that a video existed, no embed, and no transcript.
2. Page variable content is contaminated by navigation context
Xiaohongshu renders note pages as an overlay/modal on top of the previous page (favorites list, search results, etc.). The underlying page's DOM remains present and gets swept into the extracted content. The specific contamination depends on how the user navigated to the note:
- From a favorites/collection page: The extracted
contentincludes covers and descriptions of other notes from the favorites list, as well as the collection's metadata (collection URL, collection name, etc.). - From search results: The extracted
contentincludes covers and titles of other notes from the search results list. For image notes, this also causes duplicate images .
3. Comments mixed into content
User comments rendered inline in the DOM also get included in content with no separation from the note itself.
4. No structured metadata extraction
Xiaohongshu note pages embed structured data (note text, images, video URL, author, tags, publish date, etc.) in window.__INITIAL_STATE__ or similar inline JSON.
Screenshots
I'd like to try implementing a Xiaohongshu extractor to address these issues. Would you please be open to this?
Source: kepano/defuddle