特性: 用于 Pydoll 的 Scrapy 插件 (`scrapy-pydoll`)

作者: thalissonvs创建于 2025年8月22日更新于 2026年7月20日

标签enhancementfuture planning

Proposed API

Installable optional plugin: pip install scrapy-pydoll
Enable via settings:

python

PYDOLL_ENABLED = True
PYDOLL_CONCURRENCY = 2
PYDOLL_BROWSER_OPTIONS = { "geolocation": "GB", "headless": True }

Per-request opt-in (meta) or helper Request:

python

yield scrapy.Request(
    url,
    meta={
        "pydoll": {
            "actions": [
                {"type": "wait", "for": "networkidle"},
                {"type": "click", "selector": "#show-more"},
            ],
            "timeout": 15000,
        },
        "cookiejar": "sessionA",
    },
    callback=self.parse_page,
)

内容来源: autoscrape-labs/pydoll

查看 GitHub 原文在 GitHub 查看讨论