Ruby gem 可全面检查网页。它会抓取指定的 URL,并返回元数据、链接、图像等。
Ruby gem to inspect web pages. It scrapes a given URL and returns its title, description, meta tags, links, images, and more.
Add this line to your application's Gemfile:
gem 'webinspector'And then execute:
$ bundleOr install it yourself as:
$ gem install webinspectorpage = WebInspector.new('http://example.com')page = WebInspector.new('http://example.com', {
timeout: 30, # Request timeout in seconds (default: 30)
retries: 3, # Number of retries (default: 3)
headers: {'User-Agent': 'Custom UA'} # Custom HTTP headers
})page.response.status # 200
page.response.headers # { "server"=>"apache", "content-type"=>"text/html; charset=utf-8", ... }
page.status_code # 200
page.success? # true if the page was loaded successfully
page.error_message # returns the error message if anypage.url # URL of the page
page.scheme # Scheme of the page (http, https)
page.host # Hostname of the page (like, example.com, without the scheme)
page.port # Port of the page
page.title # title of the page from the head section
page.description # description of the page
page.links # array of all links found on the page (absolute URLs)
page.images # array of all images found on the page (absolute URLs)
page.meta # meta tags of the page
page.favicon # favicon URL if availablepage.meta # all meta tags
page.meta['description'] # meta description
page.meta['keywords'] # meta keywords
page.meta['og:title'] # OpenGraph titlepage.domain_links('example.com') # returns only links pointing to example.com
page.domain_images('example.com') # returns only images hosted on example.compage.find(["ruby", "rails"]) # returns [{"ruby"=>3}, {"rails"=>1}]page.javascripts # array of all JavaScript files (absolute URLs)
page.stylesheets # array of all CSS stylesheets (absolute URLs)page.language # detected language code (e.g., "en", "es", "fr")page.structured_data # array of JSON-LD structured data objects
page.microdata # array of microdata items
page.json_ld # alias for structured_datapage.security_info # hash with security details: { secure: true, hsts: true, ... }page.load_time # page load time in seconds
page.size # page size in bytespage.content_type # content type header (e.g., "text/html; charset=utf-8")page.technologies # hash of detected technologies: { jquery: true, react: true, ... }page.tag_count # hash with counts of each HTML tag: { "div" => 45, "p" => 12, ... }page.feeds # array of RSS/Atom feed URLs found on the pagepage.social_links # hash of social media profiles: { facebook: "url", twitter: "url", ... }page.robots_txt_url # URL to robots.txt
page.sitemap_url # array of sitemap URLspage.cms_info # hash with CMS details: { name: "WordPress", version: "6.0", themes: [...], plugins: [...] }page.accessibility_score # hash with score (0-100) and details: { score: 85, details: [...] }page.mobile_friendly? # true if the page has viewport meta tag and responsive CSSpage.to_hash # returns a hash with all page dataNew Features:
feeds methodsocial_links methodcms_info method (WordPress, Drupal, Joomla, Shopify, Wix, Squarespace)accessibility_score methodmobile_friendly? methodrobots_txt_url and sitemap_url methodsImprovements:
Request module with valid? and ssl? methods for better URL validationMeta module with author and publisher extractionThe WebInspector gem is released under the MIT License.
git checkout -b my-new-feature)git commit -am 'Add some feature')git push origin my-new-feature)暂无开放 Issues,或尚未同步最近议题。