Baike.dev
All toolsOnline toolsAI codingTrendingOpen sourceNewsSubmit
Log in
< Back to tools
I

ineed

> 编程语言
Open source

Web scraping and HTML-reprocessing. The easy way.

386 stars0 likes0 views
WebsiteGitHub

About

Web scraping and HTML-reprocessing. The easy way.

Web scraping and HTML-reprocessing. The easy way.

ineed allows you collect useful data from web pages using simple and nice API. Let's collect images, hyperlinks, scripts and stylesheets from http://google.com:

javascript

var ineed = require('ineed');

ineed.collect.images.hyperlinks.scripts.stylesheets.from('http://google.com',
    function (err, response, result) {
        console.log(result);
    });

Also, it can be used to build HTML-reprocessing pipelines (like jch/html-pipeline but for Node) with elegance. E.g. we have the following html:

xml

    
    

This is an H1
=============
*   Red
*   Green
*   Blue

Let's render it's `

This is an H1

Red

Green

Blue

…

$ npm install ineed


## Usage
The general form:

#### ```ineed.[....].```

### from* methods
##### .fromHtml(html)
Accepts `html` string as an argument and synchronously returns `result` of the action.

*Example:*
```js
var result = ineed.collect.texts.fromHtml('
');
.from(options, callback)

Asynchronously loads specified page and invokes callback(err, response, result). It passes response object to callback in case if you need response headers or status codes. The first argument can be either a url or an options object. The set of options is the same as in @mikeal's request, so you can use POST method, set request headers or do any other advanced setup for the page request.

Examples:

javascript
ineed.collect.jsCode.from('https://github.com', function (err, response, result) {
    console.log(response.statusCode);
    console.log(response.headers);    
});
javascript
ineed.collect.title.from({
    url: 'https://test.domain/',
    method: 'POST',
    form: 'input=test'
}, function (err, response, result) {
    console.log(result);
});

.collect action

Collects information specified by plugin set. The result of the action is an object that contains individual plugin outputs as properties.

Example:

javascript
var result = ineed.collect.texts.images.scripts.fromHtml(html);

will produce result:

javascript
{
    "texts" : ,
    "images" : ,
    "scripts" : 
}

…

js
//Delete all HTML comments and render emoji
var result = ineed.reprocess
    .comments(function() {
        return null;
    })
    .texts(function(text, escapeHtml) {
        return escapeHtml(text).replace(/:beer:/g, '');
    })
    .fromHtml(html);

…

js
ineed
    .using(myPlugin1)
    .using(myPlugin2)
    .collect
    ...

Common structure

Plugins of both kinds are objects and in addition to the kind-specific properties they should have the following properties:

.extends

Indicates which action will be extended by plugin. Can be 'collect' or 'reprocess'. Required property.

.name

Name of the plugin. Required property. It should reflect the target of the plugin action. Plugin will be accessable under the name in the .collect or .reprocess objects and will be used as result property name for .collect action. E.g. plugin has name='tagNames'. If it extends .collect:

javascript
//Enable plugin and use it
var result = ineed.using(plugin).collect.tagNames.fromHtml(html);

//Access plugin results
var pluginResults = result.tagNames;

If it extends .reprocess:

javascript
 var reprocessedHtml = ineed
        .using(plugin)
        .reprocess
        .tagNames(function (tagName) {
            if (tagName === 'applet')
                return 'object';
        })
        .fromHtml(html);

…

js
{
    name: [String],
    publicId: [String],
    systemId: [String]
}
  • .onStartTag(startTag)

Where startTag:

javascript
{
    tagName: [String],
    attrs: [Array],
    selfClosing: [Boolean]
}
  • .onEndTag(tagName)
  • .onText(text)
  • .onComment(commentText)

Collecting plugins

Collecting plugins in addition to common properties should have .getCollection() method that should return items collected by plugin.

Example of the collecting plugin:

javascript
//Collects tagNames in 
var plugin = {
    extends: 'collect',
    name: 'tagNamesInBody',

    init: function (ctx) {
        this.ctx = ctx;
        this.tagNames = [];
    },

    addTagName: function (tagName) {
        if (this.ctx.inBody && this.tagNames.indexOf(tagName)  start and end tags
        if (tagName === 'noscript')
            return null;
        
        //Replace  with 

        if(tagName === 'div')
            return 'p';
            
        return tagName;
    }).fromHtml(html);

Testing

$ npm test

Questions or suggestions?

If you have any questions, please feel free to create an issue here on github.

Author

Ivan Nikulin ([email protected])

Issues· 0 open

View all issuesOpen on GitHub

No open issues yet, or sync has not completed.

> Tags

JavaScript

No comments yet. Be the first to share.

> Details

PublishedAug 1, 2026
UpdatedSep 17, 2026
Category编程语言
PricingOpen source

> Related tools

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言