百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
I

ineed

> 编程语言
开源

Web 爬虫和 HTML 重新处理。简单易用。

386 stars0 点赞0 次浏览
访问官网GitHub

工具介绍

Web 爬虫和 HTML 重新处理。简单易用。

Web scraping and HTML-reprocessing. The easy way.

ineed allows you collect useful data from web pages using simple and nice API. Let's collect images, hyperlinks, scripts and stylesheets from http://google.com:

javascript

var ineed = require('ineed');

ineed.collect.images.hyperlinks.scripts.stylesheets.from('http://google.com',
    function (err, response, result) {
        console.log(result);
    });

Also, it can be used to build HTML-reprocessing pipelines (like jch/html-pipeline but for Node) with elegance. E.g. we have the following html:

xml

    
    

This is an H1
=============
*   Red
*   Green
*   Blue

Let's render it's `

This is an H1

Red

Green

Blue

…

$ npm install ineed


## Usage
The general form:

#### ```ineed.[....].```

### from* methods
##### .fromHtml(html)
Accepts `html` string as an argument and synchronously returns `result` of the action.

*Example:*
```js
var result = ineed.collect.texts.fromHtml('
');
.from(options, callback)

Asynchronously loads specified page and invokes callback(err, response, result). It passes response object to callback in case if you need response headers or status codes. The first argument can be either a url or an options object. The set of options is the same as in @mikeal's request, so you can use POST method, set request headers or do any other advanced setup for the page request.

Examples:

javascript
ineed.collect.jsCode.from('https://github.com', function (err, response, result) {
    console.log(response.statusCode);
    console.log(response.headers);    
});
javascript
ineed.collect.title.from({
    url: 'https://test.domain/',
    method: 'POST',
    form: 'input=test'
}, function (err, response, result) {
    console.log(result);
});

.collect action

Collects information specified by plugin set. The result of the action is an object that contains individual plugin outputs as properties.

Example:

javascript
var result = ineed.collect.texts.images.scripts.fromHtml(html);

will produce result:

javascript
{
    "texts" : ,
    "images" : ,
    "scripts" : 
}

…

js
//Delete all HTML comments and render emoji
var result = ineed.reprocess
    .comments(function() {
        return null;
    })
    .texts(function(text, escapeHtml) {
        return escapeHtml(text).replace(/:beer:/g, '');
    })
    .fromHtml(html);

…

js
ineed
    .using(myPlugin1)
    .using(myPlugin2)
    .collect
    ...

Common structure

Plugins of both kinds are objects and in addition to the kind-specific properties they should have the following properties:

.extends

Indicates which action will be extended by plugin. Can be 'collect' or 'reprocess'. Required property.

.name

Name of the plugin. Required property. It should reflect the target of the plugin action. Plugin will be accessable under the name in the .collect or .reprocess objects and will be used as result property name for .collect action. E.g. plugin has name='tagNames'. If it extends .collect:

javascript
//Enable plugin and use it
var result = ineed.using(plugin).collect.tagNames.fromHtml(html);

//Access plugin results
var pluginResults = result.tagNames;

If it extends .reprocess:

javascript
 var reprocessedHtml = ineed
        .using(plugin)
        .reprocess
        .tagNames(function (tagName) {
            if (tagName === 'applet')
                return 'object';
        })
        .fromHtml(html);

…

js
{
    name: [String],
    publicId: [String],
    systemId: [String]
}
  • .onStartTag(startTag)

Where startTag:

javascript
{
    tagName: [String],
    attrs: [Array],
    selfClosing: [Boolean]
}
  • .onEndTag(tagName)
  • .onText(text)
  • .onComment(commentText)

Collecting plugins

Collecting plugins in addition to common properties should have .getCollection() method that should return items collected by plugin.

Example of the collecting plugin:

javascript
//Collects tagNames in 
var plugin = {
    extends: 'collect',
    name: 'tagNamesInBody',

    init: function (ctx) {
        this.ctx = ctx;
        this.tagNames = [];
    },

    addTagName: function (tagName) {
        if (this.ctx.inBody && this.tagNames.indexOf(tagName)  start and end tags
        if (tagName === 'noscript')
            return null;
        
        //Replace  with 

        if(tagName === 'div')
            return 'p';
            
        return tagName;
    }).fromHtml(html);

Testing

$ npm test

Questions or suggestions?

If you have any questions, please feel free to create an issue here on github.

Author

Ivan Nikulin ([email protected])

Issues· 0 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

JavaScript

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月17日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言