基于luin/readability的二次开发
extract article title and content from html page
- Optimized for more websites.
- Supporting HTML5 tags(
article,section) and Microdata API. - Focusing on both accuracy and performance. 4x times faster than arc90's version.
- Supporting encodings such as GBK and GB2312.
- Converting relative urls to absolute for images and links automatically(Thank Guillermo Baigorria & Tom Sutton).
$ npm install wahack/readability
Note that as of our 2.0.0 release, this module only works with Node.js >= 2.0. In the meantime you are still welcome to install a release in the 1.x series(by npm install node-readability@1) if you use an older Node.js version.
read(html [, options], callback)
Where
- html url or html code.
- options is an optional options object
- callback is the callback to run -
callback(error, article, meta)
Example
var read = require('node-readability');
read('<html><body>hello</body></html>', function(err, article, meta) {
// Main Article
console.log(article.content);
// Title
console.log(article.title);
// HTML Source Code
console.log(article.html);
// DOM
console.log(article.document);
// Close article to clean up jsdom and prevent leaks
article.close();
});node-readability has two additional options:
cleanRulerswhich allow set your own validation rule for tags.
If true rule is valid, otherwise no. options.cleanRulers = [callback(obj, tagName)]
read(url, {
cleanRulers: [
function(obj, tag) {
if(tag === 'object') {
if(obj.getAttribute('class') === 'BrightcoveExperience') {
return true;
}
}
}
]}, function(err, article) {
//...
});The article content of the web page. Return false if failed.
The article title of the web page. It's may not same to the text in the <title> tag.
A string containing all the text found on the page
The original html of the web page.
The document of the web page generated by jsdom. You can use it to access the DOM directly(for example, article.document.getElementById('main')).
This lib is using jsdom to parse HTML instead of cheerio because some data such as image size and element visibility isn't able to acquire when using cheerio, which will significantly affect the result.
This code is under the Apache License 2.0. http://www.apache.org/licenses/LICENSE-2.0
