|
2 | 2 |
|
3 | 3 | English | [简体中文](https://github.com/coder-hxl/x-crawl/blob/main/docs/cn.md)
|
4 | 4 |
|
5 |
| -x-crawl is a flexible nodejs crawler library. It can crawl pages in batches, network requests in batches, download file resources in batches, polling and crawling, etc. Supports asynchronous/synchronous mode crawling. Running on nodejs, the usage is flexible and simple, friendly to JS/TS developers. |
| 5 | +x-crawl is a flexible nodejs crawler library. It can crawl pages in batches, network requests in batches, download file resources in batches, polling and crawling, etc. Flexible and simple to use, friendly to JS/TS developers. |
6 | 6 |
|
7 |
| -> If you feel good, you can give [x-crawl repository](https://github.com/coder-hxl/x-crawl) a Star to support it, your Star will be the motivation for my update. |
| 7 | +> If you like x-crawl, you can give [x-crawl repository](https://github.com/coder-hxl/x-crawl) a Star to support it, which is its recognition. |
8 | 8 |
|
9 | 9 | ## Features
|
10 | 10 |
|
11 | 11 | - **🔥 Async/Sync** - Just change the mode property to toggle async/sync crawling mode.
|
12 | 12 | - **⚙️ Multiple functions** - Batch crawling of pages, batch network requests, batch download of file resources, polling crawling, etc.
|
13 |
| -- **🖋️ Flexible writing style** - Multiple crawling configurations and ways to get crawling results. |
14 | 13 | - **⏱️ Interval crawling** - no interval/fixed interval/random interval, you can use/avoid high concurrent crawling.
|
15 |
| -- **🚀 Crawl Repost** - Under development. |
| 14 | +- **🔄 Crawl retry** - under development. |
| 15 | +- **🚀 Priority Queue** - under development. |
16 | 16 | - **☁️ Crawl SPA** - Batch crawl SPA (Single Page Application) to generate pre-rendered content (ie "SSR" (Server Side Rendering)).
|
17 | 17 | - **⚒️ Controlling Pages** - Headless browsers can submit forms, keystrokes, event actions, generate screenshots of pages, etc.
|
18 |
| -- **🧾 Capture Record** - Capture and record the crawled results, and highlight the reminders. |
| 18 | +- **🧾 Capture Record** - Capture and record the crawled results, and highlight them on the console. |
| 19 | +- **🖋️ Flexible writing style** - It is very flexible to adapt to various crawling configurations and obtain crawling results. |
19 | 20 | - **🦾TypeScript** - Own types, implement complete types through generics.
|
20 | 21 |
|
21 | 22 | ## Relationship with puppeteer
|
|
0 commit comments