@@ -8,10 +8,11 @@ x-crawl is a flexible nodejs crawler library. It can crawl pages in batches, net
8
8
9
9
## Features
10
10
11
- - ** 🔥 Asynchronous/Synchronous ** - Support asynchronous/synchronous mode batch crawling.
11
+ - ** 🔥 Async/Sync ** - Just change the mode property to toggle async/sync crawling mode .
12
12
- ** ⚙️ Multiple functions** - Batch crawling of pages, batch network requests, batch download of file resources, polling crawling, etc.
13
13
- ** 🖋️ Flexible writing style** - Multiple crawling configurations and ways to get crawling results.
14
14
- ** ⏱️ Interval crawling** - no interval/fixed interval/random interval, you can use/avoid high concurrent crawling.
15
+ - ** 🚀 Crawl Repost** - Under development.
15
16
- ** ☁️ Crawl SPA** - Batch crawl SPA (Single Page Application) to generate pre-rendered content (ie "SSR" (Server Side Rendering)).
16
17
- ** ⚒️ Controlling Pages** - Headless browsers can submit forms, keystrokes, event actions, generate screenshots of pages, etc.
17
18
- ** 🧾 Capture Record** - Capture and record the crawled results, and highlight the reminders.
@@ -130,6 +131,7 @@ running result:
130
131
<div align =" center " >
131
132
<img src =" https://raw.githubusercontent.com/coder-hxl/x-crawl/main/assets/en/crawler-result.png " />
132
133
</div >
134
+
133
135
** Note:** Do not crawl at will, you can check the ** robots.txt** protocol before crawling. This is just to demonstrate how to use x-crawl.
134
136
135
137
## Core concepts
@@ -857,4 +859,4 @@ interface FileInfo {
857
859
858
860
## More
859
861
860
- If you have any ** questions ** or ** needs ** , please submit ** Issues in ** https :// github.com/coder-hxl/x-crawl/issues .
862
+ If you have ** problems , needs , good suggestions ** please raise ** Issues ** in https :// github.com/coder-hxl/x-crawl/issues.
0 commit comments