Docs: Default value

Author: coder-hxl

README.md

@@ -8,12 +8,12 @@ x-crawl is a flexible Node.js multifunctional crawler library. Used to crawl pag
 
 ## Features
 
-- **🔥 Async/Sync** - Just change the mode attribute value to switch async or sync crawling mode.
+- **🔥 AsyncSync** - Just change the mode attribute value to switch async or sync crawling mode.
 - **⚙️Multiple functions** - It can crawl pages, crawl interfaces, crawl files and polling crawls, and supports crawling single or multiple.
 - **🖋️ Flexible writing style** - Simple target configuration, detailed target configuration, mixed target array configuration and advanced configuration, the same crawling API can adapt to multiple configurations.
 - **👀Device Fingerprinting** - Zero configuration or custom configuration to avoid fingerprinting to identify and track us from different locations.
 - **⏱️ Interval Crawling** - No interval, fixed interval and random interval can generate or avoid high concurrent crawling.
-- **🔄 Retry on failure** - Global settings, local settings and individual settings. It can avoid crawling failure caused by temporary problems.
+- **🔄 Retry on failure** - Global settings, local settings and individual settings, It can avoid crawling failure caused by temporary problems.
 - **🚀 Priority Queue** - According to the priority of a single crawling target, it can be crawled ahead of other targets.
 - **☁️ Crawl SPA** - Crawl SPA (Single Page Application) to generate pre-rendered content (aka "SSR" (Server Side Rendering)).
 - **⚒️ Controlling Pages** - Headless browsers can submit forms, keystrokes, event actions, generate screenshots of pages, etc.
@@ -1137,6 +1137,15 @@ export interface XCrawlConfig extends CrawlCommonConfig {
 }
 ```
 
+**Default Value**
+
+- mode: 'async'
+- enableRandomFingerprint: true
+- baseUrl: undefined
+- intervalTime: undefined
+- crawlPage: undefined
+  - launchBrowser: undefined
+
 #### Detail target config
 
 ##### CrawlPageDetailTargetConfig
@@ -1159,6 +1168,14 @@ export interface CrawlPageDetailTargetConfig extends CrawlCommonConfig {
 }
 ```
 
+**Default Value**
+
+- headers: undefined
+- method: undefined
+- priority: undefined
+- viewport: undefined
+- fingerprint: undefined
+
 ##### CrawlDataDetailTargetConfig
 
 ```ts
@@ -1173,6 +1190,16 @@ export interface CrawlDataDetailTargetConfig extends CrawlCommonConfig {
 }
 ```
 
+**Default Value**
+
+- method: 'GET'
+
+- headers: undefined
+- params: undefined
+- data: undefined
+- priority: undefined
+- fingerprint: undefined
+
 ##### CrawlFileDetailTargetConfig
 
 ```ts
@@ -1187,6 +1214,15 @@ export interface CrawlFileDetailTargetConfig extends CrawlCommonConfig {
 }
 ```
 
+**Default Value**
+
+- headers: undefined
+- priority: undefined
+- storeDir: \_\_dirname
+- fileName: string
+- extension: string
+- fingerprint: undefined
+
 #### Advanced config
 
 ##### CrawlPageAdvancedConfig
@@ -1210,6 +1246,15 @@ export interface CrawlPageAdvancedConfig extends CrawlCommonConfig {
 }
 ```
 
+**Default Value**
+
+- intervalTime: undefined
+- fingerprint: undefined
+- headers: undefined
+- cookies: undefined
+- viewport: undefined
+- onCrawlItemComplete: undefined
+
 ##### CrawlDataAdvancedConfig
 
 ```ts
@@ -1224,6 +1269,13 @@ export interface CrawlDataAdvancedConfig<T> extends CrawlCommonConfig {
 }
 ```
 
+**Default Value**
+
+- intervalTime: undefined
+- fingerprint: undefined
+- headers: undefined
+- onCrawlItemComplete: undefined
+
 ##### CrawlFileAdvancedConfig
 
 ```ts
@@ -1246,6 +1298,16 @@ export interface CrawlFileAdvancedConfig extends CrawlCommonConfig {
 }
 ```
 
+**Default Value**
+
+- intervalTime: undefined
+- fingerprint: undefined
+- headers: undefined
+- storeDir: \_\_dirname
+- extension: string
+- onCrawlItemComplete: undefined
+- onBeforeSaveItemFile: undefined
+
 #### StartPollingConfig
 
 ```ts
@@ -1256,6 +1318,12 @@ export interface StartPollingConfig {
 }
 ```
 
+**Default Value**
+
+- d: undefined
+- h: undefined
+- m: undefined
+
 #### Crawl other config
 
 ##### CrawlCommonConfig
@@ -1268,6 +1336,12 @@ export interface CrawlCommonConfig {
 }
 ```
 
+**Default Value**
+
+- timeout: 10000
+- proxy: undefined
+- maxRetry: 0
+
 ##### DetailTargetFingerprintCommon
 
 ```ts
@@ -1281,6 +1355,15 @@ export interface DetailTargetFingerprintCommon {
 }
 ```
 
+**Default Value**
+
+- userAgent: undefined
+- ua: undefined
+- platform: undefined
+- platformVersion: undefined
+- mobile: undefined
+- acceptLanguage: undefined
+
 ##### AdvancedFingerprintCommon
 
 ```ts
@@ -1294,6 +1377,15 @@ export interface AdvancedFingerprintCommon {
 }
 ```
 
+**Default Value**
+
+- userAgents: undefined
+- uas: undefined
+- platforms: undefined
+- platformVersions: undefined
+- mobiles: undefined
+- acceptLanguages: undefined
+
 ##### Mobile
 
 ```ts

docs/cn.md

@@ -8,12 +8,12 @@ x-crawl 是一个灵活的 Node.js 多功能爬虫库。用于爬页面、爬接
 
 ## 特征
 
-- **🔥 异步/同步** - 只需更改一下 mode 属性值即可切换异步或同步爬取模式。
+- **🔥 异步同步** - 只需更改一下 mode 属性值即可切换异步或同步爬取模式。
 - **⚙️ 多种功能** - 可爬页面、爬接口、爬文件以及轮询爬，并且支持爬取单个或多个。
 - **🖋️ 写法灵活** - 简单目标配置、详细目标配置、混合目标数组配置以及进阶配置，同种爬取 API 适配多种配置。
 - **👀 设备指纹** - 零配置或自定义配置，即可避免通过指纹识别从不同位置识别并跟踪我们。
 - **⏱️ 间隔爬取** - 无间隔、固定间隔以及随机间隔，即可产生或避免高并发爬取。
-- **🔄 失败重试** - 全局设置、局部设置以及单独设置。即可避免因一时问题而造成爬取失败。
+- **🔄 失败重试** - 全局设置、局部设置以及单独设置, 即可避免因一时问题而造成爬取失败。
 - **🚀 优先队列** - 根据单个爬取目标的优先级可以优先于其他目标提前进行爬取。
 - **☁️ 爬取 SPA** - 爬取 SPA（单页应用程序）生成预渲染内容（即“SSR”（服务器端渲染））。
 - **⚒️ 控制页面** - 无头浏览器可以表单提交、键盘输入、事件操作、生成页面的屏幕截图等。
@@ -1128,6 +1128,15 @@ export interface XCrawlConfig extends CrawlCommonConfig {
 }
 ```
 
+**默认值**
+
+- mode: 'async'
+- enableRandomFingerprint: true
+- baseUrl: undefined
+- intervalTime: undefined
+- crawlPage: undefined
+  - launchBrowser: undefined
+
 #### Detail target config
 
 ##### CrawlPageDetailTargetConfig
@@ -1150,6 +1159,14 @@ export interface CrawlPageDetailTargetConfig extends CrawlCommonConfig {
 }
 ```
 
+**默认值**
+
+- headers: undefined
+- method: undefined
+- priority: undefined
+- viewport: undefined
+- fingerprint: undefined
+
 ##### CrawlDataDetailTargetConfig
 
 ```ts
@@ -1164,6 +1181,16 @@ export interface CrawlDataDetailTargetConfig extends CrawlCommonConfig {
 }
 ```
 
+**默认值**
+
+- method: 'GET'
+
+- headers: undefined
+- params: undefined
+- data: undefined
+- priority: undefined
+- fingerprint: undefined
+
 ##### CrawlFileDetailTargetConfig
 
 ```ts
@@ -1178,6 +1205,15 @@ export interface CrawlFileDetailTargetConfig extends CrawlCommonConfig {
 }
 ```
 
+**默认值**
+
+- headers: undefined
+- priority: undefined
+- storeDir: \_\_dirname
+- fileName: string
+- extension: string
+- fingerprint: undefined
+
 #### Advanced config
 
 ##### CrawlPageAdvancedConfig
@@ -1201,6 +1237,15 @@ export interface CrawlPageAdvancedConfig extends CrawlCommonConfig {
 }
 ```
 
+**默认值**
+
+- intervalTime: undefined
+- fingerprint: undefined
+- headers: undefined
+- cookies: undefined
+- viewport: undefined
+- onCrawlItemComplete: undefined
+
 ##### CrawlDataAdvancedConfig
 
 ```ts
@@ -1215,6 +1260,13 @@ export interface CrawlDataAdvancedConfig<T> extends CrawlCommonConfig {
 }
 ```
 
+**默认值**
+
+- intervalTime: undefined
+- fingerprint: undefined
+- headers: undefined
+- onCrawlItemComplete: undefined
+
 ##### CrawlFileAdvancedConfig
 
 ```ts
@@ -1237,6 +1289,16 @@ export interface CrawlFileAdvancedConfig extends CrawlCommonConfig {
 }
 ```
 
+**默认值**
+
+- intervalTime: undefined
+- fingerprint: undefined
+- headers: undefined
+- storeDir: \_\_dirname
+- extension: string
+- onCrawlItemComplete: undefined
+- onBeforeSaveItemFile: undefined
+
 #### StartPollingConfig
 
 ```ts
@@ -1247,6 +1309,12 @@ export interface StartPollingConfig {
 }
 ```
 
+**默认值**
+
+- d: undefined
+- h: undefined
+- m: undefined
+
 #### Crawl other config
 
 ##### CrawlCommonConfig
@@ -1259,6 +1327,12 @@ export interface CrawlCommonConfig {
 }
 ```
 
+**默认值**
+
+- timeout: 10000
+- proxy: undefined
+- maxRetry: 0
+
 ##### DetailTargetFingerprintCommon
 
 ```ts
@@ -1272,6 +1346,15 @@ export interface DetailTargetFingerprintCommon {
 }
 ```
 
+**默认值**
+
+- userAgent: undefined
+- ua: undefined
+- platform: undefined
+- platformVersion: undefined
+- mobile: undefined
+- acceptLanguage: undefined
+
 ##### AdvancedFingerprintCommon
 
 ```ts
@@ -1285,6 +1368,15 @@ export interface AdvancedFingerprintCommon {
 }
 ```
 
+**默认值**
+
+- userAgents: undefined
+- uas: undefined
+- platforms: undefined
+- platformVersions: undefined
+- mobiles: undefined
+- acceptLanguages: undefined
+
 ##### Mobile
 
 ```ts

package.json

@@ -12,8 +12,8 @@
     "crawl",
     "crawler",
     "spider",
-    "flexible",
     "fingerprint",
+    "flexible",
     "multifunction"
   ],
   "main": "src/index.ts",
@@ -32,7 +32,7 @@
   "dependencies": {
     "chalk": "4.1.2",
     "https-proxy-agent": "^5.0.1",
-    "puppeteer": "19.9.1",
+    "puppeteer": "19.10.0",
     "x-crawl": "link:"
   },
   "devDependencies": {