Skip to content

Commit 73564c4

Browse files
coder-hxlcoder-hxl
committed
Docs: Configure Title update
1 parent e240126 commit 73564c4

File tree

3 files changed

+99
-66
lines changed

3 files changed

+99
-66
lines changed

‎README.md

Lines changed: 33 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -62,14 +62,25 @@ The crawlPage API has [puppeteer](https://github.com/puppeteer/puppeteer) built
6262
- [Type](#Type-1)
6363
- [Example](#Example-2)
6464
- [Config](#Config)
65+
- [1.Simple target config - string](#1.Simple-target-config---string)
66+
- [2.Detailed target config - CrawlPageDetailTargetConfig](#2.Detailed-target-config---CrawlPageDetailTargetConfig)
67+
- [3.Mixed target array config - (string | CrawlPageDetailTargetConfig)[]](<#3.Mixed-target-array-config---(string-|-CrawlPageDetailTargetConfig)[]>)
68+
- [4.Advanced config - CrawlPageAdvancedConfig](#4.Advanced-config---CrawlPageAdvancedConfig)
6569
- [crawlData](#crawlData)
6670
- [Type](#Type-2)
6771
- [Example](#Example-3)
6872
- [Config](#Config-1)
73+
- [1.Simple target config - string](#1.Simple-target-config---string-1)
74+
- [2.Detailed target config - CrawlDataDetailTargetConfig](#2.Detailed-target-config---CrawlDataDetailTargetConfig)
75+
- [3.Mixed target array config - (string | CrawlDataDetailTargetConfig)[]](<#3.Mixed-target-array-config---(string-|-CrawlDataDetailTargetConfig)[]>)
76+
- [4.Advanced config - CrawlDataAdvancedConfig](#4.Advanced-config---CrawlDataAdvancedConfig)
6977
- [crawlFile](#crawlFile)
7078
- [Type](#Type-3)
7179
- [Example](#Example-4)
7280
- [Config](#Config-2)
81+
- [1.Detailed target config - CrawlFileDetailTargetConfig](#1.Detailed-target-config---CrawlFileDetailTargetConfig)
82+
- [2.Detailed target array config - CrawlFileDetailTargetConfig[]](#2.Detailed-target-array-config---CrawlFileDetailTargetConfig[])
83+
- [3.Advanced config - CrawlFileAdvancedConfig](#3.Advanced-config---CrawlFileAdvancedConfig)
7384
- [crawlPolling](#crawlPolling)
7485
- [Type](#Type-4)
7586
- [Example](#Example-5)
@@ -690,12 +701,12 @@ myXCrawl.crawlPage('https://www.example.com').then((res) => {
690701
691702
There are 4 types:
692703
693-
- string
694-
- CrawlPageDetailTargetConfig
695-
- (string | CrawlPageDetailTargetConfig)[]
696-
- CrawlPageAdvancedConfig
704+
- Simple target config - string
705+
- Detailed target config - CrawlPageDetailTargetConfig
706+
- Mixed target array config - (string | CrawlPageDetailTargetConfig)[]
707+
- Advanced config - CrawlPageAdvancedConfig
697708
698-
**1.string**
709+
##### 1.Simple target config - string
699710
700711
This is a simple target configuration. if you just want to simply crawl this page, you can try this way of writing:
701712
@@ -709,7 +720,7 @@ myXCrawl.crawlPage('https://www.example.com').then((res) => {})
709720
710721
The res you get will be an object.
711722
712-
**2. CrawlPageDetailTargetConfig**
723+
##### 2.Detailed target config - CrawlPageDetailTargetConfig
713724
714725
This is the detailed target configuration. if you want to crawl this page and need to retry on failure, you can try this way of writing:
715726
@@ -731,7 +742,7 @@ The res you get will be an object.
731742
732743
More configuration options can view [CrawlPageDetailTargetConfig](#CrawlPageDetailTargetConfig).
733744
734-
**3.(string | CrawlPageDetailTargetConfig)[]**
745+
##### 3.Mixed target array config - (string | CrawlPageDetailTargetConfig)[]
735746
736747
This is a mixed target array configuration. if you want to crawl multiple pages, and some pages need to fail and retry, you can try this way of writing:
737748
@@ -752,7 +763,7 @@ The res you get will be an array of objects.
752763
753764
More configuration options can view [CrawlPageDetailTargetConfig](#CrawlPageDetailTargetConfig).
754765
755-
**4. CrawlPageAdvancedConfig**
766+
##### 4.Advanced config - CrawlPageAdvancedConfig
756767
757768
This is an advanced configuration, targets is a mixed target array configuration. if you want to crawl multiple pages, and the request configuration (proxy, cookies, retry, etc.) does not want to be written repeatedly, if you need an interval, you can try this way of writing:
758769
@@ -847,12 +858,12 @@ myXCrawl
847858
848859
There are 4 types:
849860
850-
- string
851-
- CrawlDataDetailTargetConfig
852-
- (string | CrawlDataDetailTargetConfig)[]
853-
- CrawlDataAdvancedConfig<T>
861+
- Simple target config - string
862+
- Detailed target config - CrawlDataDetailTargetConfig
863+
- Mixed target array config - (string | CrawlDataDetailTargetConfig)[]
864+
- Advanced config - CrawlDataAdvancedConfig
854865
855-
**1.string**
866+
##### 1.Simple target config - string
856867
857868
This is a simple target configuration. if you just want to simply crawl the data, and the interface is GET, you can try this way of writing:
858869
@@ -866,7 +877,7 @@ myXCrawl.crawlData('https://www.example.com/api').then((res) => {})
866877
867878
The res you get will be an object.
868879
869-
**2. CrawlDataDetailTargetConfig**
880+
##### 2.Detailed target config - CrawlDataDetailTargetConfig
870881
871882
This is the detailed target configuration. if you want to crawl this data and need to retry on failure, you can try this way of writing:
872883
@@ -888,7 +899,7 @@ The res you get will be an object.
888899
889900
More configuration options can view [CrawlDataDetailTargetConfig](#CrawlDataDetailTargetConfig).
890901
891-
**3.(string | CrawlDataDetailTargetConfig)[]**
902+
##### 3.Mixed target array config - (string | CrawlDataDetailTargetConfig)[]
892903
893904
This is a mixed target array configuration. if you want to crawl multiple data, and some data needs to fail and retry, you can try this way of writing:
894905
@@ -909,7 +920,7 @@ The res you get will be an array of objects.
909920
910921
More configuration options can view [CrawlDataDetailTargetConfig](#CrawlDataDetailTargetConfig).
911922
912-
**4.CrawlDataAdvancedConfig**
923+
##### 4.Advanced config - CrawlDataAdvancedConfig
913924
914925
This is an advanced configuration, targets is a mixed target array configuration. if you want to crawl multiple data, and the request configuration (proxy, cookies, retry, etc.) does not want to be written repeatedly, if you need an interval, you can try this writing method:
915926
@@ -1001,12 +1012,12 @@ myXCrawl
10011012
10021013
There are 3 types:
10031014
1004-
- CrawlFileDetailTargetConfig
1015+
- Detailed target config - CrawlFileDetailTargetConfig
10051016
1006-
- CrawlFileDetailTargetConfig[]
1007-
- CrawlFileAdvancedConfig
1017+
- Detailed target array config - CrawlFileDetailTargetConfig[]
1018+
- Advanced config CrawlFileAdvancedConfig
10081019
1009-
**1. CrawlFileDetailTargetConfig**
1020+
##### 1.Detailed target config - CrawlFileDetailTargetConfig
10101021
10111022
This is the detailed target configuration. if you want to crawl this file and need to retry on failure, you can try this way of writing:
10121023
@@ -1030,7 +1041,7 @@ The res you get will be an object.
10301041
10311042
More configuration options can view [CrawlFileDetailTargetConfig](#CrawlFileDetailTargetConfig).
10321043
1033-
**2. CrawlFileDetailTargetConfig[]**
1044+
##### 2.Detailed target array config - CrawlFileDetailTargetConfig[]
10341045
10351046
This is the detailed target array configuration. if you want to crawl multiple files, and some data needs to be retried after failure, you can try this way of writing:
10361047
@@ -1051,7 +1062,7 @@ The res you get will be an array of objects.
10511062
10521063
More configuration options can view [CrawlFileDetailTargetConfig](#CrawlFileDetailTargetConfig).
10531064
1054-
**3. CrawlFileAdvancedConfig**
1065+
##### 3.Advanced config CrawlFileAdvancedConfig
10551066
10561067
This is an advanced configuration, targets is a mixed target array configuration. if you want to crawl multiple data, and the request configuration (storeDir, proxy, retry, etc.) does not want to be written repeatedly, and you need interval time, etc., you can try this way of writing:
10571068

‎docs/cn.md

Lines changed: 33 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -62,14 +62,25 @@ crawlPage API 内置了 [puppeteer](https://github.com/puppeteer/puppeteer) ，
6262
- [类型](#类型-1)
6363
- [示例](#示例-2)
6464
- [配置](#配置)
65+
- [1.简单目标配置 - string](#1.简单目标配置---string)
66+
- [2.详细目标配置 - CrawlPageDetailTargetConfig](#2.详细目标配置---CrawlPageDetailTargetConfig)
67+
- [3.混合目标数组配置 - (string | CrawlPageDetailTargetConfig)[]](<#3.混合目标数组配置---(string-|-CrawlPageDetailTargetConfig)[]>)
68+
- [4.进阶配置 - CrawlPageAdvancedConfig](#4.进阶配置---CrawlPageAdvancedConfig)
6569
- [crawlData](#crawlData)
6670
- [类型](#类型-2)
6771
- [示例](#示例-3)
6872
- [配置](#配置-1)
73+
- [1.简单目标配置 - string](#1.简单目标配置---string-1)
74+
- [2.详细目标配置 - CrawlDataDetailTargetConfig](#2.详细目标配置---CrawlDataDetailTargetConfig)
75+
- [3.混合目标数组配置 - (string | CrawlDataDetailTargetConfig)[]](<#3.混合目标数组配置---(string-|-CrawlDataDetailTargetConfig)[]>)
76+
- [4.进阶配置 - CrawlDataAdvancedConfig](#4.进阶配置---CrawlDataAdvancedConfig)
6977
- [crawlFile](#crawlFile)
7078
- [类型](#类型-3)
7179
- [示例](#示例-4)
7280
- [配置](#配置-2)
81+
- [1.详细目标配置 - CrawlFileDetailTargetConfig](#1.详细目标配置---CrawlFileDetailTargetConfig)
82+
- [2.详细目标数组配置 - CrawlFileDetailTargetConfig[]](2.详细目标数组配置---CrawlFileDetailTargetConfig[])
83+
- [3.进阶配置 - CrawlFileAdvancedConfig](#3.进阶配置---CrawlFileAdvancedConfig)
7384
- [startPolling](#startPolling)
7485
- [类型](#类型-4)
7586
- [示例](#示例-5)
@@ -678,12 +689,12 @@ myXCrawl.crawlPage('https://www.example.com').then((res) => {
678689
679690
一共有 4 种:
680691
681-
- string
682-
- CrawlPageDetailTargetConfig
683-
- (string | CrawlPageDetailTargetConfig)[]
684-
- CrawlPageAdvancedConfig
692+
- 简单目标配置 - string
693+
- 详细目标配置 - CrawlPageDetailTargetConfig
694+
- 混合目标数组配置 - (string | CrawlPageDetailTargetConfig)[]
695+
- 进阶配置 - CrawlPageAdvancedConfig
685696
686-
**1.string**
697+
##### 1.简单目标配置 - string
687698
688699
这是简单目标配置。如果你只想单纯爬一下这个页面，可以试试这种写法：
689700
@@ -697,7 +708,7 @@ myXCrawl.crawlPage('https://www.example.com').then((res) => {})
697708
698709
拿到的 res 将是一个对象。
699710
700-
**2.CrawlPageDetailTargetConfig**
711+
##### 2.详细目标配置 - CrawlPageDetailTargetConfig
701712
702713
这是详细目标配置。如果你想爬一下这个页面，并且需要失败重试之类的，可以试试这种写法：
703714
@@ -719,7 +730,7 @@ myXCrawl
719730
720731
更多配置选项可以查看 [CrawlPageDetailTargetConfig](#CrawlPageDetailTargetConfig) 。
721732
722-
**3.(string | CrawlPageDetailTargetConfig)[]**
733+
##### 3.混合目标数组配置 - (string | CrawlPageDetailTargetConfig)[]
723734
724735
这是混合目标数组配置。如果你想爬取多个页面，并且有些页面需要失败重试之类的，可以试试这种写法：
725736
@@ -740,7 +751,7 @@ myXCrawl
740751
741752
更多配置选项可以查看 [CrawlPageDetailTargetConfig](#CrawlPageDetailTargetConfig) 。
742753
743-
**4.CrawlPageAdvancedConfig**
754+
##### 4.进阶配置 - CrawlPageAdvancedConfig
744755
745756
这是进阶配置，targets 是混合目标数组配置。如果你想爬取多个页面，并且请求配置（proxy、cookies、重试等等）不想重复写，需要间隔时间的话，可以试试这种写法：
746757
@@ -836,12 +847,12 @@ myXCrawl
836847
837848
一共有 4 种:
838849
839-
- string
840-
- CrawlDataDetailTargetConfig
841-
- (string | CrawlDataDetailTargetConfig)[]
842-
- CrawlDataAdvancedConfig
850+
- 简单目标配置 - string
851+
- 详细目标配置 - CrawlDataDetailTargetConfig
852+
- 混合目标数组配置 - (string | CrawlDataDetailTargetConfig)[]
853+
- 进阶配置 - CrawlDataAdvancedConfig
843854
844-
**1.string**
855+
##### 1.简单目标配置 - string
845856
846857
这是简单目标配置。如果你只想单纯爬一下这个数据，并且该接口是 GET 方式的，可以试试这种写法：
847858
@@ -855,7 +866,7 @@ myXCrawl.crawlData('https://www.example.com/api').then((res) => {})
855866
856867
拿到的 res 将是一个对象。
857868
858-
**2.CrawlDataDetailTargetConfig**
869+
##### 2.详细目标配置 - CrawlDataDetailTargetConfig
859870
860871
这是详细目标配置。如果你想爬一下这个数据，并且需要失败重试之类的，可以试试这种写法：
861872
@@ -877,7 +888,7 @@ myXCrawl
877888
878889
更多配置选项可以查看 [CrawlDataDetailTargetConfig](#CrawlDataDetailTargetConfig) 。
879890
880-
**3.(string | CrawlDataDetailTargetConfig)[]**
891+
##### 3.混合目标数组配置 - (string | CrawlDataDetailTargetConfig)[]
881892
882893
这是混合目标数组配置。如果你想爬取多个数据，并且有些数据需要失败重试之类的，可以试试这种写法：
883894
@@ -898,7 +909,7 @@ myXCrawl
898909
899910
更多配置选项可以查看 [CrawlDataDetailTargetConfig](#CrawlDataDetailTargetConfig) 。
900911
901-
**4.CrawlDataAdvancedConfig**
912+
##### 4.进阶配置 - CrawlDataAdvancedConfig
902913
903914
这是进阶配置，targets 是混合目标数组配置。如果你想爬取多个数据，并且请求配置（proxy、cookies、重试等等）不想重复写，需要间隔时间的话，可以试试这种写法：
904915
@@ -990,11 +1001,11 @@ myXCrawl
9901001
9911002
一共有 3 种:
9921003
993-
- CrawlFileDetailTargetConfig
994-
- CrawlFileDetailTargetConfig[]
995-
- CrawlFileAdvancedConfig
1004+
- 详细目标配置 - CrawlFileDetailTargetConfig
1005+
- 详细目标数组配置 - CrawlFileDetailTargetConfig[]
1006+
- 进阶配置 - CrawlFileAdvancedConfig
9961007
997-
**1.CrawlFileDetailTargetConfig**
1008+
##### 1.详细目标配置 - CrawlFileDetailTargetConfig
9981009
9991010
这是详细目标配置。如果你想爬一下这个文件，并且需要失败重试之类的，可以试试这种写法：
10001011
@@ -1018,7 +1029,7 @@ myXCrawl
10181029
10191030
更多配置选项可以查看 [CrawlFileDetailTargetConfig](#CrawlFileDetailTargetConfig) 。
10201031
1021-
**2.CrawlFileDetailTargetConfig[]**
1032+
##### 2.详细目标数组配置 - CrawlFileDetailTargetConfig[]
10221033
10231034
这是详细目标数组配置。如果你想爬取多个文件，并且有些数据需要失败重试之类的，可以试试这种写法：
10241035
@@ -1039,7 +1050,7 @@ myXCrawl
10391050
10401051
更多配置选项可以查看 [CrawlFileDetailTargetConfig](#CrawlFileDetailTargetConfig) 。
10411052
1042-
**3.CrawlFileAdvancedConfig**
1053+
##### 3.进阶配置 - CrawlFileAdvancedConfig
10431054
10441055
这是进阶配置，targets 是混合目标数组配置。如果你想爬取多个数据，并且请求配置（storeDir、proxy、重试等等）不想重复写，需要间隔时间等等的话，可以试试这种写法：
10451056

0 commit comments

Comments
 (0)