Skip to content

Commit 044a5d3

Browse files
authored
Merge pull request #98 from extractus/6.3.0
v6.3.0
2 parents 93c7dcb + b7d94a1 commit 044a5d3

File tree

21 files changed

+83
-8358
lines changed

21 files changed

+83
-8358
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,3 +17,4 @@ package-lock.json
1717
pnpm-lock.yaml
1818

1919
output.json
20+
deno.lock

README.md

Lines changed: 38 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -174,9 +174,14 @@ await extract('https://news.google.com/rss', {
174174

175175
##### `fetchOptions` *optional*
176176

177-
You can use this param to set request headers to fetch.
177+
`fetchOptions` is an object that can have the following properties:
178178

179-
For example:
179+
- `headers`: to set request headers
180+
- `proxy`: another endpoint to forward the request to
181+
- `agent`: a HTTP proxy agent
182+
- `signal`: AbortController signal or AbortSignal timeout to terminate the request
183+
184+
For example, you can use this param to set request headers to fetch as below:
180185

181186
```js
182187
import { extract } from '@extractus/feed-extractor'
@@ -234,6 +239,37 @@ console.log(feed)
234239

235240
For more info about [https-proxy-agent](https://www.npmjs.com/package/https-proxy-agent), check [its repo](https://github.com/TooTallNate/proxy-agents).
236241

242+
By default, there is no request timeout. You can use the option `signal` to cancel request at the right time.
243+
244+
The common way is to use AbortControler:
245+
246+
```js
247+
const controller = new AbortController()
248+
249+
// stop after 5 seconds
250+
setTimeout(() => {
251+
controller.abort()
252+
}, 5000)
253+
254+
const data = await extract(url, null, {
255+
signal: controller.signal,
256+
})
257+
```
258+
259+
A newer solution is AbortSignal's `timeout()` static method:
260+
261+
```js
262+
// stop after 5 seconds
263+
const data = await extract(url, null, {
264+
signal: AbortSignal.timeout(5000),
265+
})
266+
```
267+
268+
For more info:
269+
270+
- [AbortController constructor](https://developer.mozilla.org/en-US/docs/Web/API/AbortController)
271+
- [AbortSignal: timeout() static method](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal/timeout_static)
272+
237273

238274
### `extractFromJson()`
239275

build.js

Lines changed: 0 additions & 70 deletions
This file was deleted.

build.test.js

Lines changed: 0 additions & 47 deletions
This file was deleted.

dist/cjs/feed-extractor.js

Lines changed: 0 additions & 5587 deletions
This file was deleted.

dist/cjs/index.d.ts

Lines changed: 0 additions & 90 deletions
This file was deleted.

dist/cjs/package.json

Lines changed: 0 additions & 5 deletions
This file was deleted.

0 commit comments

Comments
 (0)