- make sure that you have Go installed
- run
go build -o crawler && ./crawler <URL> <maxConcurrency> <maxPages>
This webcrawler fetches all internal links on a webpage. It uses concurrency to be faster and can be limited to a max number of pages.
-
Notifications
You must be signed in to change notification settings - Fork 0
davidelng/webcrawler
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
Webcrawler written in Go
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published