Python website crawler
This is an open source, multi-threaded website crawler written in Python. There is still room for a lot of improvements, so feel free to help out with the development.
Note: For now, this tool gathers links only.
shan18/Spider
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|