Skip to content

Add custom headers support and also nonrender#16

Open
lee-hodg wants to merge 5 commits intoattilaolah:masterfrom
lee-hodg:master
Open

Add custom headers support and also nonrender#16
lee-hodg wants to merge 5 commits intoattilaolah:masterfrom
lee-hodg:master

Conversation

@lee-hodg
Copy link

Implements

  • Allow custom headers setting, which is useful for some sites you may wish to scrape
  • Also when doing a crawl we may want the process url to have the &norender parameter to stop javascript rendering to speed up performance of the crawl, so here allow for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant