docs: Add deployment guide "deploy on AWS" #1679
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request adds comprehensive deployment guides for running Crawlee crawlers on AWS Lambda, addressing issue #698. The guides cover both HTTP-based crawlers (BeautifulSoupCrawler) and browser-based crawlers (PlaywrightCrawler), with different deployment strategies appropriate for each.
Changes:
- Added BeautifulSoup crawler AWS Lambda deployment guide with ZIP archive deployment method
- Added Playwright crawler AWS Lambda deployment guide with Docker container deployment method
- Updated sidebar configuration to expose the new AWS deployment guides
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| website/sidebars.js | Uncommented AWS deployment section with updated file references for BeautifulSoup and Playwright guides |
| docs/deployment/aws_lambda_beautifulsoup_crawler.mdx | Comprehensive guide for deploying BeautifulSoupCrawler to AWS Lambda using ZIP archives |
| docs/deployment/aws_lambda_playwright_crawler.mdx | Comprehensive guide for deploying PlaywrightCrawler to AWS Lambda using Docker containers |
| docs/deployment/code_examples/aws/beautifulsoup_crawler_lambda.py | Example Lambda handler code for BeautifulSoup crawler with MemoryStorageClient configuration |
| docs/deployment/code_examples/aws/playwright_crawler_lambda.py | Example Lambda handler code for Playwright crawler with browser launch options for serverless environments |
| docs/deployment/code_examples/aws/playwright_dockerfile | Dockerfile configuration for building Playwright crawler Lambda container image |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
I'm also not sure about this. The playwright version explains how to pack the crawler in a Docker image, which might be valuable in other use cases, too. By the way, I don't think the JS version of the guides is set in stone, if we can improve both language versions, we should do it. |
|
I will combine these guides into one. This will also allow me to create a more compact title that will fit in the Sidebar... |
3 participants
Description
Issues