Support loading/streaming data from s3 locations #1
Description
This is what's supported at the moment:
- Local - JSON, JSONL, HuggingFace parquet
- Remote
- HuggingFace Hub - HuggingFace parquet
Supporting S3 loading/streaming would entail supporting the following:
- Local - JSON, JSONL, HuggingFace parquet
- Remote
- HuggingFace Hub - HuggingFace parquet
- S3 - JSON, JSONL, HuggingFace parquet
Metadata
Metadata
Assignees
Labels
No labels