Context:
Numerblox supports Google Cloud Buckets (GCS) in downloaders and submitters with download_file_from_gcs, upload_file_to_gcs, download_directory_from_gcs and upload_directory_to_gcs. However, it always requires the user to have the Numerai data saved locally first.
Desired behavior:
- Download Numerai data directly to a GCS bucket with any downloader if
directory_path is prefixed with gs://
- Upload predictions directly from a GCS bucket with any submitter if
directory_path is prefixed with gs://
Possible bottlenecks:
- For
NumeraiClassicDownloader this may require adding a feature to numerapi before we can get this working in NumerBlox.
Context:
Numerblox supports Google Cloud Buckets (GCS) in downloaders and submitters with
download_file_from_gcs,upload_file_to_gcs,download_directory_from_gcsandupload_directory_to_gcs. However, it always requires the user to have the Numerai data saved locally first.Desired behavior:
directory_pathis prefixed withgs://directory_pathis prefixed withgs://Possible bottlenecks:
NumeraiClassicDownloaderthis may require adding a feature tonumerapibefore we can get this working in NumerBlox.