Skip to content

[backend] Add a new file provider to reference external files#274

Open
lecafard wants to merge 1 commit intomasterfrom
feature/external-file-provider
Open

[backend] Add a new file provider to reference external files#274
lecafard wants to merge 1 commit intomasterfrom
feature/external-file-provider

Conversation

@lecafard
Copy link
Copy Markdown
Contributor

@lecafard lecafard commented Jan 14, 2026

This commit introduces a new "external" file provider, allowing challenges to reference files hosted on external URLs. This is useful for large files or files that are already hosted elsewhere.

To support this, the file service has been refactored:

  • The FileProviderInstance interface has been updated. The upload method now returns an object containing the ref, hash, and size of the file. This responsibility was previously handled by the FileService itself.
  • The local and s3 providers have been updated to implement the new interface, using a new summarizeFile utility to calculate the hash and size from the upload stream.
  • A new external provider is added. Instead of a file, it accepts a small JSON payload containing the url, hash, and size.
    • The external provider is instantiated directly and cannot be removed as we don't really want users to instantiate it.
  • The admin file upload endpoint now accepts a provider query parameter to select which provider to use.
  • There is a backwards incompatible change with the file hash since it's definitely easier if it's represented as a string.

see #273

@lecafard lecafard requested a review from josephsurin January 14, 2026 11:31
@lecafard lecafard force-pushed the feature/external-file-provider branch 2 times, most recently from 67f023a to ac4d5ab Compare January 16, 2026 11:39
@mcdcam
Copy link
Copy Markdown
Contributor

mcdcam commented Jan 17, 2026

I like this design. Uploading a JSON blob feels a bit hacky, but I haven't been able to think of an alternative that doesn't compromise somewhere else.

I think it'd be good to add some comments explaining that the uploaded "file" is just arbitrary data to be processed by the file provider.

@lecafard lecafard force-pushed the feature/external-file-provider branch from ac4d5ab to 2f13661 Compare January 23, 2026 10:21
@lecafard lecafard force-pushed the master branch 3 times, most recently from 6e7074d to c4a5262 Compare April 6, 2026 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants