feat(aws_tools): add S3 File Uploader and S3 File Download tools

[leoou331]

What this PR does

Adds two new builtin tools to the aws_tools plugin (tools/aws):

• s3_file_uploader — takes a file variable from an upstream workflow node (Start file input, LLM output, another tool, etc.) and uploads it to a configurable S3 bucket/key. Optionally returns a presigned GET URL.
• s3_file_download — takes an s3://bucket/key URI and emits the object as a Dify file (via create_blob_message) plus structured metadata for downstream nodes.

Why

The existing s3_operator is text-only:

• input is text_content: string
• output is UTF-8 decoded string

It cannot be wired to a Start node's file input, nor consume the file output of nodes like Frame Extractor / Nova Canvas. These two new tools close that gap and keep the same UX as the rest of the plugin (provider-level credentials, optional per-tool overrides, three-language labels).

A typical workflow now looks like:

[Start: file input] -> [s3_file_uploader] -> ... -> [s3_file_download] -> [LLM/Code/...]

Files

tools/aws/
├── manifest.yaml                          # version 0.0.26 -> 0.0.27
├── README.md                              # +2 lines (Features list)
├── provider/aws_tools.yaml                # +2 lines (register both tools)
└── tools/
    ├── s3_file_uploader.py                # 200 lines
    ├── s3_file_uploader.yaml              # 139 lines
    ├── s3_file_download.py                # 174 lines
    └── s3_file_download.yaml              # 78 lines

Both Python modules are self-contained — credential-resolution helpers (_resolve_aws_credentials, _build_boto3_client_kwargs, _reset_clients_on_credential_change) are inlined so this PR does not introduce a shared utils/ module that the rest of the plugin doesn't use.

Tool surface

s3_file_uploader (form parameters)

Param	Type	Required	Notes
input_file	file	yes	Bound to upstream file variable.
bucket_name	string	yes	Without s3://.
key_prefix	string	no	Folder-style prefix, e.g. workflow-outputs.
object_key	string	no	Override final key; defaults to incoming filename.
aws_region	string	no	Per-tool override of provider default.
aws_access_key_id / aws_secret_access_key / aws_session_token	string	no	Per-tool override / STS support.
generate_presign_url	boolean	no	Default false.
presign_expiry	number	no	Default 3600 seconds.

Outputs three messages: text = s3://bucket/key or presigned URL; json = {bucket_name, object_key, s3_uri, presigned_url?, presign_expiry?}; no files.

s3_file_download

Param	Type	Required	Notes
s3_uri	string (LLM-fillable)	yes	s3://bucket/key.
aws_region / aws_access_key_id / aws_secret_access_key / aws_session_token	string	no	Same overrides as uploader.

Outputs files = [<Dify file>], json = {bucket, key, content_type, content_length, etag, last_modified, s3_uri}, and a key: value text block of the same metadata.

Validation

Static:

• python -m py_compile on both .py files — ✅
• yaml.safe_load on the new + modified yaml files — ✅
• Verified extra.python.source paths resolve to existing files — ✅
• black --check -l 100 and ruff check both clean on the new files — ✅
• Confirmed label / description languages match the rest of the plugin (en_US / zh_Hans / pt_BR) — ✅
• No new dependencies (boto3/botocore already pinned in pyproject.toml) — ✅

End-to-end (real run, not dry validation):

1. Built a .difypkg from tools/aws/ on this branch.
2. Installed it on a self-hosted Dify 1.14.2 Community Edition.
3. Imported a workflow [Start (file input) -> s3_file_uploader -> s3_file_download -> End], pointed at S3 bucket in cn-northwest-1.
4. Triggered the workflow via the Service API with text/plain (hello.txt, 244 B) and image/png (50×50 RGBA, 144 B).
5. Result for both: status = succeeded, total_steps = 4, elapsed ~0.4-0.8s.
6. Pulled both objects back from S3 with aws s3 cp and compared SHA-256 — byte-for-byte identical with the local source.

A companion run on the equivalent aws-samples/dify-aws-tool#168 PR (same code) additionally covered:

• PDF binary payload — succeeded, SHA-256 identical
• generate_presign_url=true — URL contains correct SigV4 fields, custom X-Amz-Expires honored, curl fetch returned HTTP 200 with byte-identical content
• STS aws_session_token — temporary credentials path round-tripped successfully

Out of scope

• Multipart upload / streaming for large files
• Any change to existing s3_operator behavior
• Shared utils/ module — left for a follow-up if more tools want the helpers

[gemini-code-assist]

Code Review

This pull request introduces two new AWS S3 tools: AWS S3 File Uploader and AWS S3 File Download, enabling workflows to upload files to S3 (with optional presigned URLs) and download S3 objects as Dify file variables. The review feedback identifies a critical thread-safety issue where caching the s3_client as an instance attribute could lead to race conditions or credential leakage across concurrent executions. It is recommended to initialize the client locally within the _invoke method instead. Additionally, the feedback suggests improving error handling for S3 bucket/key exceptions and safely parsing the presign_expiry parameter to prevent runtime crashes.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[leoou331]

Thanks for the review @gemini-code-assist[bot] — all four points accepted and applied in fafc4c4:

#	Point	Resolution
1	self.s3_client cache is not thread-safe / can leak credentials	Removed. s3_client is now a local variable inside _invoke.
2	Drop the now-redundant _reset_clients_on_credential_change / _credential_signature helpers	Done; also removed the Iterable import.
3	Use standard ClientError error-code matching in s3_file_download	Done: exc.response["Error"]["Code"] → NoSuchBucket / NoSuchKey.
4	Robust filename when the S3 key has a trailing slash	Done: key.rstrip("/").split("/")[-1] + empty-string guard.
5	Safe presign_expiry parsing	Extracted _parse_presign_expiry() that tolerates None / "" / non-numeric input, falls back to 3600.

Re-validation

black --check -l 100 and ruff check are both clean on the new files.

End-to-end re-run on a fresh self-hosted Dify 1.14.2 with this branch packaged into a .difypkg:

Path	Result
text/plain round-trip	✅ succeeded, SHA-256 identical
image/png round-trip	✅ succeeded, SHA-256 identical
application/pdf round-trip	✅ succeeded, SHA-256 identical
generate_presign_url=true + curl fetch	✅ HTTP 200, SHA-256 identical
STS aws_session_token (via aws sts get-session-token)	✅ succeeded, SHA-256 identical

Plus a small unit-style check for _parse_presign_expiry: None, "", 600, "600", "not a number", 3.14, custom default — all 7 cases produce the expected fall-back behaviour.

The companion AWS samples PR (aws-samples/dify-aws-tool#168) has been updated with the same fixes.

[gemini-code-assist]

That is excellent news, @leoou331. Thank you for the comprehensive re-validation and for applying the requested improvements so thoroughly. The extracted _parse_presign_expiry helper and the robust filename handling look great. This PR is in excellent shape.

[leoou331]

Follow-up: batch counterparts

Pushed commit 00c03330 adding s3_files_uploader (input_files: files) and s3_files_download (s3_uris: array) so a single workflow node can process N files at once — Iteration-node wrapping around the single-file tools was the only previous option.

Tool surface (new tools only)

Tool	Required input	Output
s3_files_uploader	input_files: files + bucket_name	json = {count, ok, failed, results: [{object_key, s3_uri, presigned_url?, status, error?}, ...]} + per-line text summary
s3_files_download	s3_uris: array of s3://...	json = {count, ok, failed, results: [...]} + per-line text summary + one Dify file blob per successful URI in input order

Design choices

• No object_key override on the batch uploader (a single override cannot apply to N files). Final key per file is {key_prefix}/{filename}; the uploader auto-disambiguates duplicate filenames in the same batch (image.png / image-1.png / image-2.png) so concurrent upstream branches with identical filenames don't silently overwrite each other.
• Per-entry failure isolation: a single bad file/URI does not abort the batch — status = ok | failed (+ error) is captured per entry in results[]. The whole invocation only emits a top-level error message when every entry fails.
• Same inline credential helpers as the single-file tools (no shared utils/ module introduced).

Validation

• python -m py_compile, yaml.safe_load, black --check -l 100, ruff check — all clean.
• 10 mock-boto3 unit tests covering basic batch, dedup, partial failure (ClientError), all-fail top-level error, empty input, presign error isolation, partial download (NoSuchKey), invalid URI: 10/10 pass.
• End-to-end on Dify 1.14.2 Community Edition + real S3 (cn-northwest-1):
  • Workflow [Start file-list → s3_files_uploader → extract URIs → s3_files_download → summarize → End], 3 files (txt 40 B + png 220 B + pdf 540 B), elapsed ~1.2 s, all 6 steps succeeded.
  • SHA-256 round-trip byte-identical for all 3 files (pulled back via aws s3 cp and compared with the source).
    • a.txt c377b72e7343c1642a35c7ff5108fef6c14fc5fba3aecce89c18d9ac526e4de8
    • img.png 5a2fe18dbec51b2426f8aa31f6424b0efff246497646f1aa2314abe8d09b7aec
    • doc.pdf 7b6fed1b75159c5cbc633e04f9011a1a9e4f22efce2621b8e14646064cf8c6fa
  • Each presigned URL returned by the uploader fetched via curl and verified byte-identical to the source.
  • Partial-failure run (2 valid + 1 bogus s3:// URI): downloader returned count=3, ok=2, failed=1 with a NoSuchKey error string for the bogus URI and exactly 2 file blobs yielded in input order.

Files

tools/aws/
├── manifest.yaml                          # 0.0.27 → 0.0.28
├── README.md                              # +2 Features lines
├── provider/aws_tools.yaml                # +2 lines (register both new tools)
└── tools/
    ├── s3_files_uploader.py               # 251 lines
    ├── s3_files_uploader.yaml             # 134 lines
    ├── s3_files_download.py               # 209 lines
    └── s3_files_download.yaml             # 80 lines

Out of scope (kept for a follow-up)

• Multipart upload / streaming for large files
• Any change to existing s3_operator / s3_file_uploader / s3_file_download
• Shared utils/ module