Skip to content

fix(multimodal): bound remote image downloads by the file size limit - #3906

Open
SEVEN-us wants to merge 1 commit into
Tencent:mainfrom
SEVEN-us:fix/remote-image-download-size-20260930
Open

SEVEN-us wants to merge 1 commit into
Tencent:mainfrom
SEVEN-us:fix/remote-image-download-size-20260930

Conversation

@SEVEN-us

Copy link
Copy Markdown

Description

downloadImageFromURL() calls DownloadBytes(), which currently uses an unbounded io.ReadAll. A remote image response can therefore consume memory beyond the configured file size limit, even when it has no Content-Length or understates its size.

Reuse GetMaxFileSize() (MAX_FILE_SIZE_MB, default 50 MiB). Reject oversized Content-Length before reading, then read at most the limit plus one byte to detect oversized streams. Return an explicit error without partial data and close the response body on every path.

Add nine cases covering small/empty bodies, exact-limit bodies with and without Content-Length, oversized declared and streamed bodies, understated lengths, HTTP failures, read failures, and body closure. The three oversized cases fail on the original implementation. The shared SSRF client and validation remain in the download path.

Type of Change

  • Bug fix
  • Test

Related Issue

None; found while reviewing the current remote-image download path.

Testing

Tested on Windows with Go 1.26.2. The following commands use the actual source files and existing SSRF tests, without replacement implementations or stubs for the SSRF policy:

go test internal/utils/httputil.go internal/utils/filesize.go internal/utils/security.go internal/utils/ssrf_outbound_cache.go internal/utils/shell_assignment.go internal/utils/httputil_test.go internal/utils/filesize_test.go internal/utils/security_test.go internal/utils/security_transport_test.go internal/utils/security_redirect_test.go internal/utils/security_whitelist_only_test.go internal/utils/ssrf_outbound_cache_test.go internal/utils/shell_assignment_test.go -count=1
go vet internal/utils/httputil.go internal/utils/filesize.go internal/utils/security.go internal/utils/ssrf_outbound_cache.go internal/utils/shell_assignment.go internal/utils/httputil_test.go
gofmt -l internal/utils/httputil.go internal/utils/httputil_test.go
git diff --check

All above checks pass; gofmt reports no files. The new download test uses an in-memory HTTP transport and a counting response body, so it makes no network requests and checks that reads stop at the limit plus one byte.

Package-level validation was attempted with go test ./internal/utils -run '^TestDownloadBytesSizeLimit$' -count=1. It cannot compile the unrelated SQL validator on this machine because CGO_ENABLED=0 and no C compiler is available: internal/utils/inject.go reports undefined pg_query.Parse / pg_query.Deparse. The initial run also reports a Go temporary-file cleanup permission error; source-scoped checks pass with GOTMPDIR inside the workspace. Full-repository checks and golangci-lint were not run; golangci-lint is not installed locally.

Checklist

  • Working-tree git diff --check passes
  • Changed Go source files are formatted
  • Download and related SSRF source-scoped tests pass
  • Self-reviewed the code
  • Added tests covering the change
  • Documented the configured limit in the DownloadBytes comment
  • Package-level and full-repository checks pass (local CGO blocker described above)
  • Diff-scoped golangci-lint passes (not available locally)

Remote images exceeding the existing configured file limit now return an error. There are no API schema changes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant