Skip to content

Add timeout for remote model list loading #6551

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
vicksiyi wants to merge 2 commits into
base: main
Choose a base branch
from
+84 −6
Open

Add timeout for remote model list loading #6551

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e5e76bb
Add timeout for remote model list loading
vicksiyi Jun 23, 2026
c992188
Cache remote model list loading
vicksiyi Jun 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions packages/components/src/modelLoader.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
import axios from 'axios'
import { getModelConfigByModelName, MODEL_TYPE } from './modelLoader'

jest.mock('axios')

const mockedAxios = axios as jest.Mocked<typeof axios>

describe('modelLoader', () => {
const originalModelListConfigJson = process.env.MODEL_LIST_CONFIG_JSON

afterEach(() => {
jest.resetAllMocks()
if (originalModelListConfigJson === undefined) {
delete process.env.MODEL_LIST_CONFIG_JSON
} else {
process.env.MODEL_LIST_CONFIG_JSON = originalModelListConfigJson
}
})

it('uses a bounded timeout when loading remote model config before falling back locally', async () => {
process.env.MODEL_LIST_CONFIG_JSON = 'https://example.com/models.json'
mockedAxios.get.mockRejectedValueOnce(new Error('timeout'))

const modelConfig = await getModelConfigByModelName(MODEL_TYPE.CHAT, 'awsChatBedrock', 'ai21.jamba-1-5-large-v1:0')

expect(mockedAxios.get).toHaveBeenCalledWith('https://example.com/models.json', { timeout: 5000 })
expect(modelConfig?.name).toBe('ai21.jamba-1-5-large-v1:0')
})
})
4 changes: 3 additions & 1 deletion packages/components/src/modelLoader.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ import * as fs from 'fs'
import * as path from 'path'
import { INodeOptionsValue } from './Interface'

const MODEL_LIST_FETCH_TIMEOUT_MS = 5000

@gemini-code-assist gemini-code-assist Bot Jun 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

It is recommended to make the fetch timeout configurable via an environment variable. This allows users in restricted network environments, slow proxies, or air-gapped systems to adjust the timeout value as needed.

const MODEL_LIST_FETCH_TIMEOUT_MS = process.env.MODEL_LIST_FETCH_TIMEOUT_MS
    ? parseInt(process.env.MODEL_LIST_FETCH_TIMEOUT_MS, 10) || 5000
    : 5000

@vicksiyi vicksiyi Jun 23, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in c992188. The loader now caches the loaded model file per model-list source and supports MODEL_LIST_FETCH_TIMEOUT_MS with a default fallback.

Validation: git diff --check and ./node_modules/.bin/jest src/modelLoader.test.ts --runInBand from packages/components.


export enum MODEL_TYPE {
CHAT = 'chat',
LLM = 'llm',
Expand Down Expand Up @@ -38,7 +40,7 @@ const getRawModelFile = async () => {
process.env.MODEL_LIST_CONFIG_JSON ?? 'https://raw.githubusercontent.com/FlowiseAI/Flowise/main/packages/components/models.json'
try {
if (isValidUrl(modelFile)) {
const resp = await axios.get(modelFile)
const resp = await axios.get(modelFile, { timeout: MODEL_LIST_FETCH_TIMEOUT_MS })

@gemini-code-assist gemini-code-assist Bot Jun 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Performance & UX Issue: Repeated Network Requests and Timeouts

Currently, getRawModelFile is called on every single model configuration lookup (e.g., in getModelConfig, getModelConfigByModelName, getModels, and getRegions).

With the new 5-second timeout:

  1. When Online: Every single lookup will trigger a redundant HTTP request to GitHub, causing unnecessary latency and potentially hitting GitHub's rate limits.
  2. When Offline / Slow Connection: Every single lookup will block for 5 seconds before falling back to the local file, leading to a severely degraded user experience and laggy UI.

Recommendation

Implement an in-memory cache for the loaded models so that the remote file (or local fallback) is only fetched/read once per application lifecycle.

@vicksiyi vicksiyi Jun 23, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in c992188. The loader now caches the loaded model file per model-list source and supports MODEL_LIST_FETCH_TIMEOUT_MS with a default fallback.

Validation: git diff --check and ./node_modules/.bin/jest src/modelLoader.test.ts --runInBand from packages/components.

if (resp.status === 200 && resp.data) {
return resp.data
} else {
Expand Down