[Enhancement]: Read model context size from the provider instead of using predefined sizes #17713

What features would you like to see added?

LibreChat should discover the model's actual context window from the provider at runtime (e.g. via llama.cpp's meta.n_ctx on GET /v1/models, or GET /props) and use that value for context/token budgeting, instead of relying on a predefined model - context map.

More details

LibreChat is reported to use predefined context sizes for models (an internal model/token map). For self-hosted providers such as llama.cpp, the actually-served context may be set smaller than the model's trained maximum (e.g. launched with -c 32768 / a smaller --ctx-size).

llama.cpp already exposes the real, runtime context through its OpenAI-compatible API:

GET /v1/models -> meta.n_ctx (the context the server is currently serving) and meta.n_ctx_train (the trained/architecture maximum).
GET /props -> also reports the live context.

Because LibreChat uses the predefined (trained) size, it believes a model has more context than it actually does. This can produce over-long prompts that the server then rejects - e.g. llama.cpp returns 400: request exceeds available context size (N tokens), try increasing it.

Proposed behavior

For OpenAI-compatible providers (and specifically llama.cpp-compatible servers), query the provider for the model's context size (on model load / at startup) and use the reported runtime value (meta.n_ctx) for budgeting.
Prefer the runtime context over the training maximum (n_ctx over n_ctx_train).
Fall back to the existing predefined size only when the provider does not expose a context value.

Suggested resolution priority
tokenConfig.context (explicit user override) > provider-reported runtime context (meta.n_ctx) > predefined model map.

The per-model context override in tokenConfig is a working manual workaround today — I'd like to keep it as the highest-priority explicit override, but have the runtime value auto-detected so self-hosters don't have to pin every model by hand.

Which components are impacted by your request?

General

Pictures

No response

Code of Conduct

• I agree to follow this project's Code of Conduct