-
Notifications
You must be signed in to change notification settings - Fork 14.1k
Description
Prerequisites
- I am running the latest code. Mention the version if possible as well.
- I carefully followed the README.md.
- I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
- I reviewed the Discussions, and have a new and useful enhancement to share.
Feature Description
Currently, there isn't a way for a llama-server operator to configure some default webui settings.
I would like to request a command-line argument for setting certain server props from a JSON file or environment variable, verifying that some client-side webui props, such as pasteLongTextToFileLen, can be configured this way. This will allow llama-server hosts to configure default webui settings which are not currently configurable server-side.
(@allozaur suggested I create this as a feature request and also tag @ServeurpersoCom)
Motivation
There have been clashes, such as in #14041 (also see #14042 and #14468), with decisions about default llama.cpp client-side webui settings which are currently hardcoded and not adjustable by the llama-server host without modifying source code.
Ultimately this feature gives more control back to the user to customize the server's deployment.
Possible Implementation
- Verify that default client-side webui props (namely,
pasteLongTextToFileLen) actually can be set by server props. If not, we may need to add a new key to the server props and update the webui to accommodate this. - On llama-server launch, read a JSON string either from a file passed via CLI or from an environment variable (either or both is acceptable in my opinion) - none if neither is passed in - and update the server props to reflect these changes.
- (If needed) modify the webui to read certain settings from a new key.
The code #14468 has portions of this implemented (for inspiration) and #16515 modified the webui to use server-driven props as defaults.