Skip to content

Feature Request: server/webui: allow configuring server props (default webui settings) from CLI args #17940

@woof-dog

Description

@woof-dog

Prerequisites

  • I am running the latest code. Mention the version if possible as well.
  • I carefully followed the README.md.
  • I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
  • I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

Currently, there isn't a way for a llama-server operator to configure some default webui settings.

I would like to request a command-line argument for setting certain server props from a JSON file or environment variable, verifying that some client-side webui props, such as pasteLongTextToFileLen, can be configured this way. This will allow llama-server hosts to configure default webui settings which are not currently configurable server-side.

(@allozaur suggested I create this as a feature request and also tag @ServeurpersoCom)

Motivation

There have been clashes, such as in #14041 (also see #14042 and #14468), with decisions about default llama.cpp client-side webui settings which are currently hardcoded and not adjustable by the llama-server host without modifying source code.

Ultimately this feature gives more control back to the user to customize the server's deployment.

Possible Implementation

  1. Verify that default client-side webui props (namely, pasteLongTextToFileLen) actually can be set by server props. If not, we may need to add a new key to the server props and update the webui to accommodate this.
  2. On llama-server launch, read a JSON string either from a file passed via CLI or from an environment variable (either or both is acceptable in my opinion) - none if neither is passed in - and update the server props to reflect these changes.
  3. (If needed) modify the webui to read certain settings from a new key.

The code #14468 has portions of this implemented (for inspiration) and #16515 modified the webui to use server-driven props as defaults.

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions