Add a better Cody client server-sent configuration mechanism

[emidoots]

Today, Cody clients derive some configuration from the Sourcegraph instance - namely:

• Per-user access controls for:
  • Cody access in general
  • Chat feature access
  • Autocomplete feature access
  • Commands feature access
  • Attribution ('guardrails') feature access
• (feature-flag) Whether the 'Smart Context Window' feature should be enabled, and if the server has APIs to support using it.
• (soon) whether the server supports a new HTTP API to query which models are available.

This server-sent configuration historically has been done rather ad-hoc. For example, we have to make a seperate GraphQL request to query if the server supports 'smart context window' because it is a new GraphQL field and old servers don't support querying that field, so the GraphQL request can fail. Similarly, we have a GraphQL 'cody features' resolver which gives admins control over cody features.. but adding new fields to either of these means we must deal with supporting older sourcegraph instances which do not have the new GraphQL field - a major pain point with GraphQL.

To alleviate this pain, and to make adding and removing these types of server-sent Cody client configurations easier, this PR introduces a new REST API on the backend which can contain all this configuration going forward. Here is how it looks:

curl -H 'Authorization: token sgp_redacted' -X GET https://sourcegraph.test:3443/.api/client-config
{
    "v1": {
        "CodyEnabled": true,
        "ChatEnabled": true,
        "AutoCompleteEnabled": true,
        "CommandsEnabled": true,
        "AttributionEnabled": false,
        "SmartContextWindowEnabled": true,
        "ModelsAPIEnabled": false
    }
}

Because this is not a GraphQL resolver, we are more free to change the shape of the JSON response; and there is a defined process for adding/removing fields over time, without needing hacks to workaround the issues GraphQL presents for this use case.

The response is versioned ("v1"), to enable backwards-incompatible changes with migrations over time in a very simple way: clients will look for whatever versions they support in the response, and if they do not find it they will simply produce an error during authentication like Cody cannot connect to this server (client or server version too old.)

In most cases, one should just add a new field to the current version "v1". However, over time that object will certainly grow to have fields we would like to remove, or there will be breaking changes we'd like to make - and when that time comes we can introduce a new version field "v2". For a period of time, the server may serve both versions to support old & new clients, and likewise new clients may support both versions to support old & new server versions. Eventually, though, "v1" would go away entirely and "v2" would become the only field.

The process for adding new fields and/or making breaking changes is nicely documented in internal/clientconfig/types.go so that anyone can follow it easily in the future.

Implementation

I generously borrowed from @chrsmith's awesome code structure in https://github.com/sourcegraph/sourcegraph/pull/63507 because I liked how he set that up:

• cmd/frontend/internal/clientconfig contains the frontend httpapi handlers, and a singleton service which implements 'getting the configuration' for an authenticated actor/user.
• internal/clientconfig contains the data types for client configuration.

(@chrsmith you might find some of how I set up tests for this useful, check out httpapi_test.go)

Usage in practice

This PR just adds the backend API shown above. However, since we don't have anything like this today, we need some way to let clients actually detect if the server supports this new API or not. To do this, I have added a new GraphQL field effectively mirroring how we detect smartContextWindow support today:

{
  site {
    codyLLMConfiguration {
      smartContextWindow
      disableClientConfigAPI
    }
  }
}

If disableClientConfigAPI is a valid GraphQL field, then /.api/client-config is also supported.

The boolean value of disableClientConfigAPI also acts as an escape hatch, temporarily allowing site admins to fallback to the old GraphQL-based configuration behavior by setting "disableClientConfigAPI": true, in the site config, should there be any issues.

Although the new client config API is defaulted to enabled, it is not yet in use in clients like VSCode. That will come in a subsequent PR.

Test plan

• Added HTTP handler tests
• Manual curl request shown above
• Ran GraphQL query to confirm disableClientConfigAPI field is working as expected

[abeatrix]

Didn't test it locally since the client-side change is not available yet, but the code makes sense to me. Adding @jamesmcnamara since he will be working on integrating this on the client-side 😄

[abeatrix]

This is unrelated to this PR, but what happens if Chat is disabled but Commands are enabled (since most commands are basically Chat)?

[emidoots]

I had to track this down a bit.

It looks like 'commands being disabled' actually means 'unsafe commands' specifically, e.g. https://sourcegraph.com/github.com/sourcegraph/cody/-/blob/vscode/src/main.ts?L345-348 - and I am pretty sure from recollection around the time this was implemented that the intent was only to allow admins to enable/disable CUSTOM cody commands - not all commands in general. I.e. user-defined cody commands.

I've renamed the flag to CustomCommandsEnabled to clarify this

[emidoots]

I'll be doing the client work to integrate this (I think anyway!)

[chrsmith]

These changes generally look good. Just three main suggestions:

• What do you think about removing the Service interface layer. Can we just make this a single function (that requires the database and logger as parameters)?
• The GraphQL-facing changes are extremely painful to change or (even remove) because GraphQL will fail if you query fields that aren't present in the schema. So it would be great to have the "final" field name we want to begin with.
• REST API versioning, and tradeoffs. It would be nice to introduce a "REST API versioning" approach at the frontend/internal/httpapi layer. Since that's something we'd want to build on top of rather than re-invent N-different times for N-different APIs. (e.g. there is another approach used by the Guardrails feature using URL query parameters.)

[chrsmith]

This seems a little too clunky, as being opt-out seems a little confusing. e.g. "do the thing IFF !disableClientConfigAPI?"

Also, clientConfigAPI is describing this in terms of the implementation detail, and not the user-facing functionality.

What do you think about renaming this to CodyLLMConfiguration.supportsServerDefinedModels or something? Does that make the intent clearer?

Also, because GraphQL is lame, we wouldn't be able to safely "remove the field in the future". So we unfortunately need to "get the name right the first time".

[emidoots]

outcome of our call/conversations:

Context: This feature flag does not apply to models in general, it only applies to the new config API which works regardless of whether you are using the new models API or not. The ClientConfig.ModelsAPIEnabled field indicates if the new models API is ready for use.

This being a !disableClientConfigAPI is okay/fine, because this GraphQL field is really just a temporary measure to get us to use this new API sooner. In the future, we will refactor the Cody client to make a request to this new /client-config API as its first-order of business, and then only if that fails will it fallback to the old GraphQL-based APIs.

Longer-term, we will delete these duplicate GraphQL fields entirely.

[chrsmith]

Rather than have this be "opt-out", we should instead have this be "opt-in". And later reverse the polarity to be an "opt-out" gesture. Since otherwise, we would automatically enable this new codepath for ~everyone sooner than we'd like. Right?

[emidoots]

outcome of our call/conversations:

Yes we want this to be on-by-default-for-everyone immediately. This feature flag does not apply to models in general, it only applies to the new config API which works regardless of whether you are using the new models API or not. The ClientConfig.ModelsAPIEnabled field indicates if the new models API is ready for use.

[chrsmith]

Versioning the HTTP response here is important, but of the ways to implement REST API versioning thing is one of the least effective ones. Instead, I'd suggest that we only consider:

• Including the version in the URL route, e.g. /clientconfig/v1/foo
• Including the version as a URL query parameter, e.g. /clientconfig/foo?version=2
• (My personal preference) Relying on the Accept header, e.g. curl -H "Accept: vnd.sourcegraph.v2+json"

I'd suggest we instead use the Accept header and just send the JSON response body that the client is asking for. That way the client doesn't need to sniff or inspect some potentially unknown response body and/or fallback to some lowest common denominator.

Then in the implementation, we ascribe meaning to whatever the version header means. For example:

type restAPIVersion int
const (
     initialRelease restAPIVersion = 0
     updateFooTypes restAPIVersion = 1
)

// API version to use if no `Accept` header is included.
const defaultVersion = initialRelease

...

func handlerFn(w http.ResponseWriter, r *http.Request) {
   apiVersion := getAPIVersionFromRequest(r)

   // On 2024-07-02 we introduced a breaking change to return a different
   // type of response for this endpoint.
   if apiVersion >= updateFooTypes {
        http.Error(w, `{ "foo": [ 1, 2, 3 ] }`, http.StatusOK)
   } else {
       // Older versions just returned a single value.
        http.Error(w, `{ "foo": 1 }`, http.StatusOK)
   }
}

We would still need to be extremely careful about making API changes that are backwards compatible. But using this accept header approach, we have a clean way to make a breaking change should the need arise. (Which should be _extremely rare, if ever.)

But even then, this still doesn't account for "new clients" and "extremely old backends", e.g. the client is looking for v10 but the Sourcegraph instance hasn't been updated since 1985, and only supports v2. (And to make that gracefully work requires even more work and returning the "effective API version" in the response headers, etc. etc.)

[emidoots]

Outcome of our discussions/calls around versioning:

There are 4 scenarios we care about.. and we will handle all 4 using an Accept header and Content-Type header:

Scenario: client is newer than server

• Client requests Accept: v1,v2,v3
• Server response: JSON blob v2 and header Content-Type: clientconfig/v2

Scenario: client is older than server

Client requests Accept: v1,v2
Server response: JSON blob v2 and header Content-Type: clientconfig/v2

Scenario: client REALLY older than server (server removed v1)

Client requests Accept: v1
Server response: HTTP 400 not supported status or whatever, no JSON blob

Scenario: server REALLY older than client (client removed v1, that’s all server has)

Client requests Accept: v3
Server response: HTTP 400 not supported status or whatever, no JSON blob

The exact format we will use for Accept and Content-Type strings is TBD. We also need to confirm those headers can be accessed/set in a browser fetch() request but it should work.

This PR will not implement this versioning scheme, we can implement it later once we need to break the API. This versioning scheme applies not only just to the client-fetch endpoint, but also to our other REST API endpoints (and possibly to GraphQL endpoints in some intersectional ways)

I will link this comment thread in the code for clarity sake.

[emidoots]

The revised curl response:

curl -H 'Authorization: token sgp_redacted' -X GET https://sourcegraph.test:3443/.api/client-config
{
    "codyEnabled": true,
    "chatEnabled": true,
    "autoCompleteEnabled": true,
    "customCommandsEnabled": true,
    "attributionEnabled": false,
    "smartContextWindowEnabled": true,
    "modelsAPIEnabled": false
}

[emidoots]

@chrsmith PTAL - I think I've addressed everything we went over

[chrsmith]

Thanks for recording the notes from our conversation.