Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

Add Cody providers to pings#58848

Merged
taras-yemets merged 16 commits intomainfrom
ty/cody-providers-pings
Dec 12, 2023
Merged

Add Cody providers to pings#58848
taras-yemets merged 16 commits intomainfrom
ty/cody-providers-pings

Conversation

@taras-yemets
Copy link
Contributor

@taras-yemets taras-yemets commented Dec 8, 2023

Adds Cody completions and embeddings providers data to pings.
Provider name is always included. Model name is included only for "sourcegraph" provider.

Closes https://github.com/sourcegraph/sourcegraph/issues/58826

BigQuery pings schema update

Site config Value in pings Note
{ "cody.enabled": false } "codyProviders": { "Embeddings": null, "Completions": null } If Cody is disabled, completions and embeddings are not used. Return empty values similar to GetCompletionsConfig and GetEmbeddingsConfig
{ "cody.enabled": true, "completions": { "chatModel": "claude-v1", "completionModel": "claude-instant-v1", "fastChatModel": "claude-instant-1" "provider": "anthropic", "accessToken": "REDACTED" } } "codyProviders": { "Embeddings": { "Model": "openai/text-embedding-ada-002", "Provider": "sourcegraph" }, "Completions": { "Provider": "anthropic", "ChatModel": "", "CompletionModel": "", "FastChatModel": "" } } Cody is enabled, completions config is defined, embeddings - is not. Return only provider name for completions provider (don't return model names as the provider is not "sourcegraph"). Return default embeddings provider.
{ "cody.enabled": true, "embeddings": { "provider": "sourcegraph", "accessToken": "REDACTED" } } "codyProviders": { "Embeddings": { "Model": "openai/text-embedding-ada-002", "Provider": "sourcegraph" }, "Completions": { "Provider": "sourcegraph", "ChatModel": "anthropic/claude-2.0", "CompletionModel": "anthropic/claude-instant-1", "FastChatModel": "anthropic/claude-instant-1" } } Cody is enabled, embeddings config is defined, completions - is not. Return the provider and model name for the completions provider (note, the model name is not provided in the site config - the default one is used). Return default completions provider (with model names as the provider is "sourcegraph").

Test plan

  • CI passes
  • Tested manually: most recent ping contains correct Cody providers data (see values in the table above)

Preview 🤩

Preview Link

@cla-bot cla-bot bot added the cla-signed label Dec 8, 2023
@taras-yemets taras-yemets marked this pull request as ready for review December 8, 2023 13:09
@taras-yemets taras-yemets requested review from a team and ryphil December 8, 2023 13:09
@sourcegraph-bot
Copy link
Contributor

sourcegraph-bot commented Dec 8, 2023

Codenotify: Notifying subscribers in CODENOTIFY files for diff 23634d9...0faaf3e.

Notify File(s)
@unknwon internal/updatecheck/client.go
internal/updatecheck/handler.go
internal/updatecheck/handler_test.go

@taras-yemets taras-yemets requested a review from unknwon December 8, 2023 13:11
Copy link
Contributor

@chwarwick chwarwick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See note about model's otherwise I think once you get the test passing it 👍 . Remember to wait for the data pr before merging. Also pings used to require an approved RFC.

@taras-yemets taras-yemets force-pushed the ty/cody-providers-pings branch from de89d4f to b02484b Compare December 8, 2023 14:19
@taras-yemets
Copy link
Contributor Author

taras-yemets commented Dec 8, 2023

Thanks for your review, @chwarwick! Very helpful comments!
UPD: created an RFC

c := conf.SiteConfig()
providers := types.CodyProviders{
Completions: &types.CodyCompletionProvider{
Provider: c.Completions.Provider,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just realized Completions & Embeddings are optional configurations. I think this could or will crash for instances with Cody disabled or ones with the most basic cody.enabled:true setup.

Additionally you should probably use the GetCompletionsConfig and corresponding embeddings one, because it fills in all the defaults.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @chwarwick! Updated code and PR description.

@taras-yemets taras-yemets changed the title Add Cody providers to pings [Backport 5.2] Add Cody providers to pings Dec 12, 2023
Copy link
Member

@dadlerj dadlerj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments in the thread about the future of collecting this via pings: https://sourcegraph.slack.com/archives/CN4FC7XT4/p1702399336866429?thread_ts=1702377291.476959&cid=CN4FC7XT4

For now, approved!

Co-authored-by: Dan Adler <dadlerj@users.noreply.github.com>
@taras-yemets taras-yemets changed the title [Backport 5.2] Add Cody providers to pings Add Cody providers to pings Dec 12, 2023
@taras-yemets taras-yemets merged commit 2ffd9d6 into main Dec 12, 2023
@taras-yemets taras-yemets deleted the ty/cody-providers-pings branch December 12, 2023 17:46
sourcegraph-release-bot pushed a commit that referenced this pull request Dec 12, 2023
vovakulikov pushed a commit that referenced this pull request Dec 12, 2023
jdpleiness added a commit that referenced this pull request Dec 13, 2023
* Add Cody providers to pings (#58848)

(cherry picked from commit 2ffd9d6)

* ci test

---------

Co-authored-by: Taras Yemets <yemets.taras@gmail.com>
Co-authored-by: Jacob Pleiness <jdpleiness@users.noreply.github.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add pings to report on LLM usage across enterprise customers

4 participants