codeintel: Make indexer tag/sources configurable in inference

[efritz]

This PR adds a way to configure (via site-config) the set of indexers that the inference logic uses without having to override the entire script. Fixes #42693. Fixes #33369.

Test plan

Tested on a local instance.

[efritz]

cc @varungandhi-src and @olafurpg I'll update the tags according to the response in this thread before this gets merged.

[sourcegraph-bot]

Codenotify: Notifying subscribers in CODENOTIFY files for diff 0e022bf...8455773.

Notify

File(s)

@Strum355

internal/codeintel/autoindexing/internal/inference/lang_clang_test.go internal/codeintel/autoindexing/internal/inference/lang_go_test.go internal/codeintel/autoindexing/internal/inference/lang_java_test.go internal/codeintel/autoindexing/internal/inference/lang_python_test.go internal/codeintel/autoindexing/internal/inference/lang_rust_test.go internal/codeintel/autoindexing/internal/inference/lang_typescript_test.go internal/codeintel/autoindexing/internal/inference/libs.go internal/codeintel/autoindexing/internal/inference/libs/indexes.go internal/codeintel/autoindexing/internal/inference/libs/update-shas.sh internal/codeintel/autoindexing/internal/inference/lua/clang.lua internal/codeintel/autoindexing/internal/inference/lua/go.lua internal/codeintel/autoindexing/internal/inference/lua/java.lua internal/codeintel/autoindexing/internal/inference/lua/python.lua internal/codeintel/autoindexing/internal/inference/lua/rust.lua internal/codeintel/autoindexing/internal/inference/lua/typescript.lua internal/codeintel/autoindexing/internal/inference/service.go

[varungandhi-src]

This doesn't fix https://github.com/sourcegraph/sourcegraph/issues/33369 because versions aren't pinned (there is no specific SHA being specified).

[efritz]

@varungandhi-src That's addressed in https://github.com/sourcegraph/sourcegraph/pull/43199#issuecomment-1284593342, and I asked for the SHAs in https://github.com/sourcegraph/sourcegraph/issues/33369#issuecomment-1284536245 earlier today.

[varungandhi-src]

Oh sorry, saw those out-of-order. 😅 Yeah, let me add those tags.

[varungandhi-src]

I think using the latest ones should be OK.

lsif-clang 5ef2334ac9d58f1f947651812aa8d8ba0ed584913f2429cc9952cb25f94976d8
scip-java eb3996bdc8ab3a56600e7d647bc1ef72f3db8cfffc2026550095a0af7bb762bd
lsif-go 253c991fdd8b118afadcfbe6f7a6d03ca91c44fd2860dbe8a9fd69c93c6025f6
lsif-rust 83cb769788987eb52f21a18b62d51ebb67c9436e1b0d2e99904c70fef424f9d1
scip-python 5049c4598d03af542bde5e1254a17fa6d1eb794c1bdd14d0162fb39c604581b4
scip-typescript 4184c6a771037d854e23323db2c8a65947029c93401b696f3b5281fff5c3cbe9

I'm wondering why we have lsif-clang here though -- we don't do auto-indexing for it anyways, right?

[efritz]

I'm wondering why we have lsif-clang here though -- we don't do auto-indexing for it anyways, right?

We do provide hints about how auto indexing could look (powers the braindot menu) for a particular repo, but we don't actually provide any to perform. Logic is in the lua script, and it only implements hints.

[efritz]

@varungandhi-src Updated!

[eseliger]

just want to leave a big thanks here that this very recent customer feedback/pain has been addressed so swiftly :)

[eseliger]

docs please :)