Context: more precise chunk sizing

[jtibshirani]

Currently, when retrieving context chunks, we hardcode the number of lines to
20. Historically, we've limited chunks to 1024 characters, and we chose 20
lines to roughly mirror that.

In evals, I found that we're often returning fewer than 1024 characters. This
PR updates the context resolver to load an adaptive number of lines based on
the 1024 character limit.

Addresses #61745

Test plan

Added new test. Also manually tested using GraphQL console.

[jtibshirani]

This shows an improvement on CodeSearchNet:
Before

Recall (files)	91/99
Recall (chunks)	70/99
Average chunk overlap	0.81

After

Recall (files)	91/99
Recall (chunks)	74/99
Average chunk overlap	0.89

In the near future, we should be able to increase the chunk size even more, given our increased context window limits. The current choice of 1024 is somewhat arbitrary.

[jtibshirani]

Calling attention to this -- I think an ideal context API would let callers pass in a "token budget" that the backend must respect in its responses. The backend would be allowed to optimize within that budget in whatever way it wants. (Or maybe this would be "character budget" to keep it simple?)

[keegancsmith]

you can get very long lines. Should we put in an upperbound just to ensure counting up doesn't get too big?

[jtibshirani]

hmm, how about I switch to rounding down. That way we stay within the "token budget" which seems desirable.