fix(cody-gateway): Improve prompt and request validation for gemini

[abeatrix]

This PR aims to fix the issue where Cody Gateway is returning error regarding the unsupported Stream field showing up in the request:

Changes included:

• Moved the google-specific types from google.go to a new google_types.go file
• Reorganized the types to improve readability and maintainability
• Removed unused fields and methods from the types
• Aligned the types with the latest Google API documentation

Also fix an issue with google provider for byok customers about last assistant message being empty, which is the default format sent from clients (e.g. VS Code):

This PR addresses this issue by removing the last assistant message if it's empty during the prompt building step to make it more robust:

• Validate that the input messages are not empty
• Ensure the first message is a non-empty assistant message
• Skip empty assistant messages at the end of the prompt
• Disallow consistent speaker role between consecutive messages
• Add tests for the various validation cases

Test plan

Update Site Config to connect to google API directly in your local instance, and then log into your local instance in VS Code. Verify the responses are correct with no errors.

Changelog

[abeatrix]

All the types are moved to a separate file for clarity.

[abeatrix]

This was the cause of the issue where we originally has it as Stream bool json:"stream,omitempty"``

Now it'll not be marshaled into JSON to ensure the unsupported stream field will not be sent to Google

[chrsmith]

Some nitpicks and suggestions, but this overall looks good. I appreciate you adding unit tests for the new validation logic. Though it would be nice to have even more tests to verify the other change that was made. (Ensuring that we don't serialize the Stream field into JSON.)

[chrsmith]

Can you elaborate on these comments to clarify? This is definitely going to confuse future people who come across the code.

Would it be correct to say:

// Stream is used for our internal routing of the Google Request, and is not part
// of the Google API shape. So we make sure to not include it when marshaling into JSON.

Is that right? Also, I believe it will be the case that unexported fields do not get serialized to JSON by default. So you might be able to "fix" this by renaming the field to stream. But I think having a comment calling out the details here is far preferable.

[abeatrix]

Go noob here so i might misunderstand the issue ✋

We currently have the field as stream so i thought the fix is to remove it here. Your comment makes more sense though, updating to use yours instead!

[abeatrix]

Going to test the latest commit again.

Update

Confirmed the calls worked from the latest commit build.

1. Request to the Stream endpoint with 'stream' included in the request body:

❯ curl 'https://sourcegraph.test:3443/.api/completions/stream' -i \
-X POST \
-H 'authorization: token $LOCAL_SG_TOKEN' \
--data-raw '{"messages":[{"speaker":"human","text":"Who are you?"}],"maxTokensToSample":30,"temperature":0,"stopSequences":[],"timeoutMs":5000,"stream":true}'
HTTP/2 200
access-control-allow-credentials: true
access-control-allow-origin:
alt-svc: h3=":3443"; ma=2592000
cache-control: no-cache
content-type: text/event-stream
date: Fri, 14 Jun 2024 18:28:39 GMT
server: Caddy
server: Caddy
vary: Accept-Encoding, Authorization, Cookie, Authorization, X-Requested-With, Cookie
x-accel-buffering: no
x-content-type-options: nosniff
x-frame-options: DENY
x-powered-by: Express
x-trace: f3e7e8625d6a6c47684f1cb3058611b5
x-trace-span: 3b4ef605a4f7947c
x-trace-url: https://sourcegraph.test:3443/-/debug/jaeger/trace/f3e7e8625d6a6c47684f1cb3058611b5
x-xss-protection: 1; mode=block

event: completion
data: {"completion":"I","stopReason":"STOP"}

event: completion
data: {"completion":"I am a large language model, trained by Google. \n\nHere's what","stopReason":"STOP"}

event: completion
data: {"completion":"I am a large language model, trained by Google. \n\nHere's what that means:\n\n* **I'm a computer program:**","stopReason":"STOP"}

event: done
data: {}

2. Request to the Stream endpoint with 'stream' included in the request body:

❯ curl 'https://sourcegraph.test:3443/.api/completions/stream' -i \
-X POST \
-H 'authorization: token $LOCAL_SG_TOKEN' \
--data-raw '{"messages":[{"speaker":"human","text":"Who are you?"}],"maxTokensToSample":30,"temperature":0,"stopSequences":[],"timeoutMs":5000}'
HTTP/2 200
access-control-allow-credentials: true
access-control-allow-origin:
alt-svc: h3=":3443"; ma=2592000
cache-control: no-cache
content-type: text/event-stream
date: Fri, 14 Jun 2024 18:52:11 GMT
server: Caddy
server: Caddy
vary: Accept-Encoding, Authorization, Cookie, Authorization, X-Requested-With, Cookie
x-accel-buffering: no
x-content-type-options: nosniff
x-frame-options: DENY
x-powered-by: Express
x-trace: c2400abb143637080dce9ff821fcd22b
x-trace-span: b7b00a96224a1973
x-trace-url: https://sourcegraph.test:3443/-/debug/jaeger/trace/c2400abb143637080dce9ff821fcd22b
x-xss-protection: 1; mode=block

event: completion
data: {"completion":"I","stopReason":"STOP"}

event: completion
data: {"completion":"I am a large language model, trained by Google. \n\nHere are some key","stopReason":"STOP"}

event: completion
data: {"completion":"I am a large language model, trained by Google. \n\nHere are some key things to know about me:\n\n* **I am not a","stopReason":"STOP"}

event: done
data: {}

3. Request to the Completion endpoint with the 'stream' field in the request body:

~/dev on ☁️  (us-west-1) on ☁️  beatrix@sourcegraph.com
❯ curl 'https://sourcegraph.test:3443/.api/completions/code' -i \
-X POST \
-H 'authorization: token $LOCAL_SG_TOKEN' \
--data-raw '{"messages":[{"speaker":"human","text":"Who are you?"}],"maxTokensToSample":30,"temperature":0,"stopSequences":[],"timeoutMs":5000,"stream":false}'
HTTP/2 200
access-control-allow-credentials: true
access-control-allow-origin:
alt-svc: h3=":3443"; ma=2592000
cache-control: no-cache, max-age=0
content-type: text/plain; charset=utf-8
date: Fri, 14 Jun 2024 18:28:52 GMT
server: Caddy
server: Caddy
vary: Accept-Encoding, Authorization, Cookie, Authorization, X-Requested-With, Cookie
x-content-type-options: nosniff
x-frame-options: DENY
x-powered-by: Express
x-trace: cfb6e2bb518ebf34de27497d4344048b
x-trace-span: 53a0a2c0ba89ea56
x-trace-url: https://sourcegraph.test:3443/-/debug/jaeger/trace/cfb6e2bb518ebf34de27497d4344048b
x-xss-protection: 1; mode=block
content-length: 150

{"completion":"I am a large language model, trained by Google. \n\nHere's what that means:\n\n* **Large Language Model:** I am a","stopReason":"STOP"}%

4. Request to the Completion endpoint without the 'stream' field in the request body:

~/dev on ☁️  (us-west-1) on ☁️  beatrix@sourcegraph.com
❯ curl 'https://sourcegraph.test:3443/.api/completions/code' -i \
-X POST \
-H 'authorization: token $LOCAL_SG_TOKEN' \
--data-raw '{"messages":[{"speaker":"human","text":"Who are you?"}],"maxTokensToSample":30,"temperature":0,"stopSequences":[],"timeoutMs":5000}'
HTTP/2 200
access-control-allow-credentials: true
access-control-allow-origin:
alt-svc: h3=":3443"; ma=2592000
cache-control: no-cache, max-age=0
content-type: text/plain; charset=utf-8
date: Fri, 14 Jun 2024 18:29:13 GMT
server: Caddy
server: Caddy
vary: Accept-Encoding, Authorization, Cookie, Authorization, X-Requested-With, Cookie
x-content-type-options: nosniff
x-frame-options: DENY
x-powered-by: Express
x-trace: 30b3f4399615926e3757b2a69bc0e64e
x-trace-span: e6188229ce141554
x-trace-url: https://sourcegraph.test:3443/-/debug/jaeger/trace/30b3f4399615926e3757b2a69bc0e64e
x-xss-protection: 1; mode=block
content-length: 147

{"completion":"I am a large language model, trained by Google. \n\nHere's what that means:\n\n* **Large Language Model:** I'm","stopReason":"STOP"}%