Skip to content

Commit eb150a6

Browse files
lgrammellgrammel
authored
ai/core: remove scaling of setting values. (#1418)
1 parent 202f9ca commit eb150a6

File tree

16 files changed

+46
-240
lines changed

16 files changed

+46
-240
lines changed

‎.changeset/fair-avocados-itch.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
---
2+
'@ai-sdk/provider-utils': patch
3+
'@ai-sdk/anthropic': patch
4+
'@ai-sdk/provider': patch
5+
'@ai-sdk/mistral': patch
6+
'@ai-sdk/openai': patch
7+
'ai': patch
8+
---
9+
10+
ai/core: remove scaling of setting values (breaking change). If you were using the temperature, frequency penalty, or presence penalty settings, you need to update the providers and adjust the setting values.

‎docs/pages/docs/ai-core/settings.mdx‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -10,19 +10,17 @@ All AI functions (`generateText`, `streamText`, `generateObject`, `streamObject`
1010

1111
- **maxTokens** - Maximum number of tokens to generate.
1212
- **temperature** - Temperature setting.
13-
This is a number between 0 (almost no randomness) and 1 (very random).
13+
The value is passed through to the provider. The range depends on the provider and model.
1414
It is recommended to set either `temperature` or `topP`, but not both.
15-
- **topP** - Nucleus sampling. This is a number between 0 and 1.
16-
E.g. 0.1 would mean that only tokens with the top 10% probability mass are considered.
15+
- **topP** - Nucleus sampling.
16+
The value is passed through to the provider. The range depends on the provider and model.
1717
It is recommended to set either `temperature` or `topP`, but not both.
1818
- **presencePenalty** - Presence penalty setting.
1919
It affects the likelihood of the model to repeat information that is already in the prompt.
20-
The presence penalty is a number between -1 (increase repetition) and 1 (maximum penalty, decrease repetition).
21-
0 means no penalty.
20+
The value is passed through to the provider. The range depends on the provider and model.
2221
- **frequencyPenalty** - Frequency penalty setting.
2322
It affects the likelihood of the model to repeatedly use the same words or phrases.
24-
The frequency penalty is a number between -1 (increase repetition) and 1 (maximum penalty, decrease repetition).
25-
0 means no penalty.
23+
The value is passed through to the provider. The range depends on the provider and model.
2624
- **seed** - The seed (integer) to use for random sampling.
2725
If set and supported by the model, calls will generate deterministic results.
2826
- **maxRetries** - Maximum number of retries. Set to 0 to disable retries. Default: 2.

‎packages/anthropic/src/anthropic-messages-language-model.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ export class AnthropicMessagesLanguageModel implements LanguageModelV1 {
9696

9797
// standardized settings:
9898
max_tokens: maxTokens ?? 4096, // 4096: max model output tokens
99-
temperature, // uses 0..1 scale
99+
temperature,
100100
top_p: topP,
101101

102102
// prompt:

‎packages/core/core/generate-object/generate-object.ts‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -32,19 +32,17 @@ This function does not stream the output. If you want to stream the output, use
3232
3333
@param maxTokens - Maximum number of tokens to generate.
3434
@param temperature - Temperature setting.
35-
This is a number between 0 (almost no randomness) and 1 (very random).
35+
The value is passed through to the provider. The range depends on the provider and model.
3636
It is recommended to set either `temperature` or `topP`, but not both.
37-
@param topP - Nucleus sampling. This is a number between 0 and 1.
38-
E.g. 0.1 would mean that only tokens with the top 10% probability mass are considered.
37+
@param topP - Nucleus sampling.
38+
The value is passed through to the provider. The range depends on the provider and model.
3939
It is recommended to set either `temperature` or `topP`, but not both.
4040
@param presencePenalty - Presence penalty setting.
4141
It affects the likelihood of the model to repeat information that is already in the prompt.
42-
The presence penalty is a number between -1 (increase repetition) and 1 (maximum penalty, decrease repetition).
43-
0 means no penalty.
42+
The value is passed through to the provider. The range depends on the provider and model.
4443
@param frequencyPenalty - Frequency penalty setting.
4544
It affects the likelihood of the model to repeatedly use the same words or phrases.
46-
The frequency penalty is a number between -1 (increase repetition) and 1 (maximum penalty, decrease repetition).
47-
0 means no penalty.
45+
The value is passed through to the provider. The range depends on the provider and model.
4846
@param seed - The seed (integer) to use for random sampling.
4947
If set and supported by the model, calls will generate deterministic results.
5048

‎packages/core/core/generate-object/stream-object.ts‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -37,19 +37,17 @@ This function streams the output. If you do not want to stream the output, use `
3737
3838
@param maxTokens - Maximum number of tokens to generate.
3939
@param temperature - Temperature setting.
40-
This is a number between 0 (almost no randomness) and 1 (very random).
40+
The value is passed through to the provider. The range depends on the provider and model.
4141
It is recommended to set either `temperature` or `topP`, but not both.
42-
@param topP - Nucleus sampling. This is a number between 0 and 1.
43-
E.g. 0.1 would mean that only tokens with the top 10% probability mass are considered.
42+
@param topP - Nucleus sampling.
43+
The value is passed through to the provider. The range depends on the provider and model.
4444
It is recommended to set either `temperature` or `topP`, but not both.
4545
@param presencePenalty - Presence penalty setting.
4646
It affects the likelihood of the model to repeat information that is already in the prompt.
47-
The presence penalty is a number between -1 (increase repetition) and 1 (maximum penalty, decrease repetition).
48-
0 means no penalty.
47+
The value is passed through to the provider. The range depends on the provider and model.
4948
@param frequencyPenalty - Frequency penalty setting.
5049
It affects the likelihood of the model to repeatedly use the same words or phrases.
51-
The frequency penalty is a number between -1 (increase repetition) and 1 (maximum penalty, decrease repetition).
52-
0 means no penalty.
50+
The value is passed through to the provider. The range depends on the provider and model.
5351
@param seed - The seed (integer) to use for random sampling.
5452
If set and supported by the model, calls will generate deterministic results.
5553

‎packages/core/core/generate-text/generate-text.ts‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -29,19 +29,17 @@ This function does not stream the output. If you want to stream the output, use
2929
3030
@param maxTokens - Maximum number of tokens to generate.
3131
@param temperature - Temperature setting.
32-
This is a number between 0 (almost no randomness) and 1 (very random).
32+
The value is passed through to the provider. The range depends on the provider and model.
3333
It is recommended to set either `temperature` or `topP`, but not both.
34-
@param topP - Nucleus sampling. This is a number between 0 and 1.
35-
E.g. 0.1 would mean that only tokens with the top 10% probability mass are considered.
34+
@param topP - Nucleus sampling.
35+
The value is passed through to the provider. The range depends on the provider and model.
3636
It is recommended to set either `temperature` or `topP`, but not both.
3737
@param presencePenalty - Presence penalty setting.
3838
It affects the likelihood of the model to repeat information that is already in the prompt.
39-
The presence penalty is a number between -1 (increase repetition) and 1 (maximum penalty, decrease repetition).
40-
0 means no penalty.
39+
The value is passed through to the provider. The range depends on the provider and model.
4140
@param frequencyPenalty - Frequency penalty setting.
4241
It affects the likelihood of the model to repeatedly use the same words or phrases.
43-
The frequency penalty is a number between -1 (increase repetition) and 1 (maximum penalty, decrease repetition).
44-
0 means no penalty.
42+
The value is passed through to the provider. The range depends on the provider and model.
4543
@param seed - The seed (integer) to use for random sampling.
4644
If set and supported by the model, calls will generate deterministic results.
4745

‎packages/core/core/generate-text/stream-text.ts‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -39,19 +39,17 @@ This function streams the output. If you do not want to stream the output, use `
3939
4040
@param maxTokens - Maximum number of tokens to generate.
4141
@param temperature - Temperature setting.
42-
This is a number between 0 (almost no randomness) and 1 (very random).
42+
The value is passed through to the provider. The range depends on the provider and model.
4343
It is recommended to set either `temperature` or `topP`, but not both.
44-
@param topP - Nucleus sampling. This is a number between 0 and 1.
45-
E.g. 0.1 would mean that only tokens with the top 10% probability mass are considered.
44+
@param topP - Nucleus sampling.
45+
The value is passed through to the provider. The range depends on the provider and model.
4646
It is recommended to set either `temperature` or `topP`, but not both.
4747
@param presencePenalty - Presence penalty setting.
4848
It affects the likelihood of the model to repeat information that is already in the prompt.
49-
The presence penalty is a number between -1 (increase repetition) and 1 (maximum penalty, decrease repetition).
50-
0 means no penalty.
49+
The value is passed through to the provider. The range depends on the provider and model.
5150
@param frequencyPenalty - Frequency penalty setting.
5251
It affects the likelihood of the model to repeatedly use the same words or phrases.
53-
The frequency penalty is a number between -1 (increase repetition) and 1 (maximum penalty, decrease repetition).
54-
0 means no penalty.
52+
The value is passed through to the provider. The range depends on the provider and model.
5553
@param seed - The seed (integer) to use for random sampling.
5654
If set and supported by the model, calls will generate deterministic results.
5755

‎packages/core/core/prompt/prepare-call-settings.ts‎

Lines changed: 0 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -39,14 +39,6 @@ export function prepareCallSettings({
3939
message: 'temperature must be a number',
4040
});
4141
}
42-
43-
if (temperature < 0 || temperature > 1) {
44-
throw new InvalidArgumentError({
45-
parameter: 'temperature',
46-
value: temperature,
47-
message: 'temperature must be between 0 and 1 (inclusive)',
48-
});
49-
}
5042
}
5143

5244
if (topP != null) {
@@ -57,14 +49,6 @@ export function prepareCallSettings({
5749
message: 'topP must be a number',
5850
});
5951
}
60-
61-
if (topP < 0 || topP > 1) {
62-
throw new InvalidArgumentError({
63-
parameter: 'topP',
64-
value: topP,
65-
message: 'topP must be between 0 and 1 (inclusive)',
66-
});
67-
}
6852
}
6953

7054
if (presencePenalty != null) {
@@ -75,14 +59,6 @@ export function prepareCallSettings({
7559
message: 'presencePenalty must be a number',
7660
});
7761
}
78-
79-
if (presencePenalty < -1 || presencePenalty > 1) {
80-
throw new InvalidArgumentError({
81-
parameter: 'presencePenalty',
82-
value: presencePenalty,
83-
message: 'presencePenalty must be between -1 and 1 (inclusive)',
84-
});
85-
}
8662
}
8763

8864
if (frequencyPenalty != null) {
@@ -93,14 +69,6 @@ export function prepareCallSettings({
9369
message: 'frequencyPenalty must be a number',
9470
});
9571
}
96-
97-
if (frequencyPenalty < -1 || frequencyPenalty > 1) {
98-
throw new InvalidArgumentError({
99-
parameter: 'frequencyPenalty',
100-
value: frequencyPenalty,
101-
message: 'frequencyPenalty must be between -1 and 1 (inclusive)',
102-
});
103-
}
10472
}
10573

10674
if (seed != null) {

‎packages/mistral/src/mistral-chat-language-model.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ export class MistralChatLanguageModel implements LanguageModelV1 {
8787

8888
// standardized settings:
8989
max_tokens: maxTokens,
90-
temperature, // uses 0..1 scale
90+
temperature,
9191
top_p: topP,
9292
random_seed: seed,
9393

‎packages/openai/src/openai-chat-language-model.test.ts‎

Lines changed: 0 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -330,51 +330,6 @@ describe('doStream', () => {
330330
});
331331
});
332332

333-
it('should scale the temperature', async () => {
334-
prepareStreamResponse({ content: [] });
335-
336-
await provider.chat('gpt-3.5-turbo').doStream({
337-
inputFormat: 'prompt',
338-
mode: { type: 'regular' },
339-
prompt: TEST_PROMPT,
340-
temperature: 0.5,
341-
});
342-
343-
expect((await server.getRequestBodyJson()).temperature).toBeCloseTo(1, 5);
344-
});
345-
346-
it('should scale the frequency penalty', async () => {
347-
prepareStreamResponse({ content: [] });
348-
349-
await provider.chat('gpt-3.5-turbo').doStream({
350-
inputFormat: 'prompt',
351-
mode: { type: 'regular' },
352-
prompt: TEST_PROMPT,
353-
frequencyPenalty: 0.2,
354-
});
355-
356-
expect((await server.getRequestBodyJson()).frequency_penalty).toBeCloseTo(
357-
0.4,
358-
5,
359-
);
360-
});
361-
362-
it('should scale the presence penalty', async () => {
363-
prepareStreamResponse({ content: [] });
364-
365-
await provider.chat('gpt-3.5-turbo').doStream({
366-
inputFormat: 'prompt',
367-
mode: { type: 'regular' },
368-
prompt: TEST_PROMPT,
369-
presencePenalty: -0.9,
370-
});
371-
372-
expect((await server.getRequestBodyJson()).presence_penalty).toBeCloseTo(
373-
-1.8,
374-
5,
375-
);
376-
});
377-
378333
it('should pass custom headers', async () => {
379334
prepareStreamResponse({ content: [] });
380335

0 commit comments

Comments
 (0)