Skip to content

Adds support for StarChat for autocomplete#847

Merged
sestinj merged 1 commit intocontinuedev:previewfrom
ajalexander:add-starchat-to-autocomplete
Feb 12, 2024
Merged

Adds support for StarChat for autocomplete#847
sestinj merged 1 commit intocontinuedev:previewfrom
ajalexander:add-starchat-to-autocomplete

Conversation

@ajalexander
Copy link
Contributor

This change updates the handling of template generation to explicitly treat StarChat models the same as StarCoder models.

The current issue

While the current implementation of template generate does fallback to the stableCodeFirmTemplate if there aren't matches, the specific StarChat model I'm using hits an edge case. I'm using the starchat-beta-GPTQ model. Since it includes gpt in the lowercase name, the current template support ends up using the OpenAI / GPT style prompt.

The change

By adding starchat as one of the search strings alongside the starcoder models, the template code will now return the stableCodeFirmTemplate rather than the gptAutocompleteTemplate.

Testing

Prior to the change, the extension output would include:

Your task is to complete the line at the end of this code block:
\```
def add(a, b)
\```

The last line is incomplete, and you should provide the rest of that line. If the line is already complete, just return a new line. Otherwise, DO NOT provide explanation, a code block, or extra whitespace, just the code that should be added to the last line to complete 

After the change, the extension output is:

Settings:
contextLength: 4096
model: TheBloke/starchat-beta-GPTQ
maxTokens: 1024
stop: <fim_prefix>,<fim_suffix>,<fim_middle>,<|endoftext|>,

,```,function,class,module,export 
temperature: 0

############################################

<fim_prefix>def add(a, b<fim_suffix>)<fim_middle>

Updates the handling of template generation to treat StarChat models the same as StarCoder models.
@sestinj
Copy link
Contributor

sestinj commented Feb 12, 2024

Nice, thanks for doing this!

@sestinj sestinj merged commit 53c591f into continuedev:preview Feb 12, 2024
@ajalexander ajalexander deleted the add-starchat-to-autocomplete branch February 12, 2024 20:03
@nashiba
Copy link

nashiba commented Mar 8, 2026

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants