Skip to content

[bugfix]Fix chunk.choices[0].delta.content is None in query response#520

Merged
AlonsoGuevara merged 1 commit into
microsoft:mainfrom
KylinMountain:deltanone
Jul 12, 2024
Merged

[bugfix]Fix chunk.choices[0].delta.content is None in query response#520
AlonsoGuevara merged 1 commit into
microsoft:mainfrom
KylinMountain:deltanone

Conversation

@KylinMountain

@KylinMountain KylinMountain commented Jul 12, 2024

Copy link
Copy Markdown
Contributor

Description

When querying, sometimes it will report TypeError: can only concatenate str (not "NoneType") to str, it is because the ChoiceDelta.content is None as following debug log.

ChatCompletionChunk(id='chatcmpl-58de5f44-f801-48c8-9cae-8629e4e5051b', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, role=None, tool_calls=None), finish_reason='stop', index=0, logprobs=None)], created=1720755079, model='gemma2-9b-it', object='chat.completion.chunk', service_tier=None, system_fingerprint='fp_10c08bf97d', usage=None, x_groq={'id': 'req_01j2jfxhv1ff9s6y2hqhnr0zk1', 'usage': {'queue_time': 0.023216143, 'prompt_tokens': 924, 'prompt_time': 0.032787483, 'completion_tokens': 99, 'completion_time': 0.198, 'total_tokens': 1023, 'total_time': 0.23078748300000002}})

Exception in reduce_response
Traceback (most recent call last):
  File "/Users/evilkylin/Projects/graphrag/graphrag/query/structured_search/global_search/search.py", line 324, in _reduce_response
    search_response = await self.llm.agenerate(
  File "/Users/evilkylin/Projects/graphrag/graphrag/query/llm/oai/chat_openai.py", line 110, in agenerate
    async for attempt in retryer:
  File "/Users/evilkylin/Library/Caches/pypoetry/virtualenvs/graphrag-g0mKwYYC-py3.10/lib/python3.10/site-packages/tenacity/asyncio/__init__.py", line 166, in __anext__
    do = await self.iter(retry_state=self._retry_state)
  File "/Users/evilkylin/Library/Caches/pypoetry/virtualenvs/graphrag-g0mKwYYC-py3.10/lib/python3.10/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
    result = await action(retry_state)
  File "/Users/evilkylin/Library/Caches/pypoetry/virtualenvs/graphrag-g0mKwYYC-py3.10/lib/python3.10/site-packages/tenacity/_utils.py", line 99, in inner
    return call(*args, **kwargs)
  File "/Users/evilkylin/Library/Caches/pypoetry/virtualenvs/graphrag-g0mKwYYC-py3.10/lib/python3.10/site-packages/tenacity/__init__.py", line 398, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
  File "/Users/evilkylin/Projects/miniforge3/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/Users/evilkylin/Projects/miniforge3/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/Users/evilkylin/Projects/graphrag/graphrag/query/llm/oai/chat_openai.py", line 112, in agenerate
    return await self._agenerate(
  File "/Users/evilkylin/Projects/graphrag/graphrag/query/llm/oai/chat_openai.py", line 192, in _agenerate
    full_response += delta
TypeError: can only concatenate str (not "NoneType") to str

Related Issues

Proposed Changes

Since the ChoiceDelta.content is defined as content: Optional[str] = None, we should avoid assign None to delta.

Checklist

  • I have tested these changes locally.
  • I have reviewed the code changes.
  • I have updated the documentation (if necessary).
  • I have added appropriate unit tests (if applicable).

Additional Notes

[Add any additional notes or context that may be helpful for the reviewer(s).]

@KylinMountain
KylinMountain requested a review from a team as a code owner July 12, 2024 04:03
@AlonsoGuevara
AlonsoGuevara merged commit ec37389 into microsoft:main Jul 12, 2024
@KylinMountain
KylinMountain deleted the deltanone branch August 9, 2024 10:40
Brandsma pushed a commit to ThalamusLabs/MMGraphRAG that referenced this pull request Nov 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants