Repository navigation
Description
CTTransformerStreaming (funasr/models/ct_transformer_streaming/model.py), the class behind the realtime punctuation model iic/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727 used in examples/industrial_data_pretraining/ct_transformer_streaming/demo.py, fails on empty text. Since 4ad2a7b, the offline CTTransformer.inference returns text: "" and punc_array: None for empty or whitespace-only text, but the streaming class overrides inference without that check:
-
With a fresh cache,
""or" "gives no words.tokenizer.encode([])returns[],np.concatenatewith the empty int32 cache ids turns the ids into float64, and the embedding raises the error users reported for the offline model in runtime error for audio file in which there is actually no speech (blank voice) #1386, 无人声音频时指定punc_model报错 #1625 and auto_model 中punc模型入参为空触发的bug #1660:RuntimeError: Expected tensor for argument #1 'indices' to have one of the following scalar types: Long, Int; but got torch.DoubleTensor instead (while checking arguments for embedding) -
When earlier calls left words in
cache["pre_text"], an empty call runs the model on those words again but does not return them. If the model predicts no punctuation after the last of them, the output string is empty andif sentence_out[-1] in self.punc_list:raisesIndexError: string index out of range.
AutoModel.generate(input="", cache=cache) passes the empty string straight to inference. In the demo's loop, a VAD segment with empty ASR text therefore raises the RuntimeError when it comes first, and can raise the IndexError later. runtime/python/websocket/funasr_wss_server.py only calls this model when len(text) > 0, but direct AutoModel callers have no such guard.
Repro on main (8710e1c) without downloading the model, using a tiny randomly initialised model of the same class:
from funasr.models.ct_transformer_streaming.model import CTTransformerStreaming
from funasr.tokenizer.char_tokenizer import CharTokenizer
model = CTTransformerStreaming(
encoder="SANMVadEncoder",
encoder_conf=dict(
input_size=8, output_size=8, attention_heads=2, linear_units=16, num_blocks=2, input_layer="pe"
),
vocab_size=5,
punc_list=["<unk>", "_", ",", "。", "?", "、"],
embed_unit=8,
att_unit=8,
)
tokenizer = CharTokenizer(token_list=["<blank>", "<s>", "</s>", "<unk>", "是"])
print(model.inference([""], key=["utt"], tokenizer=tokenizer, cache={}, device="cpu"))This raises the RuntimeError above. Expected: an empty result (text: "", punc_array: None), as CTTransformer returns, with the cache left as it was.
Environment: Windows 11, Python 3.11.17, torch 2.10.0+cpu, numpy 2.4.6, FunASR from source (pip install -e .).
Activity
LauraGPT commentedon Nov 3, 2026
Thanks for the precise reproduction and #3824. The fix is now merged in
2c54fd25: empty/whitespace input returns the empty result without changing pendingpre_text.Both reported failures reproduced on the previous main. The merged tree passed 31 selected CPU tests and real L20Z FP32 empty/cache/nonempty contracts using deterministic tiny random weights, with actual CUDA execution recorded. Both NumPy 1.26.4 and 2.4.0 hosted jobs passed 370 tests on the PR head and were separately accepted on the merged main. This is not a pretrained-quality or full-repository-suite claim; Windows 11 and NumPy 2.4.6 were not reproduced here.
This is available in source main; already-published 1.4.20 does not include it. A new PyPI release is still pending. The issue was closed automatically by the linked merge, not by a separate administrative close.