Skip to content

Realtime punctuation (CTTransformerStreaming) fails on empty text #3823

Closed

Description

@tanishqbaweja

CTTransformerStreaming (funasr/models/ct_transformer_streaming/model.py), the class behind the realtime punctuation model iic/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727 used in examples/industrial_data_pretraining/ct_transformer_streaming/demo.py, fails on empty text. Since 4ad2a7b, the offline CTTransformer.inference returns text: "" and punc_array: None for empty or whitespace-only text, but the streaming class overrides inference without that check:

AutoModel.generate(input="", cache=cache) passes the empty string straight to inference. In the demo's loop, a VAD segment with empty ASR text therefore raises the RuntimeError when it comes first, and can raise the IndexError later. runtime/python/websocket/funasr_wss_server.py only calls this model when len(text) > 0, but direct AutoModel callers have no such guard.

Repro on main (8710e1c) without downloading the model, using a tiny randomly initialised model of the same class:

from funasr.models.ct_transformer_streaming.model import CTTransformerStreaming
from funasr.tokenizer.char_tokenizer import CharTokenizer

model = CTTransformerStreaming(
    encoder="SANMVadEncoder",
    encoder_conf=dict(
        input_size=8, output_size=8, attention_heads=2, linear_units=16, num_blocks=2, input_layer="pe"
    ),
    vocab_size=5,
    punc_list=["<unk>", "_", ",", "。", "?", "、"],
    embed_unit=8,
    att_unit=8,
)
tokenizer = CharTokenizer(token_list=["<blank>", "<s>", "</s>", "<unk>", "是"])
print(model.inference([""], key=["utt"], tokenizer=tokenizer, cache={}, device="cpu"))

This raises the RuntimeError above. Expected: an empty result (text: "", punc_array: None), as CTTransformer returns, with the cache left as it was.

Environment: Windows 11, Python 3.11.17, torch 2.10.0+cpu, numpy 2.4.6, FunASR from source (pip install -e .).

Activity

  1. LauraGPT commented on Nov 3, 2026

    @LauraGPT
    Collaborator

    Thanks for the precise reproduction and #3824. The fix is now merged in 2c54fd25: empty/whitespace input returns the empty result without changing pending pre_text.

    Both reported failures reproduced on the previous main. The merged tree passed 31 selected CPU tests and real L20Z FP32 empty/cache/nonempty contracts using deterministic tiny random weights, with actual CUDA execution recorded. Both NumPy 1.26.4 and 2.4.0 hosted jobs passed 370 tests on the PR head and were separately accepted on the merged main. This is not a pretrained-quality or full-repository-suite claim; Windows 11 and NumPy 2.4.6 were not reproduced here.

    This is available in source main; already-published 1.4.20 does not include it. A new PyPI release is still pending. The issue was closed automatically by the linked merge, not by a separate administrative close.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions