リアルタイム音声認識 API

Feature Introduction

For long-duration speech data streams, suitable for scenarios requiring continuous recognition over extended periods, such as conference speeches and live video streaming.

Real-time speech recognition only provides a WebSocket (streaming) interface. A single connection theoretically supports an audio duration limit of about 37 hours.

Request URL

ws://<ip_address>:7100/ws/v1

Interaction Process

1. Start and Send Parameters

The client initiates a request, and the server confirms the validity of the request. Parameters must be set within the request message.

Request Parameters (header object):

Request Parameters (payload object):

Example of a request:

{
    "header": {
        "namespace": "SpeechTranscriber",
        "name": "StartTranscription"
    },
    "payload": {
        "lang_type": "ja-JP",
        "format": "pcm",
        "sample_rate": 16000,
        "enable_intermediate_result": true,
        "enable_punctuation_prediction": true,
        "enable_inverse_text_normalization": true,
        "max_sentence_silence": 800,
        "enable_words":true,
        "user_id":"conversation_001"
    }
}

Response Parameters (header object):

Example of a response:

{
    "header":{
        "namespace":"SpeechTranscriber",
        "name":"TranscriptionStarted",
        "appkey":"",
        "status":"00000",
        "status_text":"success",
        "task_id":"0220a729ac9d4c9997f51592ecc83847",
        "message_id":"49b680abe737488cf50f3cd9e3953b97",
        "user_id":"conversation_001"
    },
    "payload":{
        "index":0,
        "time":0,
        "begin_time":0,
        "speaker_id":"",
        "result":"",
        "confidence":0,
        "words":null
    }
}

2. Sending Audio Data and Receiving Recognition Results

Note:

If the source_url parameter is set during the connection, the system will automatically retrieve audio data from the specified source, and you do not need to send audio data according to the content of this section.

Send audio data in a loop and continuously receive recognition results. It is recommended to send data packets of 7680 Bytes each time.

The Real-time Speech Transcription service has an automatic sentence-breaking feature that determines the beginning and end of a sentence according to the length of silence between the utterances, which is represented by events in the returned results. The SentenceBegin and SentenceEnd events respectively indicate the start and end of a sentence, while the TranscriptionResultChanged event indicates the intermediate recognition results of a sentence.

SentenceBegin Event

The SentenceBegin event indicates that the server has detected the beginning of a sentence.

Response Parameters (header object):

Response Parameters (payload object):

Example of a response:

{
    "header": {
        "namespace": "SpeechTranscriber",
        "name": "SentenceBegin",
        "status": "00000",
        "status_text": "success",
        "task_id": "3ee284e922dd4554bb6ccda7989d1973",
        "message_id": "9b680abe73748f50f83cd9e3953b974c",
        "user_id":"conversation_001"
    },
    "payload": {
        "lang_type": "ja-JP",
        "paragraph": 1,
        "index": 1,
        "time": 240,
        "begin_time": 0,
        "speaker_id": "",
        "result": "",
        "confidence": 0,
        "volume": 0        
    }
}

TranscriptionResultChanged Event

The recognition results are divided into "intermediate results" and "final results". For detailed explanations, please refer to the "Basic Terminology" section of the "Speech-to-Text Service User Manual".

The TranscriptionResultChanged event indicates that there has been a change in the recognition results, i.e., the intermediate results of a sentence.

• If enable_intermediate_result is set to true, the server will continue to return multiple TranscriptionResultChanged messages, which are the intermediate results of the recognition.

• If enable_intermediate_result is set to false, the server will not return any messages for this step.

Note:

The last intermediate result obtained may not be the same as the final result. Please take the result corresponding to the SentenceEnd event as the final recognition result.

Response Parameters (header object):

The header object parameters are the same as in the previous table (see the SentenceBegin event header object return parameters), with name being TranscriptionResultChanged indicating the intermediate recognition result of a sentence.

Response Parameters (payload object):

Example of a response:

{
    "header": {
        "namespace": "SpeechTranscriber",
        "name": "TranscriptionResultChanged",
        "status": "00000",
        "status_text": "success",
        "task_id": "3ee284e922dd4554bb6ccda7989d1973",
        "message_id": "749b680abe733cd488cf50f9e3953b97",
        "user_id":"conversation_001"
    },
    "payload": {
        "lang_type": "ja-JP",
        "paragraph": 1,
        "index": 1,
        "time": 1920,
        "begin_time": 0,
        "speaker_id": "",
        "result": "天気",
        "confidence": 1,
        "volume": 79,        
        "words": [
            {
                "word": "天気",
                "start_time": 0,
                "end_time": 1920,
                "stable": false
            }
        ]
    }
}

SentenceEnd Event

The SentenceEnd event signifies that the service has detected the end of a spoken sentence and returns the final transcription result for that sentence.

Response Parameters (header object):

The header object parameters are consistent with those described in the SentenceBegin event, with the name attribute being SentenceEnd to indicate the recognition of the sentence's end.

Response Parameters (payload object):

The structure of the final result word information object is as follows:

Example of a response:

{
    "header": {
        "namespace": "SpeechTranscriber",
        "name": "SentenceEnd",
        "status": "00000",
        "status_text": "success",
        "task_id": "3ee284e922dd4554bb6ccda7989d1973",
        "message_id": "749b680abe737488cf50f3cd9e3953b9",
        "user_id":"conversation_001"
    },
    "payload": {
        "lang_type": "zh-cmn-Hans-CN",
        "paragraph": 1,
        "index": 1,
        "time": 5670,
        "begin_time": 390,
        "speaker_id": "speaker1",
        "result": "天気がいいから、散歩しましょう。",
        "confidence": 0.9,
        "volume": 76,
        "words": [{
            "word": "天気",
            "start_time": 390,
            "end_time": 1110,
            "type": "normal"
        }, {
            "word": "が",
            "start_time": 1110,
            "end_time": 1440,
            "type": "normal"
        }, {
            "word": "いい",
            "start_time": 1440,
            "end_time": 2130,
            "type": "normal"
        }, {
            "word": "から",
            "start_time": 2160,
            "end_time": 3570,
            "type": "normal"
        }, {
            "word": "、",
            "start_time": 4290,
            "end_time": 4860,
            "type": "punc"
        },
        ……略……       
        ]
    }
}

Additional Features

The following three additional features are only available in Real-time Speech Transcription.

Note:

If the source_url parameter is set during the connection, forced sentence breaking and custom speaker numbering features are not available.

Forced Sentence Breaking

During the transmission of audio data, sending a SentenceEnd event will force the server to break the sentence at the current position. After the server processes the sentence breaking, the client will receive a SentenceEnd message with the final recognition result for that sentence. This feature is not available when using a network sound card.

Example of sending:

{
    "header": {
        "namespace": "SpeechTranscriber",
        "name": "SentenceEnd"
    }
}

Custom Speaker Numbering

Before sending audio data, send a SpeakerStart event and specify speaker_id as the speaker number. The server will consider the audio data between one SpeakerStart event and the next SpeakerStart or StopTranscription event as the specified speaker and will return this information in the speaker_id field of the recognition result. This feature is not available when using a network sound card.

Note:

The speaker_id supports up to 36 characters, and any excess will be truncated and discarded. If the speaker_id parameter is not passed in the SpeakerStart event, the speaker_id in the result will be an empty value. The SpeakerStart event will trigger forced sentence breaking. Therefore, please send a SpeakerStart event only before switching speakers.

Example of sending:

Angle brackets <> indicate audio data packets, and curly braces () indicate JSON data packets.

<Binary Audio Data Packet 1>

{
    "header": {
        "namespace": "SpeechTranscriber",
        "name": "SpeakerStart"
    },
    "payload": {
        "speaker_id": "001"
    }
}

<Binary Audio Data Packet 2>

<...>

<Binary Audio Data Packet n>

{
    "header": {
        "namespace": "SpeechTranscriber",
        "name": "SpeakerStart"
    }
    "payload": {
        "speaker_id": "002"
    }
}

<Binary Audio Data Packet n+1>

<Binary Audio Data Packet n+2>

SSE Method for Returning Results

When enable_sse is set to true, after establishing a Real-time Speech Transcription WebSocket connection and receiving the TranscriptionStarted event, another client can retrieve the recognition results using the following HTTP request:

GET http://{server_ip}:7100/getAsrResult?task_id=******

The recognition results are returned streamly via Server-sent events (SSE), and the format is consistent with the content returned over WebSocket. The connection is automatically terminated when the recognition ends.

Quick Test Example (Linux):

curl -X GET 'http://localhost:7100/getAsrResult?task_id=******'

Quick Test Example (Windows):

curl.exe -X GET "http://localhost:7100/getAsrResult?task_id=******"

3. Maintaining Connection (Heartbeat Mechanism)

When not sending audio, it is necessary to send a heartbeat packet at least once every 10 seconds; otherwise, the connection will be automatically terminated. It is recommended that the client sends a heartbeat every 8 seconds.

Note:

Heartbeat packets should be sent after sending the StartRecognition or StartTranscription event.

If the source_url parameter is set during the connection, the heartbeat mechanism does not apply.

Example of sending:

{
    "header":{
        "namespace":"SpeechTranscriber",
        "name":"Ping"
    }
}

Example of a response:

{
    "header": {
        "namespace": "SpeechTranscriber",
        "name": "Pong",
        "task_id": "71c5cb9b-fbc3-4489-843c-e902b102a569",
        "message_id": "6f9ea191-1624-4d3c-9286-0003d323f731"
    },
    "payload": {}
}

If no data is transmitted to the server within 10 seconds, the server will return an error message and then automatically disconnect the connection.

4. Stop and Retrieve Final Results

The client sends a request to stop Real-time Speech Transcription, notifying the server that the transmission of voice data has ended and to terminate speech recognition. The server returns the final recognition results and then automatically disconnects the connection.

Request Parameters (header object):

Example of a request:

{
    "header": {
        "namespace": "SpeechTranscriber",
        "name": "StopTranscription"
    }
}

Response Parameters (header object):

Response Parameters (payload object): The format is the same as the SentenceEnd event, but the result and words fields may be empty.

Example of a response:

{
    "header":{
        "namespace":"SpeechTranscriber",
        "name":"TranscriptionCompleted",
        "status":"00000",
        "status_text":"success",
        "task_id":"3ee284e922dd4554bb6ccda7989d1973",
        "message_id":"7e729bf2d4064fee83143c4d962dc6f1",
        "user_id":""
    },
    "payload":{
        "index":1,
        "time":4765,
        "begin_time":180,
        "speaker_id":"",
        "result":"",
        "confidence": 0,
        "volume": 0,        
        "words":[]
    }
}

5. Error Messages

Error messages are returned through the WebSocket connection, after which the server automatically closes the connection.

Example of a response:

{
    "header": {
        "namespace": "SpeechTranscriber",
        "name": "TaskFailed",
        "status": "20105",
        "status_text": "JSON serialization failed",
        "task_id": "df2c1604e31d4f46a7a064db73cd3b5e",
        "message_id": "",
        "user_id": ""
    },
    "payload": {
        "index": 1,
        "time": 0,
        "begin_time": 0,
        "speaker_id": "",
        "result": "",
        "confidence": 1,
        "volume": 0,        
        "words": null
    }
}

Service Status Codes