Music playback

Corresponding example:

• examples/posix/ai/rtc-tcp-client/music_play_demo.c

This chapter introduces the music playback example and its implementation. It demonstrates how to use Agentic-kit to send a text instruction that triggers the cloud AI's Music Skill, parse the returned song metadata (song title, artist, album, and audio URL), and download a preview clip locally.

Features​

The music playback example simulates a typical smart speaker scenario:

1. The user sends a text instruction, such as "播放周杰伦的歌" ("Play songs by Jay Chou").
2. The AI recognizes the intent, triggers the Music Skill, and returns song information in a SKILL response.
3. The example parses the song metadata and displays it in the console.
4. It downloads an audio preview clip to a local file using curl.

The example:

• Prints the AI's NLG text to the console as it streams in.
• Extracts and formats song information when it detects a music SKILL response.
• Downloads a preview clip to output_music.mp3.
• Includes application-side reconnection logic with exponential backoff and a circuit breaker.

Build​

cd examples/posix
cmake -S . -B build
cmake --build build --target music_play_demo

You can also build all POSIX examples:

cmake --build build

Run​

Run the following commands from examples/posix. The Chinese query strings are literal demo inputs: the default means "Play songs by Jay Chou", and the custom query means "Play pop music".

# Default query ("播放周杰伦的歌"), using built-in test credentials
./build/music_play_demo

# Custom query
./build/music_play_demo "播放流行音乐"

# All arguments
./build/music_play_demo [query] [devid] [secret_key] [local_key]

Example console output after a successful run (literal Chinese output preserved):

=== music_play_demo ===
Device ID : 6cd370251e8be96de8vwoe
Query     : 播放周杰伦的歌
[main] Connecting to TAI server...
[main] Connected.

[main] Sending text: "播放周杰伦的歌"
Response: 正在为您播放周杰伦的歌

  +------------------------------------------+
  |               MUSIC FOUND                |
  +------------------------------------------+
  | Song    : 开不了口                       |
  | Artist  : 周杰伦                         |
  | Album   : 范特西                         |
  | Format  : mp3                            |
  | AudioID : ...                            |
  +------------------------------------------+
  Audio : https://...mp3
  Cover : https://...jpg

[main] Downloading: https://...mp3
[main] Saved to: output_music.mp3
[main] Play with: afplay output_music.mp3   (macOS)
                  mpv output_music.mp3      (Linux)

Done.

The response means "Playing songs by Jay Chou for you". The song is "开不了口" ("Can't Speak"), by Jay Chou ("周杰伦"), from the album Fantasy ("范特西"). These literal metadata values also appear in the SKILL response below.

The exit code can be used directly in scripts: 0 means a conversation completed normally; 1 means the connection timed out, a detected music response failed to parse, or the preview download failed. A query that does not trigger the Music Skill (the AI replies with text only) is not a failure.

Key implementation details​

This example uses rtc-tcp-client (the open-source TCP transport layer) and its tai_* API.

Overall flow​

iot_client_init → iot_client_get_session_token → parse_token
    → tai_ctx_init → tai_connect → tai_send_text → wait for callbacks → tai_disconnect

1. Obtain a Session token​

Use iot-client to obtain a Session token containing the TAI Connection information:

iot_client_t *iot = iot_client_init(&iot_cfg);

char *token = calloc(1, 4096);
iot_client_get_session_token(iot, NULL, token, 4096);

Base64-decoding the token yields JSON with two parts: the connection address (connect_conf) and Session configuration (session_conf).

2. Initialize the TAI Connection​

tai_config_t tai_cfg = {
    .host              = cp.host,
    .port              = cp.port,
    .tls_sni           = cp.tls_sni,
    .device_id         = cp.derived_client_id,
    .local_key         = local_key,
    .protocol_version  = TAI_VER_21,
    .client_type       = TAI_CLIENT_DEVICE,
    .sign_level        = TAI_SIGN_HMAC_SHA256,
    .biz_code          = cp.biz_code,
    .biz_tag           = cp.biz_tag,
    .agent_token       = cp.agent_token,
    .session_attrs_json   = SESSION_ATTRS,
    .event_user_data_json = EVENT_USER_DATA,
    .pal               = pal,
    .on_text           = on_text,
    .on_audio          = on_audio,
    .on_event          = on_event,
    .on_disconnect     = on_disconnect,
    .user_data         = &dc,
};

tai_ctx_t *ctx = tai_ctx_init(ctx_buf, &tai_cfg);

3. Send a text instruction​

tai_connect(ctx);
tai_send_text(ctx, query, strlen(query));

4. Parse the music SKILL response​

After triggering the Music Skill, the AI returns a structured SKILL response through the on_text callback. The response format is:

{
  "bizType": "SKILL",
  "data": {
    "code": "music",
    "general": {
      "action": "play",
      "data": {
        "audios": [{
          "name": "开不了口",
          "artist": "周杰伦",
          "album": "范特西",
          "format": "mp3",
          "url": "https://...mp3",
          "audioId": "...",
          "imageUrl": "..."
        }]
      }
    }
  }
}

The example's on_text callback does two things: it prints each NLG text chunk immediately to preserve streaming output, and it accumulates all chunks to parse the SKILL structure once the stream ends:

static void on_text(tai_ctx_t *ctx, const tai_text_msg_t *msg, void *ud)
{
    demo_ctx_t *dc = (demo_ctx_t *)ud;

    /* NLG text: each chunk is a self-contained JSON line. Print it on arrival,
       bounded by msg->len, decoding \n / \" / \uXXXX escapes. A return value
       of 1 means this chunk was NLG and handled, including an empty final chunk
       such as {"content":""}: it is still NLG and must not be printed verbatim. */
    if (nlg_print_content(msg->text, msg->len))
        dc->stream_printed = 1;

    /* Also accumulate the full stream: a SKILL response is one JSON document
       that may span chunks. Parse it only after it has been fully assembled. */
    if (demo_textbuf_accum(&dc->text, msg) == 1)
        handle_complete_text(dc);   /* is_music_response → try_parse_music */
}

The server might not send a separate text END chunk (the SDK discards empty text frames). As a fallback, on_event calls demo_textbuf_flush() when it receives TAI_EVT_END (the end of the Event), delivering any buffered stream.

[demo_text] a new stream started while 214 bytes of the previous one were still buffered: dropping those — ...

[demo_text] text seq gap (11 -> 13): continuing — ...

Also, read the code field from the SKILL envelope's data object, not from the first match in the entire document. With the common outer structure {"code":0,"msg":"ok","data":{"code":"music",...}}, taking the first code yields status code 0 and silently discards the music response.

Shared helper headers​

The five examples under examples/posix/ai/rtc-tcp-client/ share the following helper headers instead of duplicating parsing code:

Every function in demo_json.h requires a \0-terminated buffer. Copy callback msg->text / msg->data first.

NLG text output​

NLG text outside the music response (the text of the AI's spoken reply) is printed incrementally as it streams in. The example uses nlg_print_content() to extract the JSON content field and print only its text after decoding JSON escapes. The server often encodes Chinese characters as \uXXXX; decoding renders readable Chinese characters rather than escape sequences. For example, the literal response below means "Playing songs by Jay Chou for you":

Response: 正在为您播放周杰伦的歌

Copyright and licensing​

Preview limitations

By default, the AI music feature returns preview versions of songs with a duration limit (typically 30 seconds), intended only for feature evaluation and development debugging. Playing complete songs in production products requires purchasing the corresponding advanced music capability license.

NetEase Cloud Music integration

Tuya's content server integrates NetEase Cloud Music to provide search and playback of licensed music. The integration process is:

1. Purchase the advanced capability: In the Product Development flow, enable AI NetEase Cloud Music Playback under Function Definition > Advanced Product Features, then pay the per-device license fee through Deliverable Procurement. The license is valid for three years from device Activation and first use.
2. Add music tools: On the agent development page, add Music and Story Skill under Skill Configuration > Tool Set. It includes tools for media playback control and searching for and playing music or children's songs.
3. Deploy the agent: Deploy the agent with the Music Skill to a product (PID) that has purchased the advanced capability. Devices can then play complete songs.

For detailed platform configuration steps, see the official Music and Story Skill documentation (Chinese).

Considerations​

• Obtain device credentials through Provisioning. The default credentials are for testing only. The device ID and keys are written to fixed-size fields in iot_client_config_t (32 bytes each). The example exits with an error if a value is too long; it does not truncate it.
• Preview downloads depend on the system's curl; make sure it is installed. The example invokes curl directly with fork + execvp, without a shell. The server-provided URL is passed as a single argv element and cannot be executed as a command. Only URLs starting with http:// / https:// are accepted. Retain this constraint when implementing downloads on a device.
• The Music Skill requires a correctly configured workflow on the Tuya AI platform; otherwise, no SKILL response is returned.
• The example waits up to 60 seconds for an AI response, then exits on timeout.
• The current example parses and displays only the first song. To play full audio, implement an audio player on the device.
• The metadata box aligns by display width (Chinese characters occupy two columns), not byte count. Overlong fields are truncated at character boundaries.
• The on_audio callback is empty in this example and does not process TTS audio.
• This example declares MCP support but implements no tools. When on_event receives TAI_EVT_MCP_CMD, it responds using demo_mcp_reply_no_tools() from demo_mcp.h. Note that the SDK's built-in default attributes already enable MCP, so devices that omit session_attrs_json also receive MCP requests and must respond correctly. See mcp_demo.c to implement actual device tools.