# 超拟人数字人交互SDK文档

# 1 服务说明

# 1.1 服务描述

基于星火大模型的多模态交互技术，直接贯穿了文本、语音、数字人，实现数字人的表情、动作、声音和情感统一；引领新时代人机交互体验，为用户为打造具象化的私人助理，增强互动体验。

# 1.2 直接体验操作说明：

步骤一：从产品页面 (opens new window)领取免费套餐包；
步骤二：在控制台页面 (opens new window)下载SDK包；
步骤三：按照如下步骤打开项目进行运行：
1、使用Android studio打开此项目，即选择打开sample文件夹下的AIUISimpleDemo
2、待android studio加载项目完成后，将project选择为project file模式
3、最上面的assets路径下找到cfg文件夹，找到aiui_phone.cfg文件，修改登录参数，填入自己的appid、api_secret和key
4、运行demo

# 2 集成方法

# 2.1 配置

SDK的aiui_phone.cfg配置文件：

{
    /* login相关参数 */
    "login":{
        "appid": "xxxxxxxx",    // 同老配置
        "key": "xxxxxxxxxx",    // 同老配置
        "api_secret": "xxxxxx"  // api密钥，在AIUI开放平台我的应用信用中查看。一般不用配置，若是从平台下载的SDK包则不用配置
    },

    /* 大模型 虚拟人 相关参数 */
    "cbmparams": {
        "nlp": {       // 语义参数,一般不用改
            "nlp": {
                "encoding": "utf8",
                "compress": "raw",
                "format": "json"
            },
            "sub_scene": "cbm_v45"
        },

        "avatar": {    // 虚拟人参数
            "width": 512,
            "height": 512,
            "avatar_id": "XXXXXXXXX"  /* 虚拟人形象id */
        }
    },

    /* 其他参数配置 */
}

# 2.2 结果

大模型SDK返回的结果由结果描述和结果数据两部分构成。结果描述信息通过EVENT_RESULT的info字段抛出，为json字符串：

{
    "data": [{
        "params": {
            "sub": "xxx",
        },
        "content": [{
            "dte": "utf8",
            "dtf": "json",
            "cnt_id": "0"
        }]
    }]
}

解析该json，通过cnt_id字段可以获取到结果内容数据，这里以Android为例：

private void processResult(AIUIEvent event) {
    JSONObject data = new JSONObject(event.info).getJSONArray("data").getJSONObject(0);
    String sub = data.getJSONObject("params").optString("sub");
    JSONObject content = data.getJSONArray("content").getJSONObject(0);
    
    if (content.has("cnt_id")) {
        String cnt_id = content.getString("cnt_id");

        // 获取结果数据
        byte[] contentData = event.data.getByteArray(cnt_id);

        // 按sub值的不同，采取不同的方法来处理结果数据
        if ("iat".equals(sub)) {

        } else if ("tts".equals(sub)) {

        } else if ("cbm_tidy".equals(sub))

        } else if ("cbm_semantic".equals(sub)) {

        } else if ("nlp".equals(sub)) {

        } else if ("cbm_vms".equals(sub)) {
        
        }
    }
}

sub字段表示业务名称（即该结果属于什么业务），主要取值说明如下：

sub	说明	结果内容格式
iat	听写结果	json字符串
tts	合成结果	pcm音频数据或者在线音频url
cbm_tidy	意图拆分，判断用户一次说了多少个意图	json字符串
cbm_semantic	只在scene配置中选取大模型语义时才有。通用语义技能结果，相当于老SDK的nlp	json字符串
nlp	当scene配置的是通用语义模型时返回通用语义结果，配置大模型时返回大模型语义结果	json字符串
cbm_vms	当配置了虚拟人参数时, 交互下发虚拟人推流地址	json字符串
event	云端推送的事件, 根据里面的事件类型做不同的业务处理	json字符串

iat和tts结果处理参考结果解析 (opens new window)。
cbm_tidy结果示例：

{
    "cbm_tidy":{
        "compress":"raw",
        "encoding":"utf8",
        "format":"plain",
        "seq":0,       // 结果序号
        "status":3,    // 流式数据状态，取值：0（开始），1（继续），2（结束），3（就一个结果）
        "text":"{\"query\":\"今天天气怎么样?明天的呢\",\"intent\":[{\"index\":0,\"value\":\"今天天气怎么样\"},{\"index\":1,\"value\":\"明天天气怎么样\"}]}"
    }
}

将text取出来并解析成json对象，其中的query为原始输入文本，intent数组为拆分结果。示例中的"今天天气怎么样？明天的呢"被拆分成了两个意图，即"今天天气怎么样"和"明天天气怎么样"。

cbm_semantic结果示例：

{
    "cbm_meta":{      // 该结果的元信息（描述信息）
        "compress":"raw",
        "encoding":"utf8",
        "format":"plain",
        "seq":0,      // 结果序号  
        "status":3,   // 流式数据状态，取值：0（开始），1（继续），2（结束），3（就一个结果）
        "text":"{\"cbm_semantic\":{\"intent\":1}}"  // intent字符取值说明该结果与意图拆分结果的对应关系，这里1对应于上边cbm_intent_split结果中的第二个意图
    },
    "cbm_semantic":{
        "compress":"raw",
        "encoding":"utf8",
        "format":"plain",
        "seq":0,
        "status":3,
        "text":"{\"answer\":{\"text\":\"明天深圳市全天小雨转阴，出门记得带伞，气温26℃ ~ 32℃，有微风，气温较高，请尽量避免午后高温时段的户外活动。\",\"type\":\"T\"},\"category\":\"IFLYTEK.weather\",\"data\":{\"result\":[{\"airData\":40,\"airQuality\":\"优\",\"city\":\"深圳市\",\"date\":\"2023-09-04\",\"dateLong\":1693756800,\"date_for_voice\":\"今天\",\"exp\":{\"ct\":{\"expName\":\"穿衣指数\",\"level\":\"闷热\",\"prompt\":\"潮湿闷热，衣物排汗透气，手帕擦汗环保时尚，不建议在露天场所逛街。\"},\"dy\":{\"expName\":\"钓鱼指数\",\"level\":\"较适宜\",\"prompt\":\"气压小幅波动，可能会影响鱼儿的进食。\"},\"gm\":{\"expName\":\"感冒指数\",\"level\":\"较易发\",\"prompt\":\"感冒较易发生，干净整洁的环境和清新流通的空气都有利于降低感冒的几率，体质较弱的童鞋们要特别加强自我保护。\"},\"jt\":{\"expName\":\"交通指数\",\"level\":\"较好\",\"prompt\":\"有降水，能见度不太好且路面潮湿，不适宜高速行驶，司机应更加集中注意力，小心驾驶。\"},\"tr\":{\"expName\":\"旅游指数\",\"level\":\"较适宜\",\"prompt\":\"有较弱降水和微风作伴的一天，虽然较热，但还是较适宜旅游，可不要错过机会呦！\"},\"uv\":{\"expName\":\"紫外线指数\",\"level\":\"强\",\"prompt\":\"辐射强，请涂擦SPF大于15、PA+防晒护肤品。\"},\"xc\":{\"expName\":\"洗车指数\",\"level\":\"不宜\",\"prompt\":\"不宜洗车，未来24小时内有雨，如果在此期间洗车，雨水和路上的泥水可能会再次弄脏您的爱车。\"},\"yd\":{\"expName\":\"运动指数\",\"level\":\"不适宜\",\"prompt\":\"气温过高，特别容易中暑，较不适宜户外运动，建议室内运动。\"}},\"extra\":\"\",\"humidity\":\"75%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/02.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"pm25\":\"33\",\"precipitation\":\"0\",\"temp\":30,\"tempHigh\":\"33℃\",\"tempLow\":\"26℃\",\"tempRange\":\"26℃ ~ 33℃\",\"tempReal\":\"35℃\",\"visibility\":\"\",\"warning\":\"\",\"weather\":\"阴\",\"weatherDescription\":\"气温较高，请尽量避免午后高温时段的户外活动。\",\"weatherDescription3\":\"25℃到33℃，明天、后天有雨，风不大，气温较高，请尽量避免午后高温时段的户外活动。\",\"weatherDescription7\":\"25℃到33℃，明天到10号有雨，风不大，气温较高，请尽量避免午后高温时段的户外活动。\",\"weatherType\":2,\"week\":\"周一\",\"wind\":\"西北风微风\",\"windLevel\":0},{\"airData\":23,\"airQuality\":\"优\",\"city\":\"深圳市\",\"date\":\"2023-09-03\",\"dateLong\":1693670400,\"date_for_voice\":\"昨天\",\"humidity\":\"76%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/03.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"tempHigh\":\"32℃\",\"tempLow\":\"25℃\",\"tempRange\":\"25℃ ~ 32℃\",\"weather\":\"阵雨转多云\",\"weatherDescription\":\"气温较高，请尽量避免午后高温时段的户外活动。\",\"weatherType\":3,\"week\":\"周日\",\"wind\":\"东南风3-4级\",\"windLevel\":1},{\"airData\":45,\"airQuality\":\"优\",\"city\":\"深圳市\",\"date\":\"2023-09-05\",\"dateLong\":1693843200,\"date_for_voice\":\"明天\",\"humidity\":\"75%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/07.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"tempHigh\":\"32℃\",\"tempLow\":\"26℃\",\"tempRange\":\"26℃ ~ 32℃\",\"weather\":\"小雨转阴\",\"weatherDescription\":\"气温较高，请尽量避免午后高温时段的户外活动。\",\"weatherType\":7,\"week\":\"周二\",\"wind\":\"微风\",\"windLevel\":0},{\"airData\":42,\"airQuality\":\"优\",\"city\":\"深圳市\",\"date\":\"2023-09-06\",\"dateLong\":1693929600,\"date_for_voice\":\"后天\",\"humidity\":\"84%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/07.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"tempHigh\":\"28℃\",\"tempLow\":\"25℃\",\"tempRange\":\"25℃ ~ 28℃\",\"weather\":\"小雨转中雨\",\"weatherDescription\":\"有点热，适合穿短袖短裙等夏季清凉衣物。\",\"weatherType\":7,\"week\":\"周三\",\"wind\":\"微风\",\"windLevel\":0},{\"airData\":45,\"airQuality\":\"优\",\"city\":\"深圳市\",\"date\":\"2023-09-07\",\"dateLong\":1694016000,\"date_for_voice\":\"7号\",\"humidity\":\"85%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/08.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"tempHigh\":\"31℃\",\"tempLow\":\"26℃\",\"tempRange\":\"26℃ ~ 31℃\",\"weather\":\"中雨\",\"weatherDescription\":\"气温较高，请尽量避免午后高温时段的户外活动。\",\"weatherType\":8,\"week\":\"周四\",\"wind\":\"东南风3-4级\",\"windLevel\":1},{\"airData\":50,\"airQuality\":\"优\",\"city\":\"深圳市\",\"date\":\"2023-09-08\",\"dateLong\":1694102400,\"date_for_voice\":\"8号\",\"humidity\":\"81%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/08.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"tempHigh\":\"31℃\",\"tempLow\":\"26℃\",\"tempRange\":\"26℃ ~ 31℃\",\"weather\":\"中雨转阵雨\",\"weatherDescription\":\"气温较高，请尽量避免午后高温时段的户外活动。\",\"weatherType\":8,\"week\":\"周五\",\"wind\":\"东南风3-4级\",\"windLevel\":1},{\"airData\":55,\"airQuality\":\"良\",\"city\":\"深圳市\",\"date\":\"2023-09-09\",\"dateLong\":1694188800,\"date_for_voice\":\"9号\",\"humidity\":\"80%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/03.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"tempHigh\":\"32℃\",\"tempLow\":\"27℃\",\"tempRange\":\"27℃ ~ 32℃\",\"weather\":\"阵雨转多云\",\"weatherDescription\":\"气温较高，请尽量避免午后高温时段的户外活动。\",\"weatherType\":3,\"week\":\"周六\",\"wind\":\"微风\",\"windLevel\":0},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-09-10\",\"dateLong\":1694275200,\"date_for_voice\":\"10号\",\"humidity\":\"79%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/07.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"tempHigh\":\"30℃\",\"tempLow\":\"26℃\",\"tempRange\":\"26℃ ~ 30℃\",\"weather\":\"小雨\",\"weatherDescription\":\"有点热，适合穿短袖短裙等夏季清凉衣物。\",\"weatherType\":7,\"week\":\"周日\",\"wind\":\"微风\",\"windLevel\":0},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-09-11\",\"dateLong\":1694361600,\"date_for_voice\":\"11号\",\"humidity\":\"82%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/07.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"tempHigh\":\"31℃\",\"tempLow\":\"26℃\",\"tempRange\":\"26℃ ~ 31℃\",\"weather\":\"小雨转阴\",\"weatherDescription\":\"气温较高，请尽量避免午后高温时段的户外活动。\",\"weatherType\":7,\"week\":\"下周一\",\"wind\":\"东南风转东风微风\",\"windLevel\":0},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-09-12\",\"dateLong\":1694448000,\"date_for_voice\":\"12号\",\"humidity\":\"83%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/02.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"tempHigh\":\"31℃\",\"tempLow\":\"26℃\",\"tempRange\":\"26℃ ~ 31℃\",\"weather\":\"阴\",\"weatherDescription\":\"气温较高，请尽量避免午后高温时段的户外活动。\",\"weatherType\":2,\"week\":\"下周二\",\"wind\":\"东南风转东北风微风\",\"windLevel\":0},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-09-13\",\"dateLong\":1694534400,\"date_for_voice\":\"13号\",\"humidity\":\"78%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/07.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"tempHigh\":\"30℃\",\"tempLow\":\"26℃\",\"tempRange\":\"26℃ ~ 30℃\",\"weather\":\"小雨转晴\",\"weatherDescription\":\"有点热，适合穿短袖短裙等夏季清凉衣物。\",\"weatherType\":7,\"week\":\"下周三\",\"wind\":\"东风微风\",\"windLevel\":0},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-09-14\",\"dateLong\":1694620800,\"date_for_voice\":\"14号\",\"humidity\":\"77%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/02.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"tempHigh\":\"30℃\",\"tempLow\":\"26℃\",\"tempRange\":\"26℃ ~ 30℃\",\"weather\":\"阴转小雨\",\"weatherDescription\":\"有点热，适合穿短袖短裙等夏季清凉衣物。\",\"weatherType\":2,\"week\":\"下周四\",\"wind\":\"东风4-5级\",\"windLevel\":2},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-09-15\",\"dateLong\":1694707200,\"date_for_voice\":\"15号\",\"humidity\":\"76%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/07.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"tempHigh\":\"30℃\",\"tempLow\":\"26℃\",\"tempRange\":\"26℃ ~ 30℃\",\"weather\":\"小雨\",\"weatherDescription\":\"有点热，适合穿短袖短裙等夏季清凉衣物。\",\"weatherType\":7,\"week\":\"下周五\",\"wind\":\"东风4-5级\",\"windLevel\":2},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-09-16\",\"dateLong\":1694793600,\"date_for_voice\":\"16号\",\"humidity\":\"77%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/07.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"tempHigh\":\"28℃\",\"tempLow\":\"25℃\",\"tempRange\":\"25℃ ~ 28℃\",\"weather\":\"小雨\",\"weatherDescription\":\"有点热，适合穿短袖短裙等夏季清凉衣物。\",\"weatherType\":7,\"week\":\"下周六\",\"wind\":\"东风3-4级\",\"windLevel\":1},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-09-17\",\"dateLong\":1694880000,\"date_for_voice\":\"17号\",\"humidity\":\"73%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/07.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"tempHigh\":\"25℃\",\"tempLow\":\"23℃\",\"tempRange\":\"23℃ ~ 25℃\",\"weather\":\"小雨\",\"weatherDescription\":\"温度适宜。\",\"weatherType\":7,\"week\":\"下周日\",\"wind\":\"东风转东北风微风\",\"windLevel\":0},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-09-18\",\"dateLong\":1694966400,\"date_for_voice\":\"18号\",\"humidity\":\"65%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/07.png\",\"lastUpdateTime\":\"2023-09-04 10:00:08\",\"tempHigh\":\"24℃\",\"tempLow\":\"21℃\",\"tempRange\":\"21℃ ~ 24℃\",\"weather\":\"小雨\",\"weatherDescription\":\"温度适宜。\",\"weatherType\":7,\"week\":\"下下周一\",\"wind\":\"东北风微风\",\"windLevel\":0}]},\"demand_semantic\":{\"datetime.date\":\"明天\",\"queryType\":\"内容\",\"service\":\"weather\",\"subfocus\":\"天气状态\",\"word\":\"有时间\"},\"dialog_stat\":\"DataValid\",\"operation\":\"QUERY\",\"orig_semantic\":{\"slots\":{\"datetime.date\":\"明天\",\"queryType\":\"内容\",\"subfocus\":\"天气状态\",\"word\":\"有时间\"}},\"rc\":0,\"save_history\":true,\"score\":1,\"search_semantic\":{\"datetime.date\":\"明天\",\"queryType\":\"内容\",\"service\":\"weather\",\"subfocus\":\"天气状态\",\"word\":\"有时间\"},\"semantic\":{\"slots\":{\"datetime\":{\"date\":\"2023-09-05\",\"dateOrig\":\"明天\",\"type\":\"DT_BASIC\"},\"location\":{\"city\":\"CURRENT_CITY\",\"poi\":\"CURRENT_POI\",\"type\":\"LOC_POI\"},\"operation\":\"QUERY\",\"queryType\":\"内容\",\"subfocus\":\"天气状态\"}},\"service\":\"weather\",\"shouldEndSession\":\"true\",\"sid\":\"wgw000cbf24@dx18a5dfcb051a14e532\",\"state\":{\"fg::weather::default::default\":{\"state\":\"default\"}},\"text\":\"明天天气怎么样\",\"used_state\":{\"state\":\"default\",\"state_key\":\"fg::weather::default::default\"},\"uuid\":\"wgw000cbf24@dx18a5dfcb051a14e532\",\"version\":\"170.0\"}"
    }
}

cbm_meta为结果的元信息，text字段为通用语义结果json字符串，字符说明参考语义协议 (opens new window)。

nlp结果

AIUI通用语义结果示例如下：

{
    "nlp":{
        "compress":"raw",
        "encoding":"utf8",
        "format":"plain",
        "seq":1,
        "status":2,
        "text":"{\"intent\":{\"answer\":{\"text\":\"今天深圳市全天晴，气温21℃ ~ 29℃，空气质量优，有东北风微风，有点热。\",\"type\":\"T\"},\"category\":\"IFLYTEK.weather\",\"data\":{\"result\":[{\"airData\":44,\"airQuality\":\"优\",\"city\":\"深圳市\",\"date\":\"2023-11-01\",\"dateLong\":1698768000,\"date_for_voice\":\"今天\",\"exp\":{\"ct\":{\"expName\":\"穿衣指数\",\"level\":\"热\",\"prompt\":\"天气较热，衣物精干简洁，室内酌情添加空调衫。\"},\"dy\":{\"expName\":\"钓鱼指数\",\"level\":\"较适宜\",\"prompt\":\"气压小幅波动，可能会影响鱼儿的进食。\"},\"gm\":{\"expName\":\"感冒指数\",\"level\":\"易发\",\"prompt\":\"感冒容易发生，少去人群密集的场所有利于降低感冒的几率。\"},\"jt\":{\"expName\":\"交通指数\",\"level\":\"良好\",\"prompt\":\"天气较好，路面干燥，交通气象条件良好，车辆可以正常行驶。\"},\"tr\":{\"expName\":\"旅游指数\",\"level\":\"适宜\",\"prompt\":\"天气较好，温度适宜，是个好天气哦。这样的天气适宜旅游，您可以尽情地享受大自然的风光。\"},\"uv\":{\"expName\":\"紫外线指数\",\"level\":\"强\",\"prompt\":\"辐射强，请涂擦SPF大于15、PA+防晒护肤品。\"},\"xc\":{\"expName\":\"洗车指数\",\"level\":\"较适宜\",\"prompt\":\"较适宜洗车，未来一天无雨，风力较小，擦洗一新的汽车至少能保持一天。\"},\"yd\":{\"expName\":\"运动指数\",\"level\":\"不适宜\",\"prompt\":\"紫外线太强了，不宜长时间在户外运动。\"}},\"extra\":\"\",\"humidity\":\"56%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/00.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"pm25\":\"29\",\"precipitation\":\"0\",\"temp\":28,\"tempHigh\":\"29℃\",\"tempLow\":\"21℃\",\"tempRange\":\"21℃ ~ 29℃\",\"tempReal\":\"30℃\",\"visibility\":\"\",\"warning\":\"\",\"weather\":\"晴\",\"weatherDescription\":\"有点热。\",\"weatherDescription3\":\"21℃到30℃，风不大，有点热。\",\"weatherDescription7\":\"19℃到30℃，风不大，气候温暖。\",\"weatherType\":0,\"week\":\"周三\",\"wind\":\"东北风微风\",\"windLevel\":0},{\"airData\":41,\"airQuality\":\"优\",\"city\":\"深圳市\",\"date\":\"2023-10-31\",\"dateLong\":1698681600,\"date_for_voice\":\"昨天\",\"humidity\":\"68%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/00.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"tempHigh\":\"29℃\",\"tempLow\":\"21℃\",\"tempRange\":\"21℃ ~ 29℃\",\"weather\":\"晴\",\"weatherDescription\":\"有点热，适合穿短袖短裙等夏季清凉衣物。\",\"weatherType\":0,\"week\":\"周二\",\"wind\":\"微风\",\"windLevel\":0},{\"airData\":50,\"airQuality\":\"优\",\"city\":\"深圳市\",\"date\":\"2023-11-02\",\"dateLong\":1698854400,\"date_for_voice\":\"明天\",\"humidity\":\"75%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/00.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"tempHigh\":\"30℃\",\"tempLow\":\"22℃\",\"tempRange\":\"22℃ ~ 30℃\",\"weather\":\"晴\",\"weatherDescription\":\"有点热。\",\"weatherType\":0,\"week\":\"周四\",\"wind\":\"微风\",\"windLevel\":0},{\"airData\":46,\"airQuality\":\"优\",\"city\":\"深圳市\",\"date\":\"2023-11-03\",\"dateLong\":1698940800,\"date_for_voice\":\"后天\",\"humidity\":\"77%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/01.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"tempHigh\":\"30℃\",\"tempLow\":\"22℃\",\"tempRange\":\"22℃ ~ 30℃\",\"weather\":\"多云\",\"weatherDescription\":\"有点热。\",\"weatherType\":1,\"week\":\"周五\",\"wind\":\"微风\",\"windLevel\":0},{\"airData\":43,\"airQuality\":\"优\",\"city\":\"深圳市\",\"date\":\"2023-11-04\",\"dateLong\":1699027200,\"date_for_voice\":\"4号\",\"humidity\":\"78%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/01.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"tempHigh\":\"29℃\",\"tempLow\":\"22℃\",\"tempRange\":\"22℃ ~ 29℃\",\"weather\":\"多云\",\"weatherDescription\":\"有点热。\",\"weatherType\":1,\"week\":\"周六\",\"wind\":\"微风\",\"windLevel\":0},{\"airData\":36,\"airQuality\":\"优\",\"city\":\"深圳市\",\"date\":\"2023-11-05\",\"dateLong\":1699113600,\"date_for_voice\":\"5号\",\"humidity\":\"81%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/01.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"tempHigh\":\"29℃\",\"tempLow\":\"20℃\",\"tempRange\":\"20℃ ~ 29℃\",\"weather\":\"多云转阴\",\"weatherDescription\":\"有点热。\",\"weatherType\":1,\"week\":\"周日\",\"wind\":\"微风\",\"windLevel\":0},{\"airData\":38,\"airQuality\":\"优\",\"city\":\"深圳市\",\"date\":\"2023-11-06\",\"dateLong\":1699200000,\"date_for_voice\":\"6号\",\"humidity\":\"72%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/02.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"tempHigh\":\"26℃\",\"tempLow\":\"19℃\",\"tempRange\":\"19℃ ~ 26℃\",\"weather\":\"阴转多云\",\"weatherDescription\":\"气候温暖。\",\"weatherType\":2,\"week\":\"下周一\",\"wind\":\"微风\",\"windLevel\":0},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-11-07\",\"dateLong\":1699286400,\"date_for_voice\":\"7号\",\"humidity\":\"68%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/01.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"tempHigh\":\"26℃\",\"tempLow\":\"22℃\",\"tempRange\":\"22℃ ~ 26℃\",\"weather\":\"多云\",\"weatherDescription\":\"气候温暖。\",\"weatherType\":1,\"week\":\"下周二\",\"wind\":\"东风3-4级\",\"windLevel\":1},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-11-08\",\"dateLong\":1699372800,\"date_for_voice\":\"8号\",\"humidity\":\"76%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/01.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"tempHigh\":\"27℃\",\"tempLow\":\"23℃\",\"tempRange\":\"23℃ ~ 27℃\",\"weather\":\"多云\",\"weatherDescription\":\"气候温暖。\",\"weatherType\":1,\"week\":\"下周三\",\"wind\":\"东风微风\",\"windLevel\":0},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-11-09\",\"dateLong\":1699459200,\"date_for_voice\":\"9号\",\"humidity\":\"79%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/01.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"tempHigh\":\"26℃\",\"tempLow\":\"22℃\",\"tempRange\":\"22℃ ~ 26℃\",\"weather\":\"多云\",\"weatherDescription\":\"气候温暖。\",\"weatherType\":1,\"week\":\"下周四\",\"wind\":\"东风微风\",\"windLevel\":0},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-11-10\",\"dateLong\":1699545600,\"date_for_voice\":\"10号\",\"humidity\":\"74%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/07.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"tempHigh\":\"25℃\",\"tempLow\":\"21℃\",\"tempRange\":\"21℃ ~ 25℃\",\"weather\":\"小雨\",\"weatherDescription\":\"气候温暖。\",\"weatherType\":7,\"week\":\"下周五\",\"wind\":\"东风转东北风微风\",\"windLevel\":0},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-11-11\",\"dateLong\":1699632000,\"date_for_voice\":\"11号\",\"humidity\":\"75%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/02.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"tempHigh\":\"24℃\",\"tempLow\":\"21℃\",\"tempRange\":\"21℃ ~ 24℃\",\"weather\":\"阴\",\"weatherDescription\":\"气候温暖。\",\"weatherType\":2,\"week\":\"下周六\",\"wind\":\"东北风转东风微风\",\"windLevel\":0},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-11-12\",\"dateLong\":1699718400,\"date_for_voice\":\"12号\",\"humidity\":\"70%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/02.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"tempHigh\":\"26℃\",\"tempLow\":\"22℃\",\"tempRange\":\"22℃ ~ 26℃\",\"weather\":\"阴\",\"weatherDescription\":\"气候温暖。\",\"weatherType\":2,\"week\":\"下周日\",\"wind\":\"西风转北风微风\",\"windLevel\":0},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-11-13\",\"dateLong\":1699804800,\"date_for_voice\":\"13号\",\"humidity\":\"69%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/01.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"tempHigh\":\"27℃\",\"tempLow\":\"19℃\",\"tempRange\":\"19℃ ~ 27℃\",\"weather\":\"多云转晴\",\"weatherDescription\":\"气候温暖。\",\"weatherType\":1,\"week\":\"下下周一\",\"wind\":\"东北风3-4级\",\"windLevel\":1},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-11-14\",\"dateLong\":1699891200,\"date_for_voice\":\"14号\",\"humidity\":\"69%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/02.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"tempHigh\":\"25℃\",\"tempLow\":\"19℃\",\"tempRange\":\"19℃ ~ 25℃\",\"weather\":\"阴\",\"weatherDescription\":\"气候温暖。\",\"weatherType\":2,\"week\":\"下下周二\",\"wind\":\"东风微风\",\"windLevel\":0},{\"airData\":-1,\"airQuality\":\"未知\",\"city\":\"深圳市\",\"date\":\"2023-11-15\",\"dateLong\":1699977600,\"date_for_voice\":\"15号\",\"humidity\":\"70%\",\"img\":\"http://cdn9002.iflyos.cn/osweathericon/02.png\",\"lastUpdateTime\":\"2023-11-01 13:00:08\",\"tempHigh\":\"25℃\",\"tempLow\":\"19℃\",\"tempRange\":\"19℃ ~ 25℃\",\"weather\":\"阴\",\"weatherDescription\":\"气候温暖。\",\"weatherType\":2,\"week\":\"下下周三\",\"wind\":\"西风转东北风微风\",\"windLevel\":0}]},\"dialog_stat\":\"DataValid\",\"rc\":0,\"save_history\":true,\"semantic\":[{\"intent\":\"QUERY\",\"slots\":[{\"name\":\"location.city\",\"normValue\":\"CURRENT_CITY\",\"value\":\"CURRENT_CITY\"},{\"name\":\"location.poi\",\"normValue\":\"CURRENT_POI\",\"value\":\"CURRENT_POI\"},{\"name\":\"location.type\",\"normValue\":\"LOC_POI\",\"value\":\"LOC_POI\"},{\"name\":\"queryType\",\"value\":\"内容\"},{\"name\":\"subfocus\",\"value\":\"天气状态\"},{\"name\":\"datetime\",\"normValue\":\"{\\\"datetime\\\":\\\"2023-11-01\\\",\\\"suggestDatetime\\\":\\\"2023-11-01\\\"}\",\"value\":\"今天\"}]}],\"service\":\"weather\",\"shouldEndSession\":\"true\",\"sid\":\"wgw000c91fa@dx18b896cba6ca14e532\",\"state\":{\"fg::weather::default::default\":{\"state\":\"default\"}},\"text\":\"今天天气怎么样？\",\"used_state\":{\"state\":\"default\",\"state_key\":\"fg::weather::default::default\"},\"uuid\":\"wgw000c91fa@dx18b896cba6ca14e532\",\"version\":\"171.0\"}}"
    }
}

如上，通用语义结果json会被当成字段串放在text字段返回。

星火交互认知大模型结果示例如下：

{
     "cbm_meta":{       // 该结果的元信息（描述信息）
        "compress":"raw",
        "encoding":"utf8",
        "format":"plain",
        "seq":0,
        "status":0,
        "text":"{\"cbm_reply\":{\"intent\":1,\"nlp_origin\":\"cbm_semantic\"}}"
    },
    "nlp":{             // 大模型语义结果
        "compress":"raw",
        "encoding":"utf8",
        "format":"plain",
        "seq":0,        // 结果序号，取值：0, 1, 2, ...
        "status":0,     // 结果状态，取值：0, 1, 2, 3
        "text":"今天深圳市天气阴,气温在26°C到33°C之间,空气质量优,"  // 应答文本
    }
}

大模型SDK（使用AIUI v2服务）的nlp结果是流式（会分多次）返回的，每次返回一段结果，客户端需要将这些结果段拼接起来才能得到完整的结果。每个json结果段中有一些字段来标记结果的状态，其中的主要字段：

字段	说明
seq	结果序号，从0开始，取值：0, 1, ...
status	结果状态，取值：0（开始），1（继续），2（结束），3（就一个结果）
text	回复语的片段，需要拼接起来得到完成回复语

一个完整的nlp结果示例：

{
    "cbm_meta":{
        "compress":"raw",
        "encoding":"utf8",
        "format":"plain",
        "seq":0,
        "status":0,
        "text":"{\"cbm_reply\":{\"intent\":0,\"nlp_origin\":\"cbm_semantic\"}}"
    },
    "nlp":{
        "compress":"raw",
        "encoding":"utf8",
        "format":"plain",
        "seq":0,
        "status":0,             // 开始状态
        "text":"今天深圳市天气"
    }
}

{
    "cbm_meta":{
        "compress":"raw",
        "encoding":"utf8",
        "format":"plain",
        "seq":0,
        "status":1,
        "text":"{\"cbm_reply\":{\"intent\":1,\"nlp_origin\":\"cbm_semantic\"}}"
    },
    "nlp":{
        "compress":"raw",
        "encoding":"utf8",
        "format":"plain",
        "seq":1,
        "status":1,             // 继续状态
        "text":"天气预计为"
    }
}

...

{
    "cbm_meta":{
        "compress":"raw",
        "encoding":"utf8",
        "format":"plain",
        "seq":0,
        "status":2,         
        "text":"{\"cbm_reply\":{\"intent\":1,\"nlp_origin\":\"cbm_semantic\"}}"
    },
    "nlp":{
        "compress":"raw",
        "encoding":"utf8",
        "format":"plain",
        "seq":9,
        "status":2,             // 结束状态
        "text":"!"
    }
}

Android结果处理示例代码参考AIUISimpleDemo中的NlpDemoActivity.java文件。

event 事件下发结果示例:

{
    "event": {
        "compress": "",
        "encoding": "",
        "format": "",
        "seq": 0,
        "status": 0,
        "text": "{\"type\":\"Vad\",\"data\":\"\",\"key\":\"Bos\",\"desc\":{}}"
    }
}

json获取text字段获取event事件内容, 解析text json: 当 type == Vad 时:

key == Bos 代表云端VAD检测到了一句话的前端点
key == Eos 代表云端VAD检测到了一句话的后端点

虚拟人下发结果示例如下：

{
    "cbm_vms": {
    "compress": "raw",
    "encoding": "utf8",
    "format": "plain",
    "seq": 0,
    "status": 0,
    "text": "{\"event_type\":\"stream_info\",\"stream_extend\":{\"appid\":\"1000000001\",\"user_sign\":\"Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiIxMDAwMDAwMDAxIiwidGltZSI6MTY0ODAxODQ2MTU0MywiaWF0IjoxNjQ4MTkxMjQyfQ.CTcOh_kCLqvvglo5VLVnjgpZzoFpzk7Un3Et0c9dhUs\"},\"stream_url\":\"xrtcs://xrtc.cn-huabei-1.xf-yun.com/ase00018ca1dx1931a644a68b826152\",\"vmr_status\":0}"
}

json解析text字段, 根据 event_type 的类型做不同的处理逻辑(具体的接入虚拟人调用示例参见 FastInteractionDemoActivity.java):

如果是event_type == stream_info, 此时应该去播虚拟人, 取出里面的appid/user_sign/stream_url参数, 调用 XRTC SDK 来播放虚拟人形象.
如果是 event_type == stop, 则调用XRTC来停止播放虚拟人.
如果是 event_type == pong, 此结果是心跳, 客户端无需处理.
如果是 event_type == driver_status, 此结果是代表的是虚拟人播放状态, 取出里面的vmr_status 参数, 若 vmr_status == 2 则代表本次虚拟人播放完毕(约等于TTS播完了), 客户端无需做操作.

# 3 错误码

错误码	含义	处理方式
0	成功
10106	缺失参数	按错误描述信息补全参数
10024	识别结果为空	检查上传音频中是否有有效语音
10114	连接服务端超时	检查网络状况
10120	结果超时	检查网络状况
10407	传入的appid与SDK库不匹配	使用匹配的appid
10408	登录授权服务异常
11216	按台授权装机量不足	联系商务加装机量授权
11217	已进行sn授权的设备发生uid变更	联系技术支持排查
100000	appid参数授权失败	检查appid参数是否正确
100001	流控限制	联系商务增加qps
100002	会话链接超时	会话完成后60s内新会话请求，服务超时
100003	消息推送失败	消息推送失败
100004	无效请求	无效请求
100005	schema检查错误	根据错误信息描述修改对应参数取值或参数格式
100006	schema检查中数据格式错误	根据错误信息描述修改对应参数取值或参数格式
100007	反序列化失败	根据错误信息描述修改对应参数取值或参数格式
100008	appid参数与鉴权信息应用不匹配	根据错误信息提示修改appid参数取值
100009	请求的header.status参数有问题	按照文档说填写正确参数取值
100010	无效的uid	无效的uid
100011	session服务异常：获取出错	反馈服务商排查
100012	session服务异常：超时	反馈服务商排查
100013	AIUI协议转换异常	反馈服务商排查
100014	AIPaas协议转换异常	反馈服务商排查
100015	Proto序列化失败	反馈服务商排查
100016	Proto返序列化失败	反馈服务商排查
100017	调度服务通信异常：数据发送失败	反馈服务商排查
100018	调度服务通信异常：数据接受失败	反馈服务商排查
100019	调度服务通信异常：数据无效	反馈服务商排查
100020	AIUI结果无效	反馈服务商排查
100021	服务接口出错	反馈服务商排查
100022	session服务异常：无效数据	反馈服务商排查
100023	session服务异常：状态已完成	反馈服务商排查
100024	session服务异常：队列已满	反馈服务商排查
100025	vad服务异常	反馈服务商排查
100026	vad服务异常	反馈服务商排查
100027	vad服务异常	反馈服务商排查
100028	调度服务错误	反馈服务商排查
10010116	情景模式参数配置异常	检查scene取值是否存在、或检查该情景模式下语义配置否切换到了交互大模型
20010108	数据包发送间隔超时	数据包发送间隔超时

在这篇文章中：