淘先锋技术网

首页 1 2 3 4 5 6 7

freeswitch源码unimrcp中asr的实现

通过学习回顾freeswitch源码，来完善我们对音频处理，对话的理解

我们之前有说过，我们通过fs的media bug形式进行录音的监听，对话监听，做实时语音处理。当然除了使用mediabug还可以使用unimcrp形式进行对接。 这句话不够准确，理论上unimrcp也是使用了media bug进行音频流监听。

我们简单分析一下，unimrcp源码，我们发现 recog_load 方法里面

asr_interface->asr_feed = recog_asr_feed;

初始化了asr_feed， 同样是调用了

static switch_status_t recog_asr_feed(switch_asr_handle_t *ah, void *data, unsigned int len, switch_asr_flag_t *flags)
{
	switch_size_t slen = len;
	speech_channel_t *schannel = (speech_channel_t *) ah->private_info;
	return speech_channel_write(schannel, data, &slen);
}

这里我们看见，speech_channel_write 数据data，那么一定是有函数调用了recog_asr_feed，即asr_interface->asr_feed，将音频流数据传入了。

顺腾摸瓜，找到了调用者--switch_core_asr.c 文件的 switch_core_asr_feed

SWITCH_DECLARE(switch_status_t) switch_core_asr_feed(switch_asr_handle_t *ah, void *data, unsigned int len, switch_asr_flag_t *flags)
{
	switch_size_t orig_len = len;
	switch_assert(ah != NULL);

	if (ah->native_rate && ah->samplerate && ah->native_rate != ah->samplerate) {
		if (!ah->resampler) {
			if (switch_resample_create(&ah->resampler,
									   ah->samplerate, ah->native_rate, (uint32_t) orig_len, SWITCH_RESAMPLE_QUALITY, 1) != SWITCH_STATUS_SUCCESS) {
				switch_log_printf(SWITCH_CHANNEL_LOG, SWITCH_LOG_CRIT, "Unable to create resampler!\n");
				return SWITCH_STATUS_GENERR;
			}
		}

		switch_resample_process(ah->resampler, data, len / 2);
		if (ah->resampler->to_len > orig_len) {
			if (!ah->dbuf) {
				void *mem;
				ah->dbuflen = ah->resampler->to_len * 2;
				mem = realloc(ah->dbuf, ah->dbuflen);
				switch_assert(mem);
				ah->dbuf = mem;
			}
			switch_assert(ah->resampler->to_len * 2 <= ah->dbuflen);
			memcpy(ah->dbuf, ah->resampler->to, ah->resampler->to_len * 2);
			data = ah->dbuf;
		} else {
			memcpy(data, ah->resampler->to, ah->resampler->to_len * 2);
		}

		len = ah->resampler->to_len;
	}

	return ah->asr_interface->asr_feed(ah, data, len, flags);
}

先处理resample，然后调用了asr_feed，再顺腾摸瓜就找到了今日的主角文件，switch_ivr_async.c 文件。

这就是media的回调，当出现SWITCH_ABC_TYPE_READ 时候进行了 switch_core_asr_feed调用。

一切明朗起来。media bug牛逼

罪魁祸首还是 switch_core_media_bug_add, 索然无味。

switch_ivr_detect_speech， switch_ivr_play_and_detect_speech

最终到了：mod_dptools.c 一切真香大白：

SWITCH_STANDARD_APP(play_and_detect_speech_function)

SWITCH_ADD_APP(app_interface, "play_and_detect_speech", "Play and do speech recognition", "Play and do speech recognition", play_and_detect_speech_function, PLAY_AND_DETECT_SPEECH_SYNTAX, SAF_NONE);

play_and_detect_speech 这个app，那么play_and_detect_speech具体做了什么？我们下一篇进行分析