Search in sources :

Example 1 with L16PcmEncoder

use of com.amazonaws.mobileconnectors.lex.interactionkit.internal.audio.encoder.L16PcmEncoder in project aws-sdk-android by aws-amplify.

the class InteractionClient method carryOnWithMic.

/**
 * Starts listening for the user to speak, through the microphones. The voice interaction client
 * detects the start and end of speech.
 */
private void carryOnWithMic(final Map<String, String> sessionAttributes, final Map<String, String> requestAttributes, final ResponseType mode) {
    // Ensure that the client is not pre-occupied with another dlalog
    checkBusyState();
    // Send user's response to Amazon Lex service as an audio-stream.
    final InteractionClient client = this;
    new Thread(new Runnable() {

        @Override
        public void run() {
            final Handler handler = new Handler(context.getMainLooper());
            Runnable returnCallBack;
            try {
                // Create a new voice interaction client.
                if (AudioEncoding.LPCM.equals(interactionConfig.getAudioEncoding())) {
                    audioEncoder = new BufferedAudioEncoder(new L16PcmEncoder());
                } else {
                    audioEncoder = new BufferedAudioEncoder(new OpusEncoder());
                }
                // Set time-out limits for mic audio.
                audioTimeouts = new AudioTimeouts(interactionConfig.getNoSpeechTimeoutInterval(), interactionConfig.getMaxSpeechTimeoutInterval());
                // Set VAD configuration.
                vadConfig = new DnnVADConfig(interactionConfig.getLrtThreshold(), interactionConfig.getStartPointingThreshold(), interactionConfig.getEndPointingThreshold());
                lexAudioRecorder = new LexAudioRecorderBuilder(context).audioEncoder(audioEncoder).audioTimeouts(audioTimeouts).dnnVADConfig(vadConfig).build();
                // Calculate the maximum buffer size for pipes.
                final int maxTotalAudioLengthInMills = audioTimeouts.getNoSpeechTimeout() + audioTimeouts.getMaxSpeechTimeout();
                final int pipeSize = AudioRecorder.DEFAULT_SAMPLE_RATE * (int) TimeUnit.MILLISECONDS.toSeconds(maxTotalAudioLengthInMills) * (SAMPLE_SIZE / Byte.SIZE);
                final InputStream audioInStream = new BufferedInputStream(lexAudioRecorder.getConsumerStream(), pipeSize);
                final PostContentRequest request = CreateLexServiceRequest.generatePostContentRequest(sessionAttributes, requestAttributes, interactionConfig, credentialsProvider, mode, audioInStream, audioEncoder.getMediaType().toString());
                // Start the speech listener, service api's will be called only when the speech frames are detected.
                startListening(handler, microphoneListener, lexAudioRecorder, request, client, mode);
            } catch (final Exception e) {
                returnCallBack = new Runnable() {

                    @Override
                    public void run() {
                        interactionListener.onInteractionError(null, e);
                    }
                };
                handler.post(returnCallBack);
            } finally {
                setBusyState(NOT_BUSY);
            }
        }
    }).start();
}
Also used : BufferedAudioEncoder(com.amazonaws.mobileconnectors.lex.interactionkit.internal.audio.encoder.BufferedAudioEncoder) AudioTimeouts(com.amazonaws.mobileconnectors.lex.interactionkit.internal.audio.AudioTimeouts) PostContentRequest(com.amazonaws.services.lexrts.model.PostContentRequest) BufferedInputStream(java.io.BufferedInputStream) FileInputStream(java.io.FileInputStream) InputStream(java.io.InputStream) Handler(android.os.Handler) L16PcmEncoder(com.amazonaws.mobileconnectors.lex.interactionkit.internal.audio.encoder.L16PcmEncoder) DnnVADConfig(com.amazonaws.mobileconnectors.lex.interactionkit.internal.vad.config.DnnVADConfig) InvalidParameterException(com.amazonaws.mobileconnectors.lex.interactionkit.exceptions.InvalidParameterException) LexClientException(com.amazonaws.mobileconnectors.lex.interactionkit.exceptions.LexClientException) MaxSpeechTimeOutException(com.amazonaws.mobileconnectors.lex.interactionkit.exceptions.MaxSpeechTimeOutException) DialogFailedException(com.amazonaws.mobileconnectors.lex.interactionkit.exceptions.DialogFailedException) NoSpeechTimeOutException(com.amazonaws.mobileconnectors.lex.interactionkit.exceptions.NoSpeechTimeOutException) AmazonClientException(com.amazonaws.AmazonClientException) AudioPlaybackException(com.amazonaws.mobileconnectors.lex.interactionkit.exceptions.AudioPlaybackException) OpusEncoder(com.amazon.blueshift.bluefront.android.audio.encoder.OpusEncoder) BufferedInputStream(java.io.BufferedInputStream) LexAudioRecorderBuilder(com.amazonaws.mobileconnectors.lex.interactionkit.internal.audio.LexAudioRecorderBuilder)

Example 2 with L16PcmEncoder

use of com.amazonaws.mobileconnectors.lex.interactionkit.internal.audio.encoder.L16PcmEncoder in project aws-sdk-android by aws-amplify.

the class AudioRecorder method startRecording.

/**
 * Method to setup, start the recorder, and read data.
 * When recorder is stopped by the user, clean up resources.
 * @throws Exception when there are problems while recording audio.
 */
public void startRecording() throws Exception {
    startAudioRecorder();
    final short[] buffer = new short[mNumSamplesPerRead];
    int numSamplesRead;
    final AudioSourceListener listener = getAudioSourceListener();
    final AudioEncoder pcmEncoder = new L16PcmEncoder();
    try {
        Log.v(TAG, "Starting record loop");
        while (isInValidStateToContinueRecording()) {
            // Make sure recorder is not null before recording.
            if (mRecord == null) {
                Log.e(TAG, "Recorder is null.");
                throw new AudioSourceException("Recorder null");
            }
            // Buffer bytes to be sent to callback.
            synchronized (mRecord) {
                numSamplesRead = mRecord.read(buffer, 0, mNumSamplesPerRead);
            }
            final int invalidOperation = AudioRecord.ERROR_INVALID_OPERATION;
            if (invalidOperation != numSamplesRead) {
                setPostRecordingFields();
                if (numSamplesRead > 0) {
                    // Prepare samples for the callback.
                    final byte[] callbackBuffer = pcmEncoder.encode(buffer, numSamplesRead);
                    listener.onBufferReceived(callbackBuffer);
                    updateSumSamplesForRMSCalculations(numSamplesRead, buffer);
                }
                postAudioRecordingProcessing(numSamplesRead, buffer, listener);
            } else {
                Log.v(TAG, "AudioRecord - Invalid Operation");
                throw new AudioSourceException("AudioRecord - Invalid Operation");
            }
        }
        Log.v(TAG, "Finished record loop");
    } finally {
        cleanUpAfterRecording();
    }
}
Also used : AudioEncoder(com.amazonaws.mobileconnectors.lex.interactionkit.internal.audio.encoder.AudioEncoder) L16PcmEncoder(com.amazonaws.mobileconnectors.lex.interactionkit.internal.audio.encoder.L16PcmEncoder)

Aggregations

L16PcmEncoder (com.amazonaws.mobileconnectors.lex.interactionkit.internal.audio.encoder.L16PcmEncoder)2 Handler (android.os.Handler)1 OpusEncoder (com.amazon.blueshift.bluefront.android.audio.encoder.OpusEncoder)1 AmazonClientException (com.amazonaws.AmazonClientException)1 AudioPlaybackException (com.amazonaws.mobileconnectors.lex.interactionkit.exceptions.AudioPlaybackException)1 DialogFailedException (com.amazonaws.mobileconnectors.lex.interactionkit.exceptions.DialogFailedException)1 InvalidParameterException (com.amazonaws.mobileconnectors.lex.interactionkit.exceptions.InvalidParameterException)1 LexClientException (com.amazonaws.mobileconnectors.lex.interactionkit.exceptions.LexClientException)1 MaxSpeechTimeOutException (com.amazonaws.mobileconnectors.lex.interactionkit.exceptions.MaxSpeechTimeOutException)1 NoSpeechTimeOutException (com.amazonaws.mobileconnectors.lex.interactionkit.exceptions.NoSpeechTimeOutException)1 AudioTimeouts (com.amazonaws.mobileconnectors.lex.interactionkit.internal.audio.AudioTimeouts)1 LexAudioRecorderBuilder (com.amazonaws.mobileconnectors.lex.interactionkit.internal.audio.LexAudioRecorderBuilder)1 AudioEncoder (com.amazonaws.mobileconnectors.lex.interactionkit.internal.audio.encoder.AudioEncoder)1 BufferedAudioEncoder (com.amazonaws.mobileconnectors.lex.interactionkit.internal.audio.encoder.BufferedAudioEncoder)1 DnnVADConfig (com.amazonaws.mobileconnectors.lex.interactionkit.internal.vad.config.DnnVADConfig)1 PostContentRequest (com.amazonaws.services.lexrts.model.PostContentRequest)1 BufferedInputStream (java.io.BufferedInputStream)1 FileInputStream (java.io.FileInputStream)1 InputStream (java.io.InputStream)1