Search in sources :

Example 6 with PredictionsException

use of com.amplifyframework.predictions.PredictionsException in project amplify-android by aws-amplify.

the class AWSRekognitionService method detectEntityMatches.

private List<EntityMatch> detectEntityMatches(ByteBuffer imageData, int maxEntities, String collectionId) throws PredictionsException {
    SearchFacesByImageRequest request = new SearchFacesByImageRequest().withImage(new Image().withBytes(imageData)).withMaxFaces(maxEntities).withCollectionId(collectionId);
    // Detect entities in the given image by matching against a collection of images
    final SearchFacesByImageResult result;
    try {
        result = rekognition.searchFacesByImage(request);
    } catch (AmazonClientException serviceException) {
        throw new PredictionsException("Amazon Rekognition encountered an error while searching for known faces.", serviceException, "See attached service exception for more details.");
    }
    List<EntityMatch> matches = new ArrayList<>();
    for (FaceMatch rekognitionMatch : result.getFaceMatches()) {
        Face face = rekognitionMatch.getFace();
        RectF box = RekognitionResultTransformers.fromBoundingBox(face.getBoundingBox());
        EntityMatch amplifyMatch = EntityMatch.builder().externalImageId(face.getExternalImageId()).confidence(rekognitionMatch.getSimilarity()).box(box).build();
        matches.add(amplifyMatch);
    }
    return matches;
}
Also used : RectF(android.graphics.RectF) SearchFacesByImageRequest(com.amazonaws.services.rekognition.model.SearchFacesByImageRequest) AmazonClientException(com.amazonaws.AmazonClientException) ArrayList(java.util.ArrayList) PredictionsException(com.amplifyframework.predictions.PredictionsException) EntityMatch(com.amplifyframework.predictions.models.EntityMatch) Image(com.amazonaws.services.rekognition.model.Image) SearchFacesByImageResult(com.amazonaws.services.rekognition.model.SearchFacesByImageResult) ComparedFace(com.amazonaws.services.rekognition.model.ComparedFace) Face(com.amazonaws.services.rekognition.model.Face) FaceMatch(com.amazonaws.services.rekognition.model.FaceMatch)

Example 7 with PredictionsException

use of com.amplifyframework.predictions.PredictionsException in project amplify-android by aws-amplify.

the class AWSRekognitionService method recognizeCelebrities.

void recognizeCelebrities(@NonNull ByteBuffer imageData, @NonNull Consumer<IdentifyResult> onSuccess, @NonNull Consumer<PredictionsException> onError) {
    final IdentifyEntitiesConfiguration config;
    try {
        config = pluginConfiguration.getIdentifyEntitiesConfiguration();
        if (config.isCelebrityDetectionEnabled()) {
            List<CelebrityDetails> celebrities = detectCelebrities(imageData);
            onSuccess.accept(IdentifyCelebritiesResult.fromCelebrities(celebrities));
        } else {
            onError.accept(new PredictionsException("Celebrity detection is disabled.", "Please enable celebrity detection via Amplify CLI. This feature " + "should be accessible by running `amplify update predictions` " + "in the console and updating entities detection resource with " + "advanced configuration setting."));
        }
    } catch (PredictionsException exception) {
        onError.accept(exception);
    }
}
Also used : IdentifyEntitiesConfiguration(com.amplifyframework.predictions.aws.configuration.IdentifyEntitiesConfiguration) CelebrityDetails(com.amplifyframework.predictions.models.CelebrityDetails) PredictionsException(com.amplifyframework.predictions.PredictionsException)

Example 8 with PredictionsException

use of com.amplifyframework.predictions.PredictionsException in project amplify-android by aws-amplify.

the class AWSRekognitionService method detectCelebrities.

private List<CelebrityDetails> detectCelebrities(ByteBuffer imageData) throws PredictionsException {
    RecognizeCelebritiesRequest request = new RecognizeCelebritiesRequest().withImage(new Image().withBytes(imageData));
    // Recognize celebrities in the given image via Amazon Rekognition
    final RecognizeCelebritiesResult result;
    try {
        result = rekognition.recognizeCelebrities(request);
    } catch (AmazonClientException serviceException) {
        throw new PredictionsException("Amazon Rekognition encountered an error while recognizing celebrities.", serviceException, "See attached service exception for more details.");
    }
    List<CelebrityDetails> celebrities = new ArrayList<>();
    for (com.amazonaws.services.rekognition.model.Celebrity rekognitionCelebrity : result.getCelebrityFaces()) {
        Celebrity amplifyCelebrity = Celebrity.builder().id(rekognitionCelebrity.getId()).value(rekognitionCelebrity.getName()).confidence(rekognitionCelebrity.getMatchConfidence()).build();
        // Get face-specific celebrity details from the result
        ComparedFace face = rekognitionCelebrity.getFace();
        RectF box = RekognitionResultTransformers.fromBoundingBox(face.getBoundingBox());
        Pose pose = RekognitionResultTransformers.fromRekognitionPose(face.getPose());
        List<Landmark> landmarks = RekognitionResultTransformers.fromLandmarks(face.getLandmarks());
        // Get URL links that are relevant to celebrities
        List<URL> urls = new ArrayList<>();
        for (String url : rekognitionCelebrity.getUrls()) {
            try {
                urls.add(new URL(url));
            } catch (MalformedURLException badUrl) {
            // Ignore bad URL
            }
        }
        CelebrityDetails details = CelebrityDetails.builder().celebrity(amplifyCelebrity).box(box).pose(pose).landmarks(landmarks).urls(urls).build();
        celebrities.add(details);
    }
    return celebrities;
}
Also used : MalformedURLException(java.net.MalformedURLException) CelebrityDetails(com.amplifyframework.predictions.models.CelebrityDetails) AmazonClientException(com.amazonaws.AmazonClientException) Landmark(com.amplifyframework.predictions.models.Landmark) ArrayList(java.util.ArrayList) Image(com.amazonaws.services.rekognition.model.Image) ComparedFace(com.amazonaws.services.rekognition.model.ComparedFace) URL(java.net.URL) RectF(android.graphics.RectF) RecognizeCelebritiesResult(com.amazonaws.services.rekognition.model.RecognizeCelebritiesResult) RecognizeCelebritiesRequest(com.amazonaws.services.rekognition.model.RecognizeCelebritiesRequest) Celebrity(com.amplifyframework.predictions.models.Celebrity) Pose(com.amplifyframework.predictions.models.Pose) PredictionsException(com.amplifyframework.predictions.PredictionsException)

Example 9 with PredictionsException

use of com.amplifyframework.predictions.PredictionsException in project amplify-android by aws-amplify.

the class AWSRekognitionService method detectEntities.

private List<EntityDetails> detectEntities(ByteBuffer imageData) throws PredictionsException {
    DetectFacesRequest request = new DetectFacesRequest().withImage(new Image().withBytes(imageData)).withAttributes(Attribute.ALL.toString());
    // Detect entities in the given image via Amazon Rekognition
    final DetectFacesResult result;
    try {
        result = rekognition.detectFaces(request);
    } catch (AmazonClientException serviceException) {
        throw new PredictionsException("Amazon Rekognition encountered an error while detecting faces.", serviceException, "See attached service exception for more details.");
    }
    List<EntityDetails> entities = new ArrayList<>();
    for (FaceDetail face : result.getFaceDetails()) {
        // Extract details from face detection
        RectF box = RekognitionResultTransformers.fromBoundingBox(face.getBoundingBox());
        AgeRange ageRange = RekognitionResultTransformers.fromRekognitionAgeRange(face.getAgeRange());
        Pose pose = RekognitionResultTransformers.fromRekognitionPose(face.getPose());
        List<Landmark> landmarks = RekognitionResultTransformers.fromLandmarks(face.getLandmarks());
        List<BinaryFeature> features = RekognitionResultTransformers.fromFaceDetail(face);
        // Gender detection
        com.amazonaws.services.rekognition.model.Gender rekognitionGender = face.getGender();
        Gender amplifyGender = Gender.builder().value(GenderBinaryTypeAdapter.fromRekognition(rekognitionGender.getValue())).confidence(rekognitionGender.getConfidence()).build();
        // Emotion detection
        List<Emotion> emotions = new ArrayList<>();
        for (com.amazonaws.services.rekognition.model.Emotion rekognitionEmotion : face.getEmotions()) {
            EmotionType emotion = EmotionTypeAdapter.fromRekognition(rekognitionEmotion.getType());
            Emotion amplifyEmotion = Emotion.builder().value(emotion).confidence(rekognitionEmotion.getConfidence()).build();
            emotions.add(amplifyEmotion);
        }
        Collections.sort(emotions, Collections.reverseOrder());
        EntityDetails entity = EntityDetails.builder().box(box).ageRange(ageRange).pose(pose).gender(amplifyGender).landmarks(landmarks).emotions(emotions).features(features).build();
        entities.add(entity);
    }
    return entities;
}
Also used : AgeRange(com.amplifyframework.predictions.models.AgeRange) AmazonClientException(com.amazonaws.AmazonClientException) ArrayList(java.util.ArrayList) Gender(com.amplifyframework.predictions.models.Gender) Image(com.amazonaws.services.rekognition.model.Image) FaceDetail(com.amazonaws.services.rekognition.model.FaceDetail) EntityDetails(com.amplifyframework.predictions.models.EntityDetails) Pose(com.amplifyframework.predictions.models.Pose) PredictionsException(com.amplifyframework.predictions.PredictionsException) Emotion(com.amplifyframework.predictions.models.Emotion) Landmark(com.amplifyframework.predictions.models.Landmark) BinaryFeature(com.amplifyframework.predictions.models.BinaryFeature) RectF(android.graphics.RectF) EmotionType(com.amplifyframework.predictions.models.EmotionType) DetectFacesResult(com.amazonaws.services.rekognition.model.DetectFacesResult) DetectFacesRequest(com.amazonaws.services.rekognition.model.DetectFacesRequest)

Example 10 with PredictionsException

use of com.amplifyframework.predictions.PredictionsException in project amplify-android by aws-amplify.

the class TextClassificationDictionary method loadDictionaryFile.

// This code comes from the official TensorFlow Lite sample app
// https://github.com/tensorflow/examples/tree/master/lite/examples/text_classification/android
private void loadDictionaryFile() throws PredictionsException {
    try (InputStream inputStream = assets.open(DICTIONARY_PATH);
        BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream))) {
        // First column is a word, and the second is the index of this word.
        while (reader.ready()) {
            List<String> line = Arrays.asList(reader.readLine().split(" "));
            if (line.size() < 2) {
                continue;
            }
            dictionary.put(line.get(0), Integer.parseInt(line.get(1)));
        }
    } catch (IOException exception) {
        throw new PredictionsException("Error encountered while loading dictionary.", exception, "Verify that " + DICTIONARY_PATH + " is present inside the assets directory.");
    } catch (IllegalArgumentException exception) {
        throw new PredictionsException("Loaded dictionary does not contain required keywords.", exception, "Verify the validity of the dictionary asset.");
    }
    // Make sure that the reserved words are there
    final String message;
    if (dictionary.get(PAD) == null) {
        message = "Reserved word for padding \"<PAD>\" not found.";
    } else if (dictionary.get(START) == null) {
        message = "Reserved word for start indicator \"<START>\" not found.";
    } else if (dictionary.get(UNKNOWN) == null) {
        message = "Reserved word for unknown word \"<UNKNOWN>\" not found.";
    } else {
        return;
    }
    throw new PredictionsException(message, "Verify the validity of the dictionary asset.");
}
Also used : InputStreamReader(java.io.InputStreamReader) InputStream(java.io.InputStream) BufferedReader(java.io.BufferedReader) PredictionsException(com.amplifyframework.predictions.PredictionsException) IOException(java.io.IOException)

Aggregations

PredictionsException (com.amplifyframework.predictions.PredictionsException)32 AmazonClientException (com.amazonaws.AmazonClientException)15 ArrayList (java.util.ArrayList)9 Image (com.amazonaws.services.rekognition.model.Image)6 Test (org.junit.Test)5 RectF (android.graphics.RectF)4 IdentifyEntitiesConfiguration (com.amplifyframework.predictions.aws.configuration.IdentifyEntitiesConfiguration)3 Sentiment (com.amplifyframework.predictions.models.Sentiment)3 InputStream (java.io.InputStream)3 DominantLanguage (com.amazonaws.services.comprehend.model.DominantLanguage)2 ComparedFace (com.amazonaws.services.rekognition.model.ComparedFace)2 ModerationLabel (com.amazonaws.services.rekognition.model.ModerationLabel)2 Document (com.amazonaws.services.textract.model.Document)2 SpeechGeneratorConfiguration (com.amplifyframework.predictions.aws.configuration.SpeechGeneratorConfiguration)2 CelebrityDetails (com.amplifyframework.predictions.models.CelebrityDetails)2 KeyPhrase (com.amplifyframework.predictions.models.KeyPhrase)2 Landmark (com.amplifyframework.predictions.models.Landmark)2 LanguageType (com.amplifyframework.predictions.models.LanguageType)2 Pose (com.amplifyframework.predictions.models.Pose)2 Syntax (com.amplifyframework.predictions.models.Syntax)2