Search in sources :

Example 6 with EntityProfile

use of org.opensearch.ad.model.EntityProfile in project anomaly-detection by opensearch-project.

the class EntityProfileRunner method profile.

/**
 * Get profile info of specific entity.
 *
 * @param detectorId detector identifier
 * @param entityValue entity value
 * @param profilesToCollect profiles to collect
 * @param listener action listener to handle exception and process entity profile response
 */
public void profile(String detectorId, Entity entityValue, Set<EntityProfileName> profilesToCollect, ActionListener<EntityProfile> listener) {
    if (profilesToCollect == null || profilesToCollect.size() == 0) {
        listener.onFailure(new IllegalArgumentException(CommonErrorMessages.EMPTY_PROFILES_COLLECT));
        return;
    }
    GetRequest getDetectorRequest = new GetRequest(ANOMALY_DETECTORS_INDEX, detectorId);
    client.get(getDetectorRequest, ActionListener.wrap(getResponse -> {
        if (getResponse != null && getResponse.isExists()) {
            try (XContentParser parser = XContentType.JSON.xContent().createParser(xContentRegistry, LoggingDeprecationHandler.INSTANCE, getResponse.getSourceAsString())) {
                ensureExpectedToken(XContentParser.Token.START_OBJECT, parser.nextToken(), parser);
                AnomalyDetector detector = AnomalyDetector.parse(parser, detectorId);
                List<String> categoryFields = detector.getCategoryField();
                int maxCategoryFields = NumericSetting.maxCategoricalFields();
                if (categoryFields == null || categoryFields.size() == 0) {
                    listener.onFailure(new IllegalArgumentException(NOT_HC_DETECTOR_ERR_MSG));
                } else if (categoryFields.size() > maxCategoryFields) {
                    listener.onFailure(new IllegalArgumentException(CommonErrorMessages.getTooManyCategoricalFieldErr(maxCategoryFields)));
                } else {
                    validateEntity(entityValue, categoryFields, detectorId, profilesToCollect, detector, listener);
                }
            } catch (Exception t) {
                listener.onFailure(t);
            }
        } else {
            listener.onFailure(new IllegalArgumentException(CommonErrorMessages.FAIL_TO_FIND_DETECTOR_MSG + detectorId));
        }
    }, listener::onFailure));
}
Also used : EntityProfileRequest(org.opensearch.ad.transport.EntityProfileRequest) EntityProfileName(org.opensearch.ad.model.EntityProfileName) XContentParser(org.opensearch.common.xcontent.XContentParser) ANOMALY_DETECTOR_JOB_INDEX(org.opensearch.ad.model.AnomalyDetectorJob.ANOMALY_DETECTOR_JOB_INDEX) EntityProfileResponse(org.opensearch.ad.transport.EntityProfileResponse) Map(java.util.Map) AnomalyDetector(org.opensearch.ad.model.AnomalyDetector) SearchRequest(org.opensearch.action.search.SearchRequest) ActionListener(org.opensearch.action.ActionListener) EntityProfile(org.opensearch.ad.model.EntityProfile) NestedQueryBuilder(org.opensearch.index.query.NestedQueryBuilder) QueryBuilders(org.opensearch.index.query.QueryBuilders) Client(org.opensearch.client.Client) CommonName(org.opensearch.ad.constant.CommonName) NumericSetting(org.opensearch.ad.settings.NumericSetting) Preference(org.opensearch.cluster.routing.Preference) InitProgressProfile(org.opensearch.ad.model.InitProgressProfile) IndexNotFoundException(org.opensearch.index.IndexNotFoundException) ANOMALY_DETECTORS_INDEX(org.opensearch.ad.model.AnomalyDetector.ANOMALY_DETECTORS_INDEX) GetRequest(org.opensearch.action.get.GetRequest) LoggingDeprecationHandler(org.opensearch.common.xcontent.LoggingDeprecationHandler) Set(java.util.Set) TermQueryBuilder(org.opensearch.index.query.TermQueryBuilder) XContentParserUtils.ensureExpectedToken(org.opensearch.common.xcontent.XContentParserUtils.ensureExpectedToken) ScoreMode(org.apache.lucene.search.join.ScoreMode) MultiResponsesDelegateActionListener(org.opensearch.ad.util.MultiResponsesDelegateActionListener) AggregationBuilders(org.opensearch.search.aggregations.AggregationBuilders) List(java.util.List) CommonErrorMessages(org.opensearch.ad.constant.CommonErrorMessages) Logger(org.apache.logging.log4j.Logger) AnomalyResult(org.opensearch.ad.model.AnomalyResult) Entity(org.opensearch.ad.model.Entity) EntityState(org.opensearch.ad.model.EntityState) SearchSourceBuilder(org.opensearch.search.builder.SearchSourceBuilder) AnomalyDetectorJob(org.opensearch.ad.model.AnomalyDetectorJob) NamedXContentRegistry(org.opensearch.common.xcontent.NamedXContentRegistry) Optional(java.util.Optional) XContentType(org.opensearch.common.xcontent.XContentType) IntervalTimeConfiguration(org.opensearch.ad.model.IntervalTimeConfiguration) LogManager(org.apache.logging.log4j.LogManager) EntityProfileAction(org.opensearch.ad.transport.EntityProfileAction) ParseUtils(org.opensearch.ad.util.ParseUtils) BoolQueryBuilder(org.opensearch.index.query.BoolQueryBuilder) GetRequest(org.opensearch.action.get.GetRequest) List(java.util.List) XContentParser(org.opensearch.common.xcontent.XContentParser) AnomalyDetector(org.opensearch.ad.model.AnomalyDetector) IndexNotFoundException(org.opensearch.index.IndexNotFoundException)

Example 7 with EntityProfile

use of org.opensearch.ad.model.EntityProfile in project anomaly-detection by opensearch-project.

the class EntityProfileRunner method getJob.

private void getJob(String detectorId, Entity entityValue, Set<EntityProfileName> profilesToCollect, AnomalyDetector detector, EntityProfileResponse entityProfileResponse, ActionListener<EntityProfile> listener) {
    GetRequest getRequest = new GetRequest(ANOMALY_DETECTOR_JOB_INDEX, detectorId);
    client.get(getRequest, ActionListener.wrap(getResponse -> {
        if (getResponse != null && getResponse.isExists()) {
            try (XContentParser parser = XContentType.JSON.xContent().createParser(xContentRegistry, LoggingDeprecationHandler.INSTANCE, getResponse.getSourceAsString())) {
                ensureExpectedToken(XContentParser.Token.START_OBJECT, parser.nextToken(), parser);
                AnomalyDetectorJob job = AnomalyDetectorJob.parse(parser);
                int totalResponsesToWait = 0;
                if (profilesToCollect.contains(EntityProfileName.INIT_PROGRESS) || profilesToCollect.contains(EntityProfileName.STATE)) {
                    totalResponsesToWait++;
                }
                if (profilesToCollect.contains(EntityProfileName.ENTITY_INFO)) {
                    totalResponsesToWait++;
                }
                if (profilesToCollect.contains(EntityProfileName.MODELS)) {
                    totalResponsesToWait++;
                }
                MultiResponsesDelegateActionListener<EntityProfile> delegateListener = new MultiResponsesDelegateActionListener<EntityProfile>(listener, totalResponsesToWait, CommonErrorMessages.FAIL_FETCH_ERR_MSG + entityValue + " of detector " + detectorId, false);
                if (profilesToCollect.contains(EntityProfileName.MODELS)) {
                    EntityProfile.Builder builder = new EntityProfile.Builder();
                    if (false == job.isEnabled()) {
                        delegateListener.onResponse(builder.build());
                    } else {
                        delegateListener.onResponse(builder.modelProfile(entityProfileResponse.getModelProfile()).build());
                    }
                }
                if (profilesToCollect.contains(EntityProfileName.INIT_PROGRESS) || profilesToCollect.contains(EntityProfileName.STATE)) {
                    profileStateRelated(entityProfileResponse.getTotalUpdates(), detectorId, entityValue, profilesToCollect, detector, job, delegateListener);
                }
                if (profilesToCollect.contains(EntityProfileName.ENTITY_INFO)) {
                    long enabledTimeMs = job.getEnabledTime().toEpochMilli();
                    SearchRequest lastSampleTimeRequest = createLastSampleTimeRequest(detectorId, enabledTimeMs, entityValue, detector.getResultIndex());
                    EntityProfile.Builder builder = new EntityProfile.Builder();
                    Optional<Boolean> isActiveOp = entityProfileResponse.isActive();
                    if (isActiveOp.isPresent()) {
                        builder.isActive(isActiveOp.get());
                    }
                    builder.lastActiveTimestampMs(entityProfileResponse.getLastActiveMs());
                    client.search(lastSampleTimeRequest, ActionListener.wrap(searchResponse -> {
                        Optional<Long> latestSampleTimeMs = ParseUtils.getLatestDataTime(searchResponse);
                        if (latestSampleTimeMs.isPresent()) {
                            builder.lastSampleTimestampMs(latestSampleTimeMs.get());
                        }
                        delegateListener.onResponse(builder.build());
                    }, exception -> {
                        // sth wrong like result index not created. Return what we have
                        if (exception instanceof IndexNotFoundException) {
                            // don't print out stack trace since it is not helpful
                            logger.info("Result index hasn't been created", exception.getMessage());
                        } else {
                            logger.warn("fail to get last sample time", exception);
                        }
                        delegateListener.onResponse(builder.build());
                    }));
                }
            } catch (Exception e) {
                logger.error(CommonErrorMessages.FAIL_TO_GET_PROFILE_MSG, e);
                listener.onFailure(e);
            }
        } else {
            sendUnknownState(profilesToCollect, entityValue, true, listener);
        }
    }, exception -> {
        if (exception instanceof IndexNotFoundException) {
            logger.info(exception.getMessage());
            sendUnknownState(profilesToCollect, entityValue, true, listener);
        } else {
            logger.error(CommonErrorMessages.FAIL_TO_GET_PROFILE_MSG + detectorId, exception);
            listener.onFailure(exception);
        }
    }));
}
Also used : EntityProfileRequest(org.opensearch.ad.transport.EntityProfileRequest) EntityProfileName(org.opensearch.ad.model.EntityProfileName) XContentParser(org.opensearch.common.xcontent.XContentParser) ANOMALY_DETECTOR_JOB_INDEX(org.opensearch.ad.model.AnomalyDetectorJob.ANOMALY_DETECTOR_JOB_INDEX) EntityProfileResponse(org.opensearch.ad.transport.EntityProfileResponse) Map(java.util.Map) AnomalyDetector(org.opensearch.ad.model.AnomalyDetector) SearchRequest(org.opensearch.action.search.SearchRequest) ActionListener(org.opensearch.action.ActionListener) EntityProfile(org.opensearch.ad.model.EntityProfile) NestedQueryBuilder(org.opensearch.index.query.NestedQueryBuilder) QueryBuilders(org.opensearch.index.query.QueryBuilders) Client(org.opensearch.client.Client) CommonName(org.opensearch.ad.constant.CommonName) NumericSetting(org.opensearch.ad.settings.NumericSetting) Preference(org.opensearch.cluster.routing.Preference) InitProgressProfile(org.opensearch.ad.model.InitProgressProfile) IndexNotFoundException(org.opensearch.index.IndexNotFoundException) ANOMALY_DETECTORS_INDEX(org.opensearch.ad.model.AnomalyDetector.ANOMALY_DETECTORS_INDEX) GetRequest(org.opensearch.action.get.GetRequest) LoggingDeprecationHandler(org.opensearch.common.xcontent.LoggingDeprecationHandler) Set(java.util.Set) TermQueryBuilder(org.opensearch.index.query.TermQueryBuilder) XContentParserUtils.ensureExpectedToken(org.opensearch.common.xcontent.XContentParserUtils.ensureExpectedToken) ScoreMode(org.apache.lucene.search.join.ScoreMode) MultiResponsesDelegateActionListener(org.opensearch.ad.util.MultiResponsesDelegateActionListener) AggregationBuilders(org.opensearch.search.aggregations.AggregationBuilders) List(java.util.List) CommonErrorMessages(org.opensearch.ad.constant.CommonErrorMessages) Logger(org.apache.logging.log4j.Logger) AnomalyResult(org.opensearch.ad.model.AnomalyResult) Entity(org.opensearch.ad.model.Entity) EntityState(org.opensearch.ad.model.EntityState) SearchSourceBuilder(org.opensearch.search.builder.SearchSourceBuilder) AnomalyDetectorJob(org.opensearch.ad.model.AnomalyDetectorJob) NamedXContentRegistry(org.opensearch.common.xcontent.NamedXContentRegistry) Optional(java.util.Optional) XContentType(org.opensearch.common.xcontent.XContentType) IntervalTimeConfiguration(org.opensearch.ad.model.IntervalTimeConfiguration) LogManager(org.apache.logging.log4j.LogManager) EntityProfileAction(org.opensearch.ad.transport.EntityProfileAction) ParseUtils(org.opensearch.ad.util.ParseUtils) BoolQueryBuilder(org.opensearch.index.query.BoolQueryBuilder) SearchRequest(org.opensearch.action.search.SearchRequest) Optional(java.util.Optional) NestedQueryBuilder(org.opensearch.index.query.NestedQueryBuilder) TermQueryBuilder(org.opensearch.index.query.TermQueryBuilder) SearchSourceBuilder(org.opensearch.search.builder.SearchSourceBuilder) BoolQueryBuilder(org.opensearch.index.query.BoolQueryBuilder) MultiResponsesDelegateActionListener(org.opensearch.ad.util.MultiResponsesDelegateActionListener) AnomalyDetectorJob(org.opensearch.ad.model.AnomalyDetectorJob) IndexNotFoundException(org.opensearch.index.IndexNotFoundException) GetRequest(org.opensearch.action.get.GetRequest) IndexNotFoundException(org.opensearch.index.IndexNotFoundException) EntityProfile(org.opensearch.ad.model.EntityProfile) XContentParser(org.opensearch.common.xcontent.XContentParser)

Example 8 with EntityProfile

use of org.opensearch.ad.model.EntityProfile in project anomaly-detection by opensearch-project.

the class EntityProfileRunner method sendInitState.

private void sendInitState(Set<EntityProfileName> profilesToCollect, Entity entityValue, AnomalyDetector detector, long updates, MultiResponsesDelegateActionListener<EntityProfile> delegateListener) {
    EntityProfile.Builder builder = new EntityProfile.Builder();
    if (profilesToCollect.contains(EntityProfileName.STATE)) {
        builder.state(EntityState.INIT);
    }
    if (profilesToCollect.contains(EntityProfileName.INIT_PROGRESS)) {
        long intervalMins = ((IntervalTimeConfiguration) detector.getDetectionInterval()).toDuration().toMinutes();
        InitProgressProfile initProgress = computeInitProgressProfile(updates, intervalMins);
        builder.initProgress(initProgress);
    }
    delegateListener.onResponse(builder.build());
}
Also used : NestedQueryBuilder(org.opensearch.index.query.NestedQueryBuilder) TermQueryBuilder(org.opensearch.index.query.TermQueryBuilder) SearchSourceBuilder(org.opensearch.search.builder.SearchSourceBuilder) BoolQueryBuilder(org.opensearch.index.query.BoolQueryBuilder) InitProgressProfile(org.opensearch.ad.model.InitProgressProfile) EntityProfile(org.opensearch.ad.model.EntityProfile)

Aggregations

EntityProfile (org.opensearch.ad.model.EntityProfile)8 InitProgressProfile (org.opensearch.ad.model.InitProgressProfile)8 AnomalyDetector (org.opensearch.ad.model.AnomalyDetector)6 AnomalyDetectorJob (org.opensearch.ad.model.AnomalyDetectorJob)6 BoolQueryBuilder (org.opensearch.index.query.BoolQueryBuilder)6 NestedQueryBuilder (org.opensearch.index.query.NestedQueryBuilder)6 TermQueryBuilder (org.opensearch.index.query.TermQueryBuilder)6 SearchSourceBuilder (org.opensearch.search.builder.SearchSourceBuilder)6 Map (java.util.Map)5 Set (java.util.Set)5 ActionListener (org.opensearch.action.ActionListener)5 GetRequest (org.opensearch.action.get.GetRequest)5 SearchRequest (org.opensearch.action.search.SearchRequest)5 CommonErrorMessages (org.opensearch.ad.constant.CommonErrorMessages)5 CommonName (org.opensearch.ad.constant.CommonName)5 ANOMALY_DETECTORS_INDEX (org.opensearch.ad.model.AnomalyDetector.ANOMALY_DETECTORS_INDEX)5 ANOMALY_DETECTOR_JOB_INDEX (org.opensearch.ad.model.AnomalyDetectorJob.ANOMALY_DETECTOR_JOB_INDEX)5 Entity (org.opensearch.ad.model.Entity)5 EntityProfileName (org.opensearch.ad.model.EntityProfileName)5 EntityState (org.opensearch.ad.model.EntityState)5