Search in sources :

Example 76 with POST

use of javax.ws.rs.POST in project pinot by linkedin.

the class DetectionJobResource method tuneAlertFilter.

/**
   *
   * @param id anomaly function id
   * @param startTime start time of anomalies to tune alert filter
   * @param endTime end time of anomalies to tune alert filter
   * @param autoTuneType the type of auto tune to invoke (default is "AUTOTUNE")
   * @return HTTP response of request: string of alert filter
   */
@POST
@Path("/autotune/filter/{functionId}")
public Response tuneAlertFilter(@PathParam("functionId") long id, @QueryParam("startTime") long startTime, @QueryParam("endTime") long endTime, @QueryParam("autoTuneType") String autoTuneType) {
    // get anomalies by function id, start time and end time
    AnomalyFunctionDTO anomalyFunctionSpec = DAO_REGISTRY.getAnomalyFunctionDAO().findById(id);
    AnomalyFunctionManager anomalyFunctionDAO = DAO_REGISTRY.getAnomalyFunctionDAO();
    MergedAnomalyResultManager anomalyMergedResultDAO = DAO_REGISTRY.getMergedAnomalyResultDAO();
    List<MergedAnomalyResultDTO> anomalyResultDTOS = anomalyMergedResultDAO.findByStartTimeInRangeAndFunctionId(startTime, endTime, id);
    // create alert filter and evaluator
    AlertFilter alertFilter = alertFilterFactory.fromSpec(anomalyFunctionSpec.getAlertFilter());
    AlertFilterEvaluationUtil evaluator = new AlertFilterEvaluationUtil(alertFilter);
    // create alert filter auto tune
    AlertFilterAutoTune alertFilterAutotune = alertFilterAutotuneFactory.fromSpec(autoTuneType);
    LOG.info("initiated alertFilterAutoTune of Type {}", alertFilterAutotune.getClass().toString());
    try {
        //evaluate current alert filter (calculate current precision and recall)
        evaluator.updatePrecisionAndRecall(anomalyResultDTOS);
        LOG.info("AlertFilter of Type {}, has been evaluated with precision: {}, recall: {}", alertFilter.getClass().toString(), evaluator.getPrecision(), evaluator.getRecall());
        // get tuned alert filter
        Map<String, String> tunedAlertFilter = alertFilterAutotune.tuneAlertFilter(anomalyResultDTOS, evaluator.getPrecision(), evaluator.getRecall());
        LOG.info("tuned AlertFilter");
        // otherwise do nothing and return alert filter
        if (alertFilterAutotune.isUpdated()) {
            anomalyFunctionSpec.setAlertFilter(tunedAlertFilter);
            anomalyFunctionDAO.update(anomalyFunctionSpec);
            LOG.info("Model has been updated");
        } else {
            LOG.info("Model hasn't been updated because tuned model cannot beat original model");
        }
    } catch (Exception e) {
        LOG.warn("AutoTune throws exception due to: {}", e.getMessage());
    }
    return Response.ok(alertFilterAutotune.isUpdated()).build();
}
Also used : AnomalyFunctionManager(com.linkedin.thirdeye.datalayer.bao.AnomalyFunctionManager) AlertFilterAutoTune(com.linkedin.thirdeye.anomalydetection.alertFilterAutotune.AlertFilterAutoTune) MergedAnomalyResultDTO(com.linkedin.thirdeye.datalayer.dto.MergedAnomalyResultDTO) AlertFilter(com.linkedin.thirdeye.detector.email.filter.AlertFilter) AnomalyFunctionDTO(com.linkedin.thirdeye.datalayer.dto.AnomalyFunctionDTO) MergedAnomalyResultManager(com.linkedin.thirdeye.datalayer.bao.MergedAnomalyResultManager) AlertFilterEvaluationUtil(com.linkedin.thirdeye.detector.email.filter.AlertFilterEvaluationUtil) NullArgumentException(org.apache.commons.lang.NullArgumentException) Path(javax.ws.rs.Path) POST(javax.ws.rs.POST)

Example 77 with POST

use of javax.ws.rs.POST in project pinot by linkedin.

the class OverrideConfigResource method createOverrideConfig.

@POST
@Path("/override-config/create")
public Response createOverrideConfig(@NotNull @QueryParam("startTime") long startTimeMillis, @NotNull @QueryParam("endTime") long endTimeMillis, @QueryParam("targetLevel") String targetLevelJson, @NotNull @QueryParam("targetEntity") String targetEntity, @QueryParam("overrideProperties") String overridePropertiesJson, @QueryParam("active") boolean active) {
    Map<String, List<String>> targetLevel;
    if (StringUtils.isEmpty(targetLevelJson)) {
        targetLevel = Collections.emptyMap();
    } else {
        try {
            targetLevel = OBJECT_MAPPER.readValue(targetLevelJson, HashMap.class);
        } catch (IOException e) {
            LOG.error("Invalid JSON string {}", targetLevelJson);
            return Response.status(Response.Status.NOT_ACCEPTABLE).build();
        }
    }
    if (StringUtils.isEmpty(targetEntity)) {
        LOG.error("Received null for one of the mandatory params \"targetEntity\": {}", targetEntity);
        return Response.status(Response.Status.NOT_ACCEPTABLE).build();
    }
    Map<String, String> overrideProperties;
    if (StringUtils.isEmpty(overridePropertiesJson)) {
        overrideProperties = Collections.emptyMap();
    } else {
        try {
            overrideProperties = OBJECT_MAPPER.readValue(overridePropertiesJson, HashMap.class);
        } catch (IOException e) {
            LOG.error("Invalid JSON string {}", overridePropertiesJson);
            return Response.status(Response.Status.NOT_ACCEPTABLE).build();
        }
    }
    OverrideConfigDTO overrideConfigDTO = new OverrideConfigDTO();
    overrideConfigDTO.setStartTime(startTimeMillis);
    overrideConfigDTO.setEndTime(endTimeMillis);
    overrideConfigDTO.setTargetLevel(targetLevel);
    overrideConfigDTO.setTargetEntity(targetEntity);
    overrideConfigDTO.setOverrideProperties(overrideProperties);
    overrideConfigDTO.setActive(active);
    OverrideConfigManager overrideConfigDAO = DAO_REGISTRY.getOverrideConfigDAO();
    // Check if there exists any duplicate override config
    List<OverrideConfigDTO> existingOverrideConfigDTOs = overrideConfigDAO.findAllConflictByTargetType(targetEntity, startTimeMillis, endTimeMillis);
    for (OverrideConfigDTO existingOverrideConfig : existingOverrideConfigDTOs) {
        if (existingOverrideConfig.equals(overrideConfigDTO)) {
            LOG.error("Exists a duplicate override config: {}", existingOverrideConfig.toString());
            return Response.status(Response.Status.NOT_ACCEPTABLE).build();
        }
    }
    overrideConfigDAO.save(overrideConfigDTO);
    return Response.ok().build();
}
Also used : OverrideConfigDTO(com.linkedin.thirdeye.datalayer.dto.OverrideConfigDTO) HashMap(java.util.HashMap) List(java.util.List) IOException(java.io.IOException) OverrideConfigManager(com.linkedin.thirdeye.datalayer.bao.OverrideConfigManager) Path(javax.ws.rs.Path) POST(javax.ws.rs.POST)

Example 78 with POST

use of javax.ws.rs.POST in project pinot by linkedin.

the class AnomalyFunctionResource method analyze.

@POST
@Path("/analyze")
@Consumes(MediaType.APPLICATION_JSON)
public Response analyze(AnomalyFunctionDTO anomalyFunctionSpec, @QueryParam("startTime") Long startTime, @QueryParam("endTime") Long endTime) throws Exception {
    // TODO: replace this with Job/Task framework and job tracker page
    BaseAnomalyFunction anomalyFunction = anomalyFunctionFactory.fromSpec(anomalyFunctionSpec);
    List<Pair<Long, Long>> startEndTimeRanges = anomalyFunction.getDataRangeIntervals(startTime, endTime);
    Map<DimensionKey, MetricTimeSeries> dimensionKeyMetricTimeSeriesMap = TimeSeriesUtil.getTimeSeriesForAnomalyDetection(anomalyFunctionSpec, startEndTimeRanges);
    List<RawAnomalyResultDTO> anomalyResults = new ArrayList<>();
    List<RawAnomalyResultDTO> results = new ArrayList<>();
    List<String> collectionDimensions = DAO_REGISTRY.getDatasetConfigDAO().findByDataset(anomalyFunctionSpec.getCollection()).getDimensions();
    for (Map.Entry<DimensionKey, MetricTimeSeries> entry : dimensionKeyMetricTimeSeriesMap.entrySet()) {
        DimensionKey dimensionKey = entry.getKey();
        DimensionMap dimensionMap = DimensionMap.fromDimensionKey(dimensionKey, collectionDimensions);
        if (entry.getValue().getTimeWindowSet().size() < 2) {
            LOG.warn("Insufficient data for {} to run anomaly detection function", dimensionMap);
            continue;
        }
        try {
            // Run algorithm
            MetricTimeSeries metricTimeSeries = entry.getValue();
            LOG.info("Analyzing anomaly function with dimensionKey: {}, windowStart: {}, windowEnd: {}", dimensionMap, startTime, endTime);
            List<RawAnomalyResultDTO> resultsOfAnEntry = anomalyFunction.analyze(dimensionMap, metricTimeSeries, new DateTime(startTime), new DateTime(endTime), new ArrayList<>());
            if (resultsOfAnEntry.size() != 0) {
                results.addAll(resultsOfAnEntry);
            }
            LOG.info("{} has {} anomalies in window {} to {}", dimensionMap, resultsOfAnEntry.size(), new DateTime(startTime), new DateTime(endTime));
        } catch (Exception e) {
            LOG.error("Could not compute for {}", dimensionMap, e);
        }
    }
    if (results.size() > 0) {
        List<RawAnomalyResultDTO> validResults = new ArrayList<>();
        for (RawAnomalyResultDTO anomaly : results) {
            if (!anomaly.isDataMissing()) {
                LOG.info("Found anomaly, sev [{}] start [{}] end [{}]", anomaly.getWeight(), new DateTime(anomaly.getStartTime()), new DateTime(anomaly.getEndTime()));
                validResults.add(anomaly);
            }
        }
        anomalyResults.addAll(validResults);
    }
    return Response.ok(anomalyResults).build();
}
Also used : BaseAnomalyFunction(com.linkedin.thirdeye.detector.function.BaseAnomalyFunction) ArrayList(java.util.ArrayList) MetricTimeSeries(com.linkedin.thirdeye.api.MetricTimeSeries) DateTime(org.joda.time.DateTime) IOException(java.io.IOException) InvocationTargetException(java.lang.reflect.InvocationTargetException) RawAnomalyResultDTO(com.linkedin.thirdeye.datalayer.dto.RawAnomalyResultDTO) DimensionKey(com.linkedin.thirdeye.api.DimensionKey) DimensionMap(com.linkedin.thirdeye.api.DimensionMap) HashMap(java.util.HashMap) Map(java.util.Map) DimensionMap(com.linkedin.thirdeye.api.DimensionMap) Pair(com.linkedin.pinot.pql.parsers.utils.Pair) Path(javax.ws.rs.Path) POST(javax.ws.rs.POST) Consumes(javax.ws.rs.Consumes)

Example 79 with POST

use of javax.ws.rs.POST in project pinot by linkedin.

the class AnomalyResource method updateAnomalyMergedResultFeedback.

/**
   * @param anomalyResultId : anomaly merged result id
   * @param payload         : Json payload containing feedback @see com.linkedin.thirdeye.constant.AnomalyFeedbackType
   *                        eg. payload
   *                        <p/>
   *                        { "feedbackType": "NOT_ANOMALY", "comment": "this is not an anomaly" }
   */
@POST
@Path(value = "anomaly-merged-result/feedback/{anomaly_merged_result_id}")
public void updateAnomalyMergedResultFeedback(@PathParam("anomaly_merged_result_id") long anomalyResultId, String payload) {
    try {
        MergedAnomalyResultDTO result = anomalyMergedResultDAO.findById(anomalyResultId);
        if (result == null) {
            throw new IllegalArgumentException("AnomalyResult not found with id " + anomalyResultId);
        }
        AnomalyFeedbackDTO feedbackRequest = OBJECT_MAPPER.readValue(payload, AnomalyFeedbackDTO.class);
        AnomalyFeedbackDTO feedback = result.getFeedback();
        if (feedback == null) {
            feedback = new AnomalyFeedbackDTO();
            result.setFeedback(feedback);
        }
        if (feedbackRequest.getStatus() == null) {
            feedback.setStatus(FeedbackStatus.NEW);
        } else {
            feedback.setStatus(feedbackRequest.getStatus());
        }
        feedback.setComment(feedbackRequest.getComment());
        feedback.setFeedbackType(feedbackRequest.getFeedbackType());
        anomalyMergedResultDAO.updateAnomalyFeedback(result);
    } catch (IOException e) {
        throw new IllegalArgumentException("Invalid payload " + payload, e);
    }
}
Also used : MergedAnomalyResultDTO(com.linkedin.thirdeye.datalayer.dto.MergedAnomalyResultDTO) AnomalyFeedbackDTO(com.linkedin.thirdeye.datalayer.dto.AnomalyFeedbackDTO) IOException(java.io.IOException) Path(javax.ws.rs.Path) POST(javax.ws.rs.POST)

Example 80 with POST

use of javax.ws.rs.POST in project pinot by linkedin.

the class AnomalyResource method updateAnomalyResultFeedback.

@POST
@Path(value = "anomaly-result/feedback/{anomaly_result_id}")
public void updateAnomalyResultFeedback(@PathParam("anomaly_result_id") long anomalyResultId, String payload) {
    try {
        RawAnomalyResultDTO result = rawAnomalyResultDAO.findById(anomalyResultId);
        if (result == null) {
            throw new IllegalArgumentException("AnomalyResult not found with id " + anomalyResultId);
        }
        AnomalyFeedbackDTO feedbackRequest = OBJECT_MAPPER.readValue(payload, AnomalyFeedbackDTO.class);
        AnomalyFeedbackDTO feedback = result.getFeedback();
        if (feedback == null) {
            feedback = new AnomalyFeedbackDTO();
            result.setFeedback(feedback);
        }
        if (feedbackRequest.getStatus() == null) {
            feedback.setStatus(FeedbackStatus.NEW);
        } else {
            feedback.setStatus(feedbackRequest.getStatus());
        }
        feedback.setComment(feedbackRequest.getComment());
        feedback.setFeedbackType(feedbackRequest.getFeedbackType());
        rawAnomalyResultDAO.update(result);
    } catch (IOException e) {
        throw new IllegalArgumentException("Invalid payload " + payload, e);
    }
}
Also used : RawAnomalyResultDTO(com.linkedin.thirdeye.datalayer.dto.RawAnomalyResultDTO) AnomalyFeedbackDTO(com.linkedin.thirdeye.datalayer.dto.AnomalyFeedbackDTO) IOException(java.io.IOException) Path(javax.ws.rs.Path) POST(javax.ws.rs.POST)

Aggregations

POST (javax.ws.rs.POST)513 Path (javax.ws.rs.Path)360 Consumes (javax.ws.rs.Consumes)242 Produces (javax.ws.rs.Produces)222 ApiOperation (io.swagger.annotations.ApiOperation)133 ApiResponses (io.swagger.annotations.ApiResponses)107 IOException (java.io.IOException)74 URI (java.net.URI)63 WebApplicationException (javax.ws.rs.WebApplicationException)62 Timed (com.codahale.metrics.annotation.Timed)55 Response (javax.ws.rs.core.Response)50 TimedResource (org.killbill.commons.metrics.TimedResource)36 CallContext (org.killbill.billing.util.callcontext.CallContext)35 AuditEvent (org.graylog2.audit.jersey.AuditEvent)33 HashMap (java.util.HashMap)32 BadRequestException (co.cask.cdap.common.BadRequestException)24 AuditPolicy (co.cask.cdap.common.security.AuditPolicy)24 ResponseBuilder (javax.ws.rs.core.Response.ResponseBuilder)23 Account (org.killbill.billing.account.api.Account)22 ExceptionMetered (com.codahale.metrics.annotation.ExceptionMetered)20