Search in sources :

Example 1 with TaskDriver

use of com.linkedin.thirdeye.anomaly.task.TaskDriver in project pinot by linkedin.

the class ThirdEyeAnomalyApplication method run.

@Override
public void run(final ThirdEyeAnomalyConfiguration config, final Environment environment) throws Exception {
    LOG.info("Starting ThirdeyeAnomalyApplication : Scheduler {} Worker {}", config.isScheduler(), config.isWorker());
    super.initDAOs();
    ThirdEyeCacheRegistry.initializeCaches(config);
    environment.lifecycle().manage(new Managed() {

        @Override
        public void start() throws Exception {
            if (config.isWorker()) {
                anomalyFunctionFactory = new AnomalyFunctionFactory(config.getFunctionConfigPath());
                alertFilterFactory = new AlertFilterFactory(config.getAlertFilterConfigPath());
                taskDriver = new TaskDriver(config, anomalyFunctionFactory, alertFilterFactory);
                taskDriver.start();
            }
            if (config.isScheduler()) {
                detectionJobScheduler = new DetectionJobScheduler();
                alertFilterFactory = new AlertFilterFactory(config.getAlertFilterConfigPath());
                alertFilterAutotuneFactory = new AlertFilterAutotuneFactory(config.getFilterAutotuneConfigPath());
                detectionJobScheduler.start();
                environment.jersey().register(new DetectionJobResource(detectionJobScheduler, alertFilterFactory, alertFilterAutotuneFactory));
                environment.jersey().register(new AnomalyFunctionResource(config.getFunctionConfigPath()));
            }
            if (config.isMonitor()) {
                monitorJobScheduler = new MonitorJobScheduler(config.getMonitorConfiguration());
                monitorJobScheduler.start();
            }
            if (config.isAlert()) {
                alertJobScheduler = new AlertJobScheduler();
                alertJobScheduler.start();
                // start alert scheduler v2
                alertJobSchedulerV2 = new AlertJobSchedulerV2();
                alertJobSchedulerV2.start();
                environment.jersey().register(new AlertJobResource(alertJobScheduler, emailConfigurationDAO));
            }
            if (config.isMerger()) {
                // anomalyFunctionFactory might have initiated if current machine is also a worker
                if (anomalyFunctionFactory == null) {
                    anomalyFunctionFactory = new AnomalyFunctionFactory(config.getFunctionConfigPath());
                }
                ScheduledExecutorService executorService = Executors.newSingleThreadScheduledExecutor();
                anomalyMergeExecutor = new AnomalyMergeExecutor(executorService, anomalyFunctionFactory);
                anomalyMergeExecutor.start();
            }
            if (config.isAutoload()) {
                autoLoadPinotMetricsService = new AutoLoadPinotMetricsService(config);
                autoLoadPinotMetricsService.start();
            }
            if (config.isDataCompleteness()) {
                dataCompletenessScheduler = new DataCompletenessScheduler();
                dataCompletenessScheduler.start();
            }
        }

        @Override
        public void stop() throws Exception {
            if (config.isWorker()) {
                taskDriver.stop();
            }
            if (config.isScheduler()) {
                detectionJobScheduler.shutdown();
            }
            if (config.isMonitor()) {
                monitorJobScheduler.stop();
            }
            if (config.isAlert()) {
                alertJobScheduler.shutdown();
                alertJobSchedulerV2.shutdown();
            }
            if (config.isMerger()) {
                anomalyMergeExecutor.stop();
            }
            if (config.isAutoload()) {
                autoLoadPinotMetricsService.shutdown();
            }
            if (config.isDataCompleteness()) {
                dataCompletenessScheduler.shutdown();
            }
        }
    });
}
Also used : DataCompletenessScheduler(com.linkedin.thirdeye.completeness.checker.DataCompletenessScheduler) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) AlertFilterFactory(com.linkedin.thirdeye.detector.email.filter.AlertFilterFactory) TaskDriver(com.linkedin.thirdeye.anomaly.task.TaskDriver) MonitorJobScheduler(com.linkedin.thirdeye.anomaly.monitor.MonitorJobScheduler) AlertJobScheduler(com.linkedin.thirdeye.anomaly.alert.AlertJobScheduler) AnomalyMergeExecutor(com.linkedin.thirdeye.anomaly.merge.AnomalyMergeExecutor) AnomalyFunctionResource(com.linkedin.thirdeye.dashboard.resources.AnomalyFunctionResource) AlertJobResource(com.linkedin.thirdeye.anomaly.alert.AlertJobResource) DetectionJobResource(com.linkedin.thirdeye.anomaly.detection.DetectionJobResource) DetectionJobScheduler(com.linkedin.thirdeye.anomaly.detection.DetectionJobScheduler) AutoLoadPinotMetricsService(com.linkedin.thirdeye.autoload.pinot.metrics.AutoLoadPinotMetricsService) AnomalyFunctionFactory(com.linkedin.thirdeye.detector.function.AnomalyFunctionFactory) Managed(io.dropwizard.lifecycle.Managed) AlertFilterAutotuneFactory(com.linkedin.thirdeye.anomalydetection.alertFilterAutotune.AlertFilterAutotuneFactory) AlertJobSchedulerV2(com.linkedin.thirdeye.anomaly.alert.v2.AlertJobSchedulerV2)

Example 2 with TaskDriver

use of com.linkedin.thirdeye.anomaly.task.TaskDriver in project pinot by linkedin.

the class AnomalyApplicationEndToEndTest method startWorker.

private void startWorker() throws Exception {
    taskDriver = new TaskDriver(thirdeyeAnomalyConfig, anomalyFunctionFactory, alertFilterFactory);
    taskDriver.start();
}
Also used : TaskDriver(com.linkedin.thirdeye.anomaly.task.TaskDriver)

Aggregations

TaskDriver (com.linkedin.thirdeye.anomaly.task.TaskDriver)2 AlertJobResource (com.linkedin.thirdeye.anomaly.alert.AlertJobResource)1 AlertJobScheduler (com.linkedin.thirdeye.anomaly.alert.AlertJobScheduler)1 AlertJobSchedulerV2 (com.linkedin.thirdeye.anomaly.alert.v2.AlertJobSchedulerV2)1 DetectionJobResource (com.linkedin.thirdeye.anomaly.detection.DetectionJobResource)1 DetectionJobScheduler (com.linkedin.thirdeye.anomaly.detection.DetectionJobScheduler)1 AnomalyMergeExecutor (com.linkedin.thirdeye.anomaly.merge.AnomalyMergeExecutor)1 MonitorJobScheduler (com.linkedin.thirdeye.anomaly.monitor.MonitorJobScheduler)1 AlertFilterAutotuneFactory (com.linkedin.thirdeye.anomalydetection.alertFilterAutotune.AlertFilterAutotuneFactory)1 AutoLoadPinotMetricsService (com.linkedin.thirdeye.autoload.pinot.metrics.AutoLoadPinotMetricsService)1 DataCompletenessScheduler (com.linkedin.thirdeye.completeness.checker.DataCompletenessScheduler)1 AnomalyFunctionResource (com.linkedin.thirdeye.dashboard.resources.AnomalyFunctionResource)1 AlertFilterFactory (com.linkedin.thirdeye.detector.email.filter.AlertFilterFactory)1 AnomalyFunctionFactory (com.linkedin.thirdeye.detector.function.AnomalyFunctionFactory)1 Managed (io.dropwizard.lifecycle.Managed)1 ScheduledExecutorService (java.util.concurrent.ScheduledExecutorService)1