Search in sources :

Example 1 with MonitorConfiguration

use of com.linkedin.thirdeye.anomaly.monitor.MonitorConfiguration in project pinot by linkedin.

the class TaskGenerator method createMonitorTasks.

public List<MonitorTaskInfo> createMonitorTasks(MonitorJobContext monitorJobContext) {
    List<MonitorTaskInfo> tasks = new ArrayList<>();
    // TODO: Currently generates 1 task for updating all the completed jobs
    // We might need to create more tasks and assign only certain number of updations to each (say 5k)
    MonitorTaskInfo updateTaskInfo = new MonitorTaskInfo();
    updateTaskInfo.setMonitorType(MonitorType.UPDATE);
    tasks.add(updateTaskInfo);
    MonitorConfiguration monitorConfiguration = monitorJobContext.getMonitorConfiguration();
    MonitorTaskInfo expireTaskInfo = new MonitorTaskInfo();
    expireTaskInfo.setMonitorType(MonitorType.EXPIRE);
    expireTaskInfo.setExpireDaysAgo(monitorConfiguration.getExpireDaysAgo());
    tasks.add(expireTaskInfo);
    return tasks;
}
Also used : MonitorTaskInfo(com.linkedin.thirdeye.anomaly.monitor.MonitorTaskInfo) MonitorConfiguration(com.linkedin.thirdeye.anomaly.monitor.MonitorConfiguration) ArrayList(java.util.ArrayList)

Example 2 with MonitorConfiguration

use of com.linkedin.thirdeye.anomaly.monitor.MonitorConfiguration in project pinot by linkedin.

the class AnomalyApplicationEndToEndTest method setup.

private void setup() throws Exception {
    // Mock query cache
    ThirdEyeClient mockThirdeyeClient = Mockito.mock(ThirdEyeClient.class);
    Mockito.when(mockThirdeyeClient.execute(Matchers.any(ThirdEyeRequest.class))).thenAnswer(new Answer<ThirdEyeResponse>() {

        @Override
        public ThirdEyeResponse answer(InvocationOnMock invocation) throws Throwable {
            Object[] args = invocation.getArguments();
            ThirdEyeRequest request = (ThirdEyeRequest) args[0];
            ThirdEyeResponse response = getMockResponse(request);
            return response;
        }
    });
    QueryCache mockQueryCache = new QueryCache(mockThirdeyeClient, Executors.newFixedThreadPool(10));
    cacheRegistry.registerQueryCache(mockQueryCache);
    MetricConfigDTO metricConfig = getTestMetricConfig(collection, metric, 1L);
    // create metric config in cache
    LoadingCache<MetricDataset, MetricConfigDTO> mockMetricConfigCache = Mockito.mock(LoadingCache.class);
    Mockito.when(mockMetricConfigCache.get(new MetricDataset(metric, collection))).thenReturn(metricConfig);
    cacheRegistry.registerMetricConfigCache(mockMetricConfigCache);
    // create dataset config in cache
    LoadingCache<String, DatasetConfigDTO> mockDatasetConfigCache = Mockito.mock(LoadingCache.class);
    Mockito.when(mockDatasetConfigCache.get(collection)).thenReturn(getTestDatasetConfig(collection));
    cacheRegistry.registerDatasetConfigCache(mockDatasetConfigCache);
    ResultSet mockResultSet = Mockito.mock(ResultSet.class);
    Mockito.when(mockResultSet.getRowCount()).thenReturn(0);
    ResultSetGroup mockResultSetGroup = Mockito.mock(ResultSetGroup.class);
    Mockito.when(mockResultSetGroup.getResultSet(0)).thenReturn(mockResultSet);
    LoadingCache<PinotQuery, ResultSetGroup> mockResultSetGroupCache = Mockito.mock(LoadingCache.class);
    Mockito.when(mockResultSetGroupCache.get(Matchers.any(PinotQuery.class))).thenAnswer(new Answer<ResultSetGroup>() {

        @Override
        public ResultSetGroup answer(InvocationOnMock invocation) throws Throwable {
            return mockResultSetGroup;
        }
    });
    cacheRegistry.registerResultSetGroupCache(mockResultSetGroupCache);
    // Application config
    thirdeyeAnomalyConfig = new ThirdEyeAnomalyConfiguration();
    thirdeyeAnomalyConfig.setId(id);
    thirdeyeAnomalyConfig.setDashboardHost(dashboardHost);
    MonitorConfiguration monitorConfiguration = new MonitorConfiguration();
    monitorConfiguration.setMonitorFrequency(new TimeGranularity(30, TimeUnit.SECONDS));
    thirdeyeAnomalyConfig.setMonitorConfiguration(monitorConfiguration);
    thirdeyeAnomalyConfig.setRootDir(System.getProperty("dw.rootDir", "NOT_SET(dw.rootDir)"));
    // create test anomaly function
    functionId = anomalyFunctionDAO.save(getTestFunctionSpec(metric, collection));
    // create test email configuration
    emailConfigurationDAO.save(getTestEmailConfiguration(metric, collection));
    // create test alert configuration
    alertConfigDAO.save(getTestAlertConfiguration("test alert v2"));
    // create test dataset config
    datasetConfigDAO.save(getTestDatasetConfig(collection));
    // setup function factory for worker and merger
    InputStream factoryStream = AnomalyApplicationEndToEndTest.class.getResourceAsStream(functionPropertiesFile);
    anomalyFunctionFactory = new AnomalyFunctionFactory(factoryStream);
    // setup alertfilter factory for worker
    InputStream alertFilterStream = AnomalyApplicationEndToEndTest.class.getResourceAsStream(alertFilterPropertiesFile);
    alertFilterFactory = new AlertFilterFactory(alertFilterStream);
}
Also used : QueryCache(com.linkedin.thirdeye.client.cache.QueryCache) AlertFilterFactory(com.linkedin.thirdeye.detector.email.filter.AlertFilterFactory) ThirdEyeClient(com.linkedin.thirdeye.client.ThirdEyeClient) ThirdEyeRequest(com.linkedin.thirdeye.client.ThirdEyeRequest) DatasetConfigDTO(com.linkedin.thirdeye.datalayer.dto.DatasetConfigDTO) ResultSet(com.linkedin.pinot.client.ResultSet) TimeGranularity(com.linkedin.thirdeye.api.TimeGranularity) AnomalyFunctionFactory(com.linkedin.thirdeye.detector.function.AnomalyFunctionFactory) MetricConfigDTO(com.linkedin.thirdeye.datalayer.dto.MetricConfigDTO) InputStream(java.io.InputStream) MonitorConfiguration(com.linkedin.thirdeye.anomaly.monitor.MonitorConfiguration) ThirdEyeResponse(com.linkedin.thirdeye.client.ThirdEyeResponse) PinotThirdEyeResponse(com.linkedin.thirdeye.client.pinot.PinotThirdEyeResponse) ThirdEyeAnomalyConfiguration(com.linkedin.thirdeye.anomaly.ThirdEyeAnomalyConfiguration) ResultSetGroup(com.linkedin.pinot.client.ResultSetGroup) MetricDataset(com.linkedin.thirdeye.client.cache.MetricDataset) InvocationOnMock(org.mockito.invocation.InvocationOnMock) PinotQuery(com.linkedin.thirdeye.client.pinot.PinotQuery)

Aggregations

MonitorConfiguration (com.linkedin.thirdeye.anomaly.monitor.MonitorConfiguration)2 ResultSet (com.linkedin.pinot.client.ResultSet)1 ResultSetGroup (com.linkedin.pinot.client.ResultSetGroup)1 ThirdEyeAnomalyConfiguration (com.linkedin.thirdeye.anomaly.ThirdEyeAnomalyConfiguration)1 MonitorTaskInfo (com.linkedin.thirdeye.anomaly.monitor.MonitorTaskInfo)1 TimeGranularity (com.linkedin.thirdeye.api.TimeGranularity)1 ThirdEyeClient (com.linkedin.thirdeye.client.ThirdEyeClient)1 ThirdEyeRequest (com.linkedin.thirdeye.client.ThirdEyeRequest)1 ThirdEyeResponse (com.linkedin.thirdeye.client.ThirdEyeResponse)1 MetricDataset (com.linkedin.thirdeye.client.cache.MetricDataset)1 QueryCache (com.linkedin.thirdeye.client.cache.QueryCache)1 PinotQuery (com.linkedin.thirdeye.client.pinot.PinotQuery)1 PinotThirdEyeResponse (com.linkedin.thirdeye.client.pinot.PinotThirdEyeResponse)1 DatasetConfigDTO (com.linkedin.thirdeye.datalayer.dto.DatasetConfigDTO)1 MetricConfigDTO (com.linkedin.thirdeye.datalayer.dto.MetricConfigDTO)1 AlertFilterFactory (com.linkedin.thirdeye.detector.email.filter.AlertFilterFactory)1 AnomalyFunctionFactory (com.linkedin.thirdeye.detector.function.AnomalyFunctionFactory)1 InputStream (java.io.InputStream)1 ArrayList (java.util.ArrayList)1 InvocationOnMock (org.mockito.invocation.InvocationOnMock)1