Search in sources :

Example 6 with FetcherConfiguration

use of com.linkedin.drelephant.configurations.fetcher.FetcherConfiguration in project dr-elephant by linkedin.

the class MapReduceFSFetcherHadoop2Test method testFetcherDefaultConfig.

@Test
public void testFetcherDefaultConfig() {
    FetcherConfiguration fetcherConf = new FetcherConfiguration(document9.getDocumentElement());
    try {
        MapReduceFSFetcherHadoop2 fetcher = new MapReduceFSFetcherHadoop2(fetcherConf.getFetchersConfigurationData().get(0));
        Assert.assertFalse("Sampling should be disabled in default", fetcher.isSamplingEnabled());
        Assert.assertEquals(fetcher.DEFALUT_MAX_LOG_SIZE_IN_MB, fetcher.getMaxLogSizeInMB(), 0.0001);
        Assert.assertEquals(TimeZone.getDefault(), fetcher.getTimeZone());
        List<Object> list = new ArrayList<Object>();
        int listLen = fetcher.MAX_SAMPLE_SIZE * 2;
        for (int i = 0; i < listLen; i++) {
            list.add(0);
        }
        Assert.assertEquals("Should not sample task list when sampling is disabled", listLen, fetcher.sampleAndGetSize("appId", list));
    } catch (IOException e) {
        Assert.assertNull("Failed to initialize FileSystem", e);
    }
}
Also used : FetcherConfiguration(com.linkedin.drelephant.configurations.fetcher.FetcherConfiguration) ArrayList(java.util.ArrayList) IOException(java.io.IOException) Test(org.junit.Test)

Aggregations

FetcherConfiguration (com.linkedin.drelephant.configurations.fetcher.FetcherConfiguration)6 IOException (java.io.IOException)5 Test (org.junit.Test)5 ArrayList (java.util.ArrayList)4 AnalyticJob (com.linkedin.drelephant.analysis.AnalyticJob)1 ApplicationType (com.linkedin.drelephant.analysis.ApplicationType)1 ElephantFetcher (com.linkedin.drelephant.analysis.ElephantFetcher)1 FetcherConfigurationData (com.linkedin.drelephant.configurations.fetcher.FetcherConfigurationData)1 MapReduceTaskData (com.linkedin.drelephant.mapreduce.data.MapReduceTaskData)1 InvocationTargetException (java.lang.reflect.InvocationTargetException)1 Calendar (java.util.Calendar)1 JobHistoryParser (org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser)1 Document (org.w3c.dom.Document)1