Search in sources :

Example 6 with MutableCounterLong

use of org.apache.hadoop.metrics2.lib.MutableCounterLong in project hadoop by apache.

the class TestEntityGroupFSTimelineStore method testSummaryRead.

@Test
public void testSummaryRead() throws Exception {
    // Load data
    EntityGroupFSTimelineStore.AppLogs appLogs = store.new AppLogs(mainTestAppId, mainTestAppDirPath, AppState.COMPLETED);
    MutableCounterLong summaryLogEntityRead = store.metrics.getGetEntityToSummaryOps();
    long numEntityReadBefore = summaryLogEntityRead.value();
    TimelineDataManager tdm = PluginStoreTestUtils.getTdmWithStore(config, store);
    appLogs.scanForLogs();
    appLogs.parseSummaryLogs(tdm);
    // Verify single entity read
    PluginStoreTestUtils.verifyTestEntities(tdm);
    // Verify multiple entities read
    TimelineEntities entities = tdm.getEntities("type_1", null, null, null, null, null, null, null, EnumSet.allOf(TimelineReader.Field.class), UserGroupInformation.getLoginUser());
    assertEquals(entities.getEntities().size(), 1);
    for (TimelineEntity entity : entities.getEntities()) {
        assertEquals((Long) 123L, entity.getStartTime());
    }
    // Verify metrics
    assertEquals(numEntityReadBefore + 5L, summaryLogEntityRead.value());
}
Also used : TimelineEntities(org.apache.hadoop.yarn.api.records.timeline.TimelineEntities) MutableCounterLong(org.apache.hadoop.metrics2.lib.MutableCounterLong) TimelineEntity(org.apache.hadoop.yarn.api.records.timeline.TimelineEntity) Test(org.junit.Test)

Example 7 with MutableCounterLong

use of org.apache.hadoop.metrics2.lib.MutableCounterLong in project hadoop by apache.

the class TestEntityGroupFSTimelineStore method testParseSummaryLogs.

@Test
public void testParseSummaryLogs() throws Exception {
    TimelineDataManager tdm = PluginStoreTestUtils.getTdmWithMemStore(config);
    MutableCounterLong scanned = store.metrics.getEntitiesReadToSummary();
    long beforeScan = scanned.value();
    EntityGroupFSTimelineStore.AppLogs appLogs = store.new AppLogs(mainTestAppId, mainTestAppDirPath, AppState.COMPLETED);
    appLogs.scanForLogs();
    appLogs.parseSummaryLogs(tdm);
    PluginStoreTestUtils.verifyTestEntities(tdm);
    assertEquals(beforeScan + 2L, scanned.value());
}
Also used : MutableCounterLong(org.apache.hadoop.metrics2.lib.MutableCounterLong) Test(org.junit.Test)

Aggregations

MutableCounterLong (org.apache.hadoop.metrics2.lib.MutableCounterLong)7 Test (org.junit.Test)5 TimelineEntities (org.apache.hadoop.yarn.api.records.timeline.TimelineEntities)2 TimelineEntity (org.apache.hadoop.yarn.api.records.timeline.TimelineEntity)2 ServiceException (com.google.protobuf.ServiceException)1 IOException (java.io.IOException)1 InterruptedIOException (java.io.InterruptedIOException)1 URL (java.net.URL)1 URLClassLoader (java.net.URLClassLoader)1 HashMap (java.util.HashMap)1 Map (java.util.Map)1 FSDataOutputStream (org.apache.hadoop.fs.FSDataOutputStream)1 Path (org.apache.hadoop.fs.Path)1 Connection (org.apache.hadoop.ipc.Server.Connection)1 MetricStringBuilder (org.apache.hadoop.metrics2.MetricStringBuilder)1 MutableStat (org.apache.hadoop.metrics2.lib.MutableStat)1 ApplicationClassLoader (org.apache.hadoop.util.ApplicationClassLoader)1