Search in sources :

Example 1 with MetricsCollector

use of org.apache.heron.common.utils.metrics.MetricsCollector in project heron by twitter.

the class MetricsManager method initSinkExecutor.

private SinkExecutor initSinkExecutor(String sinkId) {
    IMetricsSink sink;
    String classname = (String) config.getConfigForSink(sinkId).get(MetricsSinksConfig.CONFIG_KEY_CLASSNAME);
    try {
        sink = (IMetricsSink) Class.forName(classname).newInstance();
    } catch (InstantiationException e) {
        throw new RuntimeException(e + " IMetricsSink class must have a no-arg constructor.");
    } catch (IllegalAccessException e) {
        throw new RuntimeException(e + " IMetricsSink class must be concrete.");
    } catch (ClassNotFoundException e) {
        throw new RuntimeException(e + " IMetricsSink class must be a class path.");
    }
    ExecutorLooper sinkExecutorLoop = new ExecutorLooper();
    Communicator<MetricsRecord> executorInMetricsQueue = new Communicator<MetricsRecord>(null, sinkExecutorLoop);
    // Since MetricsCollector is not thread-safe,
    // we need to specify individual MetricsCollector and MultiCountMetric
    // for different SinkExecutor
    MetricsCollector sinkMetricsCollector = new MetricsCollector(sinkExecutorLoop, metricsQueue);
    MultiCountMetric internalCounters = new MultiCountMetric();
    sinkMetricsCollector.registerMetric(sinkId, internalCounters, (int) heronMetricsExportInterval.getSeconds());
    // Set up the SinkContext
    SinkContext sinkContext = new SinkContextImpl(topologyName, cluster, role, environment, metricsmgrId, sinkId, internalCounters);
    SinkExecutor sinkExecutor = new SinkExecutor(sinkId, sink, sinkExecutorLoop, executorInMetricsQueue, sinkContext);
    sinkExecutor.setPropertyMap(config.getConfigForSink(sinkId));
    return sinkExecutor;
}
Also used : IMetricsSink(org.apache.heron.spi.metricsmgr.sink.IMetricsSink) MetricsCollector(org.apache.heron.common.utils.metrics.MetricsCollector) Communicator(org.apache.heron.common.basics.Communicator) MetricsRecord(org.apache.heron.spi.metricsmgr.metrics.MetricsRecord) SinkContext(org.apache.heron.spi.metricsmgr.sink.SinkContext) SinkExecutor(org.apache.heron.metricsmgr.executor.SinkExecutor) MultiCountMetric(org.apache.heron.api.metric.MultiCountMetric) ExecutorLooper(org.apache.heron.common.basics.ExecutorLooper) SinkContextImpl(org.apache.heron.metricsmgr.sink.SinkContextImpl)

Example 2 with MetricsCollector

use of org.apache.heron.common.utils.metrics.MetricsCollector in project heron by twitter.

the class GlobalMetricsTest method testGlobalMetrics.

@Test
public void testGlobalMetrics() {
    MetricsCollector fakeCollector = new MetricsCollector(new FakeWakeableLooper(), null);
    TopologyContext fakeContext = new TopologyContextImpl(new HashMap<String, Object>(), TopologyAPI.Topology.getDefaultInstance(), new HashMap<Integer, String>(), 0, fakeCollector);
    GlobalMetrics.init(fakeContext, Duration.ofSeconds(5));
    GlobalMetrics.incr("mycounter");
    Map<String, Long> metricsContent = GlobalMetrics.getUnderlyingCounter().getValueAndReset();
    assertTrue(metricsContent.containsKey("mycounter"));
    assertEquals(1, metricsContent.size());
    assertEquals(new Long(1), metricsContent.get("mycounter"));
    // Increment two different counters
    GlobalMetrics.incr("mycounter1");
    GlobalMetrics.incr("mycounter2");
    GlobalMetrics.incr("mycounter1");
    metricsContent = GlobalMetrics.getUnderlyingCounter().getValueAndReset();
    assertTrue(metricsContent.containsKey("mycounter"));
    assertTrue(metricsContent.containsKey("mycounter1"));
    assertTrue(metricsContent.containsKey("mycounter2"));
    assertEquals(3L, metricsContent.size());
    assertEquals(new Long(0), metricsContent.get("mycounter"));
    assertEquals(new Long(1), metricsContent.get("mycounter2"));
    assertEquals(new Long(2), metricsContent.get("mycounter1"));
}
Also used : MetricsCollector(org.apache.heron.common.utils.metrics.MetricsCollector) TopologyContextImpl(org.apache.heron.common.utils.topology.TopologyContextImpl) TopologyContext(org.apache.heron.api.topology.TopologyContext) Test(org.junit.Test)

Example 3 with MetricsCollector

use of org.apache.heron.common.utils.metrics.MetricsCollector in project heron by twitter.

the class Executor method registerTasksWithExecutor.

private void registerTasksWithExecutor() {
    // Create a new MetricsCollector with the clean executorLooper and register its task
    metricsCollector = new MetricsCollector(executorLooper, metricsOutCommunicator);
    // registering the handling of control msg
    handleControlMessage();
// instance task will be registered when instance.start() is called
}
Also used : MetricsCollector(org.apache.heron.common.utils.metrics.MetricsCollector)

Example 4 with MetricsCollector

use of org.apache.heron.common.utils.metrics.MetricsCollector in project heron by twitter.

the class InstanceExecutorTest method testCreatePhysicalPlanHelper.

/**
 * Method: createPhysicalPlanHelper(PhysicalPlans.PhysicalPlan physicalPlan, String instanceId, MetricsCollector metricsCollector)
 */
@Test
public void testCreatePhysicalPlanHelper() throws Exception {
    PhysicalPlanHelper physicalPlanHelper = instanceExecutor.createPhysicalPlanHelper(plan, instanceId, Mockito.mock(MetricsCollector.class));
    Assert.assertNotNull(physicalPlanHelper.getTopologyContext());
}
Also used : MetricsCollector(org.apache.heron.common.utils.metrics.MetricsCollector) PhysicalPlanHelper(org.apache.heron.common.utils.misc.PhysicalPlanHelper) Test(org.junit.Test) TopologyManagerTest(org.apache.heron.simulator.utils.TopologyManagerTest)

Aggregations

MetricsCollector (org.apache.heron.common.utils.metrics.MetricsCollector)4 Test (org.junit.Test)2 MultiCountMetric (org.apache.heron.api.metric.MultiCountMetric)1 TopologyContext (org.apache.heron.api.topology.TopologyContext)1 Communicator (org.apache.heron.common.basics.Communicator)1 ExecutorLooper (org.apache.heron.common.basics.ExecutorLooper)1 PhysicalPlanHelper (org.apache.heron.common.utils.misc.PhysicalPlanHelper)1 TopologyContextImpl (org.apache.heron.common.utils.topology.TopologyContextImpl)1 SinkExecutor (org.apache.heron.metricsmgr.executor.SinkExecutor)1 SinkContextImpl (org.apache.heron.metricsmgr.sink.SinkContextImpl)1 TopologyManagerTest (org.apache.heron.simulator.utils.TopologyManagerTest)1 MetricsRecord (org.apache.heron.spi.metricsmgr.metrics.MetricsRecord)1 IMetricsSink (org.apache.heron.spi.metricsmgr.sink.IMetricsSink)1 SinkContext (org.apache.heron.spi.metricsmgr.sink.SinkContext)1