Examples with MetricsSystem org.apache.hadoop.metrics2.MetricsSystem used on opensource projects

Example 11 with MetricsSystem

use of org.apache.hadoop.metrics2.MetricsSystem in project hadoop by apache.

the class TestRollingFileSystemSinkWithHdfs method testSilentFailedWrite.

/**
   * Test that writing to HDFS fails silently when HDFS is unavailable.
   *
   * @throws IOException thrown when reading or writing log files
   * @throws java.lang.InterruptedException thrown if interrupted
   */
@Test
public void testSilentFailedWrite() throws IOException, InterruptedException {
    final String path = "hdfs://" + cluster.getNameNode().getHostAndPort() + "/tmp";
    MetricsSystem ms = initMetricsSystem(path, true, false);
    new MyMetrics1().registerWith(ms);
    shutdownHdfs();
    MockSink.errored = false;
    // publish the metrics
    ms.publishMetricsNow();
    assertFalse("An exception was generated writing metrics " + "while HDFS was unavailable, even though the sink is set to " + "ignore errors", MockSink.errored);
    try {
        ms.stop();
    } finally {
        ms.shutdown();
    }
}

Example 12 with MetricsSystem

use of org.apache.hadoop.metrics2.MetricsSystem in project hadoop by apache.

the class TestRollingFileSystemSinkWithHdfs method testSilentFailedClose.

/**
   * Test that closing a file in HDFS silently fails when HDFS is unavailable.
   *
   * @throws IOException thrown when reading or writing log files
   */
@Test
public void testSilentFailedClose() throws IOException {
    final String path = "hdfs://" + cluster.getNameNode().getHostAndPort() + "/tmp";
    MetricsSystem ms = initMetricsSystem(path, true, false);
    new MyMetrics1().registerWith(ms);
    // publish the metrics
    ms.publishMetricsNow();
    shutdownHdfs();
    MockSink.errored = false;
    try {
        ms.stop();
        assertFalse("An exception was generated stopping sink " + "while HDFS was unavailable, even though the sink is set to " + "ignore errors", MockSink.errored);
    } finally {
        ms.shutdown();
    }
}

Example 13 with MetricsSystem

use of org.apache.hadoop.metrics2.MetricsSystem in project hadoop by apache.

the class TestRollingFileSystemSinkWithHdfs method testFailedClose.

/**
   * Test that closing a file in HDFS fails when HDFS is unavailable.
   *
   * @throws IOException thrown when reading or writing log files
   */
@Test
public void testFailedClose() throws IOException {
    final String path = "hdfs://" + cluster.getNameNode().getHostAndPort() + "/tmp";
    MetricsSystem ms = initMetricsSystem(path, false, false);
    new MyMetrics1().registerWith(ms);
    // publish the metrics
    ms.publishMetricsNow();
    shutdownHdfs();
    MockSink.errored = false;
    try {
        ms.stop();
        assertTrue("No exception was generated while stopping sink " + "even though HDFS was unavailable", MockSink.errored);
    } catch (MetricsException ex) {
    // Expected
    } finally {
        ms.shutdown();
    }
}

Example 14 with MetricsSystem

use of org.apache.hadoop.metrics2.MetricsSystem in project hadoop by apache.

the class TestClusterMetrics method tearDown.

@After
public void tearDown() {
    ClusterMetrics.destroy();
    MetricsSystem ms = DefaultMetricsSystem.instance();
    if (ms.getSource("ClusterMetrics") != null) {
        DefaultMetricsSystem.shutdown();
    }
}

Example 15 with MetricsSystem

use of org.apache.hadoop.metrics2.MetricsSystem in project hadoop by apache.

the class TestShuffleHandler method checkShuffleMetrics.

static void checkShuffleMetrics(MetricsSystem ms, long bytes, int failed, int succeeded, int connections) {
    MetricsSource source = ms.getSource("ShuffleMetrics");
    MetricsRecordBuilder rb = getMetrics(source);
    assertCounter("ShuffleOutputBytes", bytes, rb);
    assertCounter("ShuffleOutputsFailed", failed, rb);
    assertCounter("ShuffleOutputsOK", succeeded, rb);
    assertGauge("ShuffleConnections", connections, rb);
}