Search in sources :

Example 1 with JmxReporter

use of org.apache.kafka.common.metrics.JmxReporter in project ksql by confluentinc.

the class MetricCollectors method initialize.

// visible for testing.
// We need to call this from the MetricCollectorsTest because otherwise tests clobber each
// others metric data. We also need it from the KsqlEngineMetricsTest
public static void initialize() {
    MetricConfig metricConfig = new MetricConfig().samples(100).timeWindow(1000, TimeUnit.MILLISECONDS);
    List<MetricsReporter> reporters = new ArrayList<>();
    reporters.add(new JmxReporter("io.confluent.ksql.metrics"));
    // Replace all static contents other than Time to ensure they are cleaned for tests that are
    // not aware of the need to initialize/cleanup this test, in case test processes are reused.
    // Tests aware of the class clean everything up properly to get the state into a clean state,
    // a full, fresh instantiation here ensures something like KsqlEngineMetricsTest running after
    // another test that used MetricsCollector without running cleanUp will behave correctly.
    metrics = new Metrics(metricConfig, reporters, new SystemTime());
    collectorMap = new ConcurrentHashMap<>();
}
Also used : MetricConfig(org.apache.kafka.common.metrics.MetricConfig) Metrics(org.apache.kafka.common.metrics.Metrics) MetricsReporter(org.apache.kafka.common.metrics.MetricsReporter) ArrayList(java.util.ArrayList) JmxReporter(org.apache.kafka.common.metrics.JmxReporter) SystemTime(org.apache.kafka.common.utils.SystemTime)

Example 2 with JmxReporter

use of org.apache.kafka.common.metrics.JmxReporter in project apache-kafka-on-k8s by banzaicloud.

the class FrequenciesTest method setup.

@Before
public void setup() {
    config = new MetricConfig().eventWindow(50).samples(2);
    time = new MockTime();
    metrics = new Metrics(config, Arrays.asList((MetricsReporter) new JmxReporter()), time, true);
}
Also used : MetricConfig(org.apache.kafka.common.metrics.MetricConfig) Metrics(org.apache.kafka.common.metrics.Metrics) JmxReporter(org.apache.kafka.common.metrics.JmxReporter) MockTime(org.apache.kafka.common.utils.MockTime) Before(org.junit.Before)

Example 3 with JmxReporter

use of org.apache.kafka.common.metrics.JmxReporter in project apache-kafka-on-k8s by banzaicloud.

the class KafkaAdminClient method createInternal.

static KafkaAdminClient createInternal(AdminClientConfig config, TimeoutProcessorFactory timeoutProcessorFactory) {
    Metrics metrics = null;
    NetworkClient networkClient = null;
    Time time = Time.SYSTEM;
    String clientId = generateClientId(config);
    ChannelBuilder channelBuilder = null;
    Selector selector = null;
    ApiVersions apiVersions = new ApiVersions();
    LogContext logContext = createLogContext(clientId);
    try {
        // Since we only request node information, it's safe to pass true for allowAutoTopicCreation (and it
        // simplifies communication with older brokers)
        Metadata metadata = new Metadata(config.getLong(AdminClientConfig.RETRY_BACKOFF_MS_CONFIG), config.getLong(AdminClientConfig.METADATA_MAX_AGE_CONFIG), true);
        List<MetricsReporter> reporters = config.getConfiguredInstances(AdminClientConfig.METRIC_REPORTER_CLASSES_CONFIG, MetricsReporter.class);
        Map<String, String> metricTags = Collections.singletonMap("client-id", clientId);
        MetricConfig metricConfig = new MetricConfig().samples(config.getInt(AdminClientConfig.METRICS_NUM_SAMPLES_CONFIG)).timeWindow(config.getLong(AdminClientConfig.METRICS_SAMPLE_WINDOW_MS_CONFIG), TimeUnit.MILLISECONDS).recordLevel(Sensor.RecordingLevel.forName(config.getString(AdminClientConfig.METRICS_RECORDING_LEVEL_CONFIG))).tags(metricTags);
        reporters.add(new JmxReporter(JMX_PREFIX));
        metrics = new Metrics(metricConfig, reporters, time);
        String metricGrpPrefix = "admin-client";
        channelBuilder = ClientUtils.createChannelBuilder(config);
        selector = new Selector(config.getLong(AdminClientConfig.CONNECTIONS_MAX_IDLE_MS_CONFIG), metrics, time, metricGrpPrefix, channelBuilder, logContext);
        networkClient = new NetworkClient(selector, metadata, clientId, 1, config.getLong(AdminClientConfig.RECONNECT_BACKOFF_MS_CONFIG), config.getLong(AdminClientConfig.RECONNECT_BACKOFF_MAX_MS_CONFIG), config.getInt(AdminClientConfig.SEND_BUFFER_CONFIG), config.getInt(AdminClientConfig.RECEIVE_BUFFER_CONFIG), (int) TimeUnit.HOURS.toMillis(1), time, true, apiVersions, logContext);
        return new KafkaAdminClient(config, clientId, time, metadata, metrics, networkClient, timeoutProcessorFactory, logContext);
    } catch (Throwable exc) {
        closeQuietly(metrics, "Metrics");
        closeQuietly(networkClient, "NetworkClient");
        closeQuietly(selector, "Selector");
        closeQuietly(channelBuilder, "ChannelBuilder");
        throw new KafkaException("Failed create new KafkaAdminClient", exc);
    }
}
Also used : MetricConfig(org.apache.kafka.common.metrics.MetricConfig) Metadata(org.apache.kafka.clients.Metadata) LogContext(org.apache.kafka.common.utils.LogContext) Time(org.apache.kafka.common.utils.Time) JmxReporter(org.apache.kafka.common.metrics.JmxReporter) Metrics(org.apache.kafka.common.metrics.Metrics) NetworkClient(org.apache.kafka.clients.NetworkClient) MetricsReporter(org.apache.kafka.common.metrics.MetricsReporter) ApiVersions(org.apache.kafka.clients.ApiVersions) KafkaException(org.apache.kafka.common.KafkaException) ChannelBuilder(org.apache.kafka.common.network.ChannelBuilder) Selector(org.apache.kafka.common.network.Selector)

Aggregations

JmxReporter (org.apache.kafka.common.metrics.JmxReporter)3 MetricConfig (org.apache.kafka.common.metrics.MetricConfig)3 Metrics (org.apache.kafka.common.metrics.Metrics)3 MetricsReporter (org.apache.kafka.common.metrics.MetricsReporter)2 ArrayList (java.util.ArrayList)1 ApiVersions (org.apache.kafka.clients.ApiVersions)1 Metadata (org.apache.kafka.clients.Metadata)1 NetworkClient (org.apache.kafka.clients.NetworkClient)1 KafkaException (org.apache.kafka.common.KafkaException)1 ChannelBuilder (org.apache.kafka.common.network.ChannelBuilder)1 Selector (org.apache.kafka.common.network.Selector)1 LogContext (org.apache.kafka.common.utils.LogContext)1 MockTime (org.apache.kafka.common.utils.MockTime)1 SystemTime (org.apache.kafka.common.utils.SystemTime)1 Time (org.apache.kafka.common.utils.Time)1 Before (org.junit.Before)1