Search in sources :

Example 1 with MetricsReporter

use of org.apache.kafka.common.metrics.MetricsReporter in project ksql by confluentinc.

the class MetricCollectors method initialize.

// visible for testing.
// We need to call this from the MetricCollectorsTest because otherwise tests clobber each
// others metric data. We also need it from the KsqlEngineMetricsTest
public static void initialize() {
    MetricConfig metricConfig = new MetricConfig().samples(100).timeWindow(1000, TimeUnit.MILLISECONDS);
    List<MetricsReporter> reporters = new ArrayList<>();
    reporters.add(new JmxReporter("io.confluent.ksql.metrics"));
    // Replace all static contents other than Time to ensure they are cleaned for tests that are
    // not aware of the need to initialize/cleanup this test, in case test processes are reused.
    // Tests aware of the class clean everything up properly to get the state into a clean state,
    // a full, fresh instantiation here ensures something like KsqlEngineMetricsTest running after
    // another test that used MetricsCollector without running cleanUp will behave correctly.
    metrics = new Metrics(metricConfig, reporters, new SystemTime());
    collectorMap = new ConcurrentHashMap<>();
}
Also used : MetricConfig(org.apache.kafka.common.metrics.MetricConfig) Metrics(org.apache.kafka.common.metrics.Metrics) MetricsReporter(org.apache.kafka.common.metrics.MetricsReporter) ArrayList(java.util.ArrayList) JmxReporter(org.apache.kafka.common.metrics.JmxReporter) SystemTime(org.apache.kafka.common.utils.SystemTime)

Example 2 with MetricsReporter

use of org.apache.kafka.common.metrics.MetricsReporter in project apache-kafka-on-k8s by banzaicloud.

the class KafkaAdminClient method createInternal.

static KafkaAdminClient createInternal(AdminClientConfig config, TimeoutProcessorFactory timeoutProcessorFactory) {
    Metrics metrics = null;
    NetworkClient networkClient = null;
    Time time = Time.SYSTEM;
    String clientId = generateClientId(config);
    ChannelBuilder channelBuilder = null;
    Selector selector = null;
    ApiVersions apiVersions = new ApiVersions();
    LogContext logContext = createLogContext(clientId);
    try {
        // Since we only request node information, it's safe to pass true for allowAutoTopicCreation (and it
        // simplifies communication with older brokers)
        Metadata metadata = new Metadata(config.getLong(AdminClientConfig.RETRY_BACKOFF_MS_CONFIG), config.getLong(AdminClientConfig.METADATA_MAX_AGE_CONFIG), true);
        List<MetricsReporter> reporters = config.getConfiguredInstances(AdminClientConfig.METRIC_REPORTER_CLASSES_CONFIG, MetricsReporter.class);
        Map<String, String> metricTags = Collections.singletonMap("client-id", clientId);
        MetricConfig metricConfig = new MetricConfig().samples(config.getInt(AdminClientConfig.METRICS_NUM_SAMPLES_CONFIG)).timeWindow(config.getLong(AdminClientConfig.METRICS_SAMPLE_WINDOW_MS_CONFIG), TimeUnit.MILLISECONDS).recordLevel(Sensor.RecordingLevel.forName(config.getString(AdminClientConfig.METRICS_RECORDING_LEVEL_CONFIG))).tags(metricTags);
        reporters.add(new JmxReporter(JMX_PREFIX));
        metrics = new Metrics(metricConfig, reporters, time);
        String metricGrpPrefix = "admin-client";
        channelBuilder = ClientUtils.createChannelBuilder(config);
        selector = new Selector(config.getLong(AdminClientConfig.CONNECTIONS_MAX_IDLE_MS_CONFIG), metrics, time, metricGrpPrefix, channelBuilder, logContext);
        networkClient = new NetworkClient(selector, metadata, clientId, 1, config.getLong(AdminClientConfig.RECONNECT_BACKOFF_MS_CONFIG), config.getLong(AdminClientConfig.RECONNECT_BACKOFF_MAX_MS_CONFIG), config.getInt(AdminClientConfig.SEND_BUFFER_CONFIG), config.getInt(AdminClientConfig.RECEIVE_BUFFER_CONFIG), (int) TimeUnit.HOURS.toMillis(1), time, true, apiVersions, logContext);
        return new KafkaAdminClient(config, clientId, time, metadata, metrics, networkClient, timeoutProcessorFactory, logContext);
    } catch (Throwable exc) {
        closeQuietly(metrics, "Metrics");
        closeQuietly(networkClient, "NetworkClient");
        closeQuietly(selector, "Selector");
        closeQuietly(channelBuilder, "ChannelBuilder");
        throw new KafkaException("Failed create new KafkaAdminClient", exc);
    }
}
Also used : MetricConfig(org.apache.kafka.common.metrics.MetricConfig) Metadata(org.apache.kafka.clients.Metadata) LogContext(org.apache.kafka.common.utils.LogContext) Time(org.apache.kafka.common.utils.Time) JmxReporter(org.apache.kafka.common.metrics.JmxReporter) Metrics(org.apache.kafka.common.metrics.Metrics) NetworkClient(org.apache.kafka.clients.NetworkClient) MetricsReporter(org.apache.kafka.common.metrics.MetricsReporter) ApiVersions(org.apache.kafka.clients.ApiVersions) KafkaException(org.apache.kafka.common.KafkaException) ChannelBuilder(org.apache.kafka.common.network.ChannelBuilder) Selector(org.apache.kafka.common.network.Selector)

Aggregations

JmxReporter (org.apache.kafka.common.metrics.JmxReporter)2 MetricConfig (org.apache.kafka.common.metrics.MetricConfig)2 Metrics (org.apache.kafka.common.metrics.Metrics)2 MetricsReporter (org.apache.kafka.common.metrics.MetricsReporter)2 ArrayList (java.util.ArrayList)1 ApiVersions (org.apache.kafka.clients.ApiVersions)1 Metadata (org.apache.kafka.clients.Metadata)1 NetworkClient (org.apache.kafka.clients.NetworkClient)1 KafkaException (org.apache.kafka.common.KafkaException)1 ChannelBuilder (org.apache.kafka.common.network.ChannelBuilder)1 Selector (org.apache.kafka.common.network.Selector)1 LogContext (org.apache.kafka.common.utils.LogContext)1 SystemTime (org.apache.kafka.common.utils.SystemTime)1 Time (org.apache.kafka.common.utils.Time)1