Search in sources :

Example 1 with NettyShuffleDescriptorBuilder

use of org.apache.flink.runtime.util.NettyShuffleDescriptorBuilder in project flink by apache.

the class NettyShuffleEnvironmentTest method testRegisteringDebloatingMetrics.

@Test
@SuppressWarnings("unchecked")
public void testRegisteringDebloatingMetrics() throws IOException {
    Map<String, Metric> metrics = new ConcurrentHashMap<>();
    final TaskMetricGroup taskMetricGroup = createTaskMetricGroup(metrics);
    final Configuration config = new Configuration();
    config.set(TaskManagerOptions.BUFFER_DEBLOAT_ENABLED, true);
    final NettyShuffleEnvironment shuffleEnvironment = new NettyShuffleEnvironmentBuilder().setDebloatConfig(BufferDebloatConfiguration.fromConfiguration(config)).build();
    shuffleEnvironment.createInputGates(shuffleEnvironment.createShuffleIOOwnerContext("test", new ExecutionAttemptID(), taskMetricGroup), (dsid, id, consumer) -> {
    }, Arrays.asList(new InputGateDeploymentDescriptor(new IntermediateDataSetID(), ResultPartitionType.PIPELINED, 0, new ShuffleDescriptor[] { new NettyShuffleDescriptorBuilder().buildRemote() }), new InputGateDeploymentDescriptor(new IntermediateDataSetID(), ResultPartitionType.PIPELINED, 1, new ShuffleDescriptor[] { new NettyShuffleDescriptorBuilder().buildRemote() })));
    for (int i = 0; i < 2; i++) {
        assertEquals(TaskManagerOptions.MEMORY_SEGMENT_SIZE.defaultValue().getBytes(), (long) ((Gauge<Integer>) getDebloatingMetric(metrics, i, MetricNames.DEBLOATED_BUFFER_SIZE)).getValue());
        assertEquals(0L, (long) ((Gauge<Long>) getDebloatingMetric(metrics, i, MetricNames.ESTIMATED_TIME_TO_CONSUME_BUFFERS)).getValue());
    }
}
Also used : ExecutionAttemptID(org.apache.flink.runtime.executiongraph.ExecutionAttemptID) NettyShuffleDescriptorBuilder(org.apache.flink.runtime.util.NettyShuffleDescriptorBuilder) BufferDebloatConfiguration(org.apache.flink.runtime.throughput.BufferDebloatConfiguration) Configuration(org.apache.flink.configuration.Configuration) TaskMetricGroup(org.apache.flink.runtime.metrics.groups.TaskMetricGroup) InputGateDeploymentDescriptor(org.apache.flink.runtime.deployment.InputGateDeploymentDescriptor) Gauge(org.apache.flink.metrics.Gauge) Metric(org.apache.flink.metrics.Metric) IntermediateDataSetID(org.apache.flink.runtime.jobgraph.IntermediateDataSetID) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Test(org.junit.Test)

Aggregations

ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)1 Configuration (org.apache.flink.configuration.Configuration)1 Gauge (org.apache.flink.metrics.Gauge)1 Metric (org.apache.flink.metrics.Metric)1 InputGateDeploymentDescriptor (org.apache.flink.runtime.deployment.InputGateDeploymentDescriptor)1 ExecutionAttemptID (org.apache.flink.runtime.executiongraph.ExecutionAttemptID)1 IntermediateDataSetID (org.apache.flink.runtime.jobgraph.IntermediateDataSetID)1 TaskMetricGroup (org.apache.flink.runtime.metrics.groups.TaskMetricGroup)1 BufferDebloatConfiguration (org.apache.flink.runtime.throughput.BufferDebloatConfiguration)1 NettyShuffleDescriptorBuilder (org.apache.flink.runtime.util.NettyShuffleDescriptorBuilder)1 Test (org.junit.Test)1