Search in sources :

Example 1 with RestClient

use of org.apache.flink.runtime.rest.RestClient in project flink by apache.

the class MetricsAvailabilityITCase method testReporter.

@Test
public void testReporter() throws Exception {
    try (ClusterController ignored = dist.startCluster(1)) {
        final RestClient restClient = new RestClient(new Configuration(), scheduledExecutorService);
        checkJobManagerMetricAvailability(restClient);
        final Collection<ResourceID> taskManagerIds = getTaskManagerIds(restClient);
        for (final ResourceID taskManagerId : taskManagerIds) {
            checkTaskManagerMetricAvailability(restClient, taskManagerId);
        }
    }
}
Also used : ClusterController(org.apache.flink.tests.util.flink.ClusterController) Configuration(org.apache.flink.configuration.Configuration) ResourceID(org.apache.flink.runtime.clusterframework.types.ResourceID) RestClient(org.apache.flink.runtime.rest.RestClient) Test(org.junit.Test)

Example 2 with RestClient

use of org.apache.flink.runtime.rest.RestClient in project flink by apache.

the class AbstractHandlerITCase method createRestClient.

private RestClient createRestClient(int serverPort) throws ConfigurationException {
    Configuration config = new Configuration(REST_BASE_CONFIG);
    config.setInteger(RestOptions.PORT, serverPort);
    return new RestClient(config, Executors.directExecutor());
}
Also used : Configuration(org.apache.flink.configuration.Configuration) RestClient(org.apache.flink.runtime.rest.RestClient)

Example 3 with RestClient

use of org.apache.flink.runtime.rest.RestClient in project flink by apache.

the class LocalStandaloneFlinkResource method startCluster.

@Override
public ClusterController startCluster(int numTaskManagers) throws IOException {
    distribution.setTaskExecutorHosts(Collections.nCopies(numTaskManagers, "localhost"));
    distribution.startFlinkCluster();
    try (final RestClient restClient = new RestClient(new Configuration(), Executors.directExecutor())) {
        for (int retryAttempt = 0; retryAttempt < 30; retryAttempt++) {
            final CompletableFuture<TaskManagersInfo> localhost = restClient.sendRequest("localhost", 8081, TaskManagersHeaders.getInstance(), EmptyMessageParameters.getInstance(), EmptyRequestBody.getInstance());
            try {
                final TaskManagersInfo taskManagersInfo = localhost.get(1, TimeUnit.SECONDS);
                final int numRunningTaskManagers = taskManagersInfo.getTaskManagerInfos().size();
                if (numRunningTaskManagers == numTaskManagers) {
                    return new StandaloneClusterController(distribution);
                } else {
                    LOG.info("Waiting for task managers to come up. {}/{} are currently running.", numRunningTaskManagers, numTaskManagers);
                }
            } catch (InterruptedException e) {
                LOG.info("Waiting for dispatcher REST endpoint to come up...");
                Thread.currentThread().interrupt();
            } catch (TimeoutException | ExecutionException e) {
                // ExecutionExceptions may occur if leader election is still going on
                LOG.info("Waiting for dispatcher REST endpoint to come up...");
            }
            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            }
        }
    } catch (ConfigurationException e) {
        throw new RuntimeException("Could not create RestClient.", e);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
    throw new RuntimeException("Cluster did not start in expected time-frame.");
}
Also used : Configuration(org.apache.flink.configuration.Configuration) RestClient(org.apache.flink.runtime.rest.RestClient) ConfigurationException(org.apache.flink.util.ConfigurationException) TimeoutException(java.util.concurrent.TimeoutException) IOException(java.io.IOException) ExecutionException(java.util.concurrent.ExecutionException) ConfigurationException(org.apache.flink.util.ConfigurationException) ExecutionException(java.util.concurrent.ExecutionException) TaskManagersInfo(org.apache.flink.runtime.rest.messages.taskmanager.TaskManagersInfo) TimeoutException(java.util.concurrent.TimeoutException)

Example 4 with RestClient

use of org.apache.flink.runtime.rest.RestClient in project flink by apache.

the class RestClusterClientSavepointTriggerTest method createRestClusterClient.

private RestClusterClient<StandaloneClusterId> createRestClusterClient(final int port) throws Exception {
    final Configuration clientConfig = new Configuration(REST_CONFIG);
    clientConfig.setInteger(RestOptions.PORT, port);
    return new RestClusterClient<>(clientConfig, new RestClient(REST_CONFIG, executor), StandaloneClusterId.getInstance(), (attempt) -> 0);
}
Also used : UnmodifiableConfiguration(org.apache.flink.configuration.UnmodifiableConfiguration) Configuration(org.apache.flink.configuration.Configuration) RestClient(org.apache.flink.runtime.rest.RestClient)

Example 5 with RestClient

use of org.apache.flink.runtime.rest.RestClient in project flink by apache.

the class AbstractHandlerITCase method testOOMErrorMessageEnrichment.

@Test
public void testOOMErrorMessageEnrichment() throws Exception {
    final TestMessageHeaders<EmptyRequestBody, EmptyResponseBody, EmptyMessageParameters> messageHeaders = TestMessageHeaders.emptyBuilder().setTargetRestEndpointURL("/test-handler").build();
    final TestRestHandler<RestfulGateway, EmptyRequestBody, EmptyResponseBody, EmptyMessageParameters> testRestHandler = new TestRestHandler<>(mockGatewayRetriever, messageHeaders, FutureUtils.completedExceptionally(new OutOfMemoryError("Metaspace")));
    try (final TestRestServerEndpoint server = TestRestServerEndpoint.builder(REST_BASE_CONFIG).withHandler(messageHeaders, testRestHandler).buildAndStart();
        final RestClient restClient = createRestClient(server.getServerAddress().getPort())) {
        CompletableFuture<EmptyResponseBody> response = restClient.sendRequest(server.getServerAddress().getHostName(), server.getServerAddress().getPort(), messageHeaders, EmptyMessageParameters.getInstance(), EmptyRequestBody.getInstance());
        try {
            response.get();
            fail("An ExecutionException was expected here being caused by the OutOfMemoryError.");
        } catch (ExecutionException e) {
            assertThat(e.getMessage(), StringContains.containsString("Metaspace. The metaspace out-of-memory error has occurred. "));
        }
    }
}
Also used : TestRestServerEndpoint(org.apache.flink.runtime.rest.util.TestRestServerEndpoint) RestClient(org.apache.flink.runtime.rest.RestClient) TestRestHandler(org.apache.flink.runtime.rest.util.TestRestHandler) EmptyResponseBody(org.apache.flink.runtime.rest.messages.EmptyResponseBody) ExecutionException(java.util.concurrent.ExecutionException) EmptyRequestBody(org.apache.flink.runtime.rest.messages.EmptyRequestBody) RestfulGateway(org.apache.flink.runtime.webmonitor.RestfulGateway) EmptyMessageParameters(org.apache.flink.runtime.rest.messages.EmptyMessageParameters) Test(org.junit.Test)

Aggregations

RestClient (org.apache.flink.runtime.rest.RestClient)12 Configuration (org.apache.flink.configuration.Configuration)10 ExecutorService (java.util.concurrent.ExecutorService)3 DEFAULT_COLLECT_DATA_TIMEOUT (org.apache.flink.connector.testframe.utils.ConnectorTestConstants.DEFAULT_COLLECT_DATA_TIMEOUT)3 DEFAULT_JOB_STATUS_CHANGE_TIMEOUT (org.apache.flink.connector.testframe.utils.ConnectorTestConstants.DEFAULT_JOB_STATUS_CHANGE_TIMEOUT)3 JobClient (org.apache.flink.core.execution.JobClient)3 StreamExecutionEnvironment (org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)3 Test (org.junit.Test)3 TestAbortedException (org.opentest4j.TestAbortedException)3 ExecutionException (java.util.concurrent.ExecutionException)2 TestingSinkSettings (org.apache.flink.connector.testframe.external.sink.TestingSinkSettings)2 FromElementsSource (org.apache.flink.connector.testframe.source.FromElementsSource)2 MetricQuerier (org.apache.flink.connector.testframe.utils.MetricQuerier)2 EmptyRequestBody (org.apache.flink.runtime.rest.messages.EmptyRequestBody)2 JobMessageParameters (org.apache.flink.runtime.rest.messages.JobMessageParameters)2 TaskManagersInfo (org.apache.flink.runtime.rest.messages.taskmanager.TaskManagersInfo)2 ConfigurationException (org.apache.flink.util.ConfigurationException)2 YarnConfiguration (org.apache.hadoop.yarn.conf.YarnConfiguration)2 DisplayName (org.junit.jupiter.api.DisplayName)2 TestTemplate (org.junit.jupiter.api.TestTemplate)2