Examples with QueryableStateClient org.apache.flink.queryablestate.client.QueryableStateClient used on opensource projects

Example 1 with QueryableStateClient

use of org.apache.flink.queryablestate.client.QueryableStateClient in project flink by apache.

the class QsStateClient method main.

public static void main(final String[] args) throws Exception {
    ParameterTool parameters = ParameterTool.fromArgs(args);
    // setup values
    String jobId = parameters.getRequired("job-id");
    String host = parameters.get("host", "localhost");
    int port = parameters.getInt("port", 9069);
    int numIterations = parameters.getInt("iterations", 1500);
    QueryableStateClient client = new QueryableStateClient(host, port);
    client.setExecutionConfig(new ExecutionConfig());
    MapStateDescriptor<EmailId, EmailInformation> stateDescriptor = new MapStateDescriptor<>(QsConstants.STATE_NAME, TypeInformation.of(new TypeHint<EmailId>() {
    }), TypeInformation.of(new TypeHint<EmailInformation>() {
    }));
    System.out.println("Wait until the state can be queried.");
    // wait for state to exist
    for (int i = 0; i < BOOTSTRAP_RETRIES; i++) {
        // ~120s
        try {
            getMapState(jobId, client, stateDescriptor);
            break;
        } catch (ExecutionException e) {
            if (e.getCause() instanceof UnknownKeyOrNamespaceException) {
                System.err.println("State does not exist yet; sleeping 500ms");
                Thread.sleep(500L);
            } else {
                throw e;
            }
        }
        if (i == (BOOTSTRAP_RETRIES - 1)) {
            throw new RuntimeException("Timeout: state doesn't exist after 120s");
        }
    }
    System.out.println(String.format("State exists. Start querying it %d times.", numIterations));
    // query state
    for (int iterations = 0; iterations < numIterations; iterations++) {
        MapState<EmailId, EmailInformation> mapState = getMapState(jobId, client, stateDescriptor);
        int counter = 0;
        for (Map.Entry<EmailId, EmailInformation> entry : mapState.entries()) {
            // this is to force deserialization
            entry.getKey();
            entry.getValue();
            counter++;
        }
        System.out.println(// we look for it in the test
        "MapState has " + counter + " entries");
        Thread.sleep(100L);
    }
}

Example 2 with QueryableStateClient

use of org.apache.flink.queryablestate.client.QueryableStateClient in project flink by apache.

the class NonHAQueryableStateRocksDBBackendITCase method setup.

@BeforeClass
public static void setup() throws Exception {
    client = new QueryableStateClient("localhost", QS_PROXY_PORT_RANGE_START);
    clusterClient = MINI_CLUSTER_RESOURCE.getClusterClient();
}

Example 3 with QueryableStateClient

use of org.apache.flink.queryablestate.client.QueryableStateClient in project flink by apache.

the class HAQueryableStateRocksDBBackendITCase method setup.

@BeforeClass
public static void setup() throws Exception {
    zkServer = new TestingServer();
    // we have to manage this manually because we have to create the ZooKeeper server
    // ahead of this
    miniClusterResource = new MiniClusterWithClientResource(new MiniClusterResourceConfiguration.Builder().setConfiguration(getConfig()).setNumberTaskManagers(NUM_TMS).setNumberSlotsPerTaskManager(NUM_SLOTS_PER_TM).build());
    miniClusterResource.before();
    client = new QueryableStateClient("localhost", QS_PROXY_PORT_RANGE_START);
    clusterClient = miniClusterResource.getClusterClient();
}

Example 4 with QueryableStateClient

use of org.apache.flink.queryablestate.client.QueryableStateClient in project flink by apache.

the class HAQueryableStateFsBackendITCase method setup.

@BeforeClass
public static void setup() throws Exception {
    zkServer = new TestingServer();
    // we have to manage this manually because we have to create the ZooKeeper server
    // ahead of this
    miniClusterResource = new MiniClusterWithClientResource(new MiniClusterResourceConfiguration.Builder().setConfiguration(getConfig()).setNumberTaskManagers(NUM_TMS).setNumberSlotsPerTaskManager(NUM_SLOTS_PER_TM).build());
    miniClusterResource.before();
    client = new QueryableStateClient("localhost", QS_PROXY_PORT_RANGE_START);
    clusterClient = miniClusterResource.getClusterClient();
}

Example 5 with QueryableStateClient

use of org.apache.flink.queryablestate.client.QueryableStateClient in project flink by apache.

the class NonHAQueryableStateFsBackendITCase method setup.

@BeforeClass
public static void setup() throws Exception {
    client = new QueryableStateClient("localhost", QS_PROXY_PORT_RANGE_START);
    clusterClient = MINI_CLUSTER_RESOURCE.getClusterClient();
}