Search in sources :

Example 1 with MultiZkException

use of org.apache.helix.zookeeper.exception.MultiZkException in project helix by apache.

the class TestMultiZkHelixJavaApis method testZkRoutingDataSourceConfigs.

/**
 * Testing using ZK as the routing data source. We use BaseDataAccessor as the representative
 * Helix API.
 * Two modes are tested: ZK and HTTP-ZK fallback
 */
@Test(dependsOnMethods = "testDifferentMsdsEndpointConfigs")
public void testZkRoutingDataSourceConfigs() {
    // Set up routing data in ZK by connecting directly to ZK
    BaseDataAccessor<ZNRecord> accessor = new ZkBaseDataAccessor.Builder<ZNRecord>().setZkAddress(ZK_PREFIX + ZK_START_PORT).build();
    // Create ZK realm routing data ZNRecord
    _rawRoutingData.forEach((realm, keys) -> {
        ZNRecord znRecord = new ZNRecord(realm);
        znRecord.setListField(MetadataStoreRoutingConstants.ZNRECORD_LIST_FIELD_KEY, new ArrayList<>(keys));
        accessor.set(MetadataStoreRoutingConstants.ROUTING_DATA_PATH + "/" + realm, znRecord, AccessOption.PERSISTENT);
    });
    // Create connection configs with the source type set to each type
    final RealmAwareZkClient.RealmAwareZkConnectionConfig.Builder connectionConfigBuilder = new RealmAwareZkClient.RealmAwareZkConnectionConfig.Builder();
    final RealmAwareZkClient.RealmAwareZkConnectionConfig connectionConfigZk = connectionConfigBuilder.setRoutingDataSourceType(RoutingDataReaderType.ZK.name()).setRoutingDataSourceEndpoint(ZK_PREFIX + ZK_START_PORT).build();
    final RealmAwareZkClient.RealmAwareZkConnectionConfig connectionConfigHttpZkFallback = connectionConfigBuilder.setRoutingDataSourceType(RoutingDataReaderType.HTTP_ZK_FALLBACK.name()).setRoutingDataSourceEndpoint(_msdsEndpoint + "," + ZK_PREFIX + ZK_START_PORT).build();
    final RealmAwareZkClient.RealmAwareZkConnectionConfig connectionConfigHttp = connectionConfigBuilder.setRoutingDataSourceType(RoutingDataReaderType.HTTP.name()).setRoutingDataSourceEndpoint(_msdsEndpoint).build();
    // Reset cached routing data
    RoutingDataManager.getInstance().reset();
    // Shutdown MSDS to ensure that these accessors are able to pull routing data from ZK
    _msds.stopServer();
    // Create a BaseDataAccessor instance with the connection config
    BaseDataAccessor<ZNRecord> zkBasedAccessor = new ZkBaseDataAccessor.Builder<ZNRecord>().setRealmAwareZkConnectionConfig(connectionConfigZk).build();
    BaseDataAccessor<ZNRecord> httpZkFallbackBasedAccessor = new ZkBaseDataAccessor.Builder<ZNRecord>().setRealmAwareZkConnectionConfig(connectionConfigHttpZkFallback).build();
    try {
        BaseDataAccessor<ZNRecord> httpBasedAccessor = new ZkBaseDataAccessor.Builder<ZNRecord>().setRealmAwareZkConnectionConfig(connectionConfigHttp).build();
        Assert.fail("Must fail with a MultiZkException because HTTP connection will be refused.");
    } catch (MultiZkException e) {
    // Okay
    }
    // Check that all clusters appear as existing to this accessor
    CLUSTER_LIST.forEach(cluster -> {
        Assert.assertTrue(zkBasedAccessor.exists("/" + cluster, AccessOption.PERSISTENT));
        Assert.assertTrue(httpZkFallbackBasedAccessor.exists("/" + cluster, AccessOption.PERSISTENT));
    });
    // Close all connections
    accessor.close();
    zkBasedAccessor.close();
    httpZkFallbackBasedAccessor.close();
}
Also used : MultiZkException(org.apache.helix.zookeeper.exception.MultiZkException) ZNRecord(org.apache.helix.zookeeper.datamodel.ZNRecord) RealmAwareZkClient(org.apache.helix.zookeeper.api.client.RealmAwareZkClient) Test(org.testng.annotations.Test)

Example 2 with MultiZkException

use of org.apache.helix.zookeeper.exception.MultiZkException in project helix by apache.

the class ZkRoutingDataReader method getRawRoutingData.

/**
 * Returns a map form of metadata store routing data.
 * The map fields stand for metadata store realm address (key), and a corresponding list of ZK
 * path sharding keys (key).
 * @param endpoint
 * @return
 */
@Override
public Map<String, List<String>> getRawRoutingData(String endpoint) {
    ZkClient zkClient = new ZkClient.Builder().setZkServer(endpoint).setZkSerializer(new ZNRecordSerializer()).build();
    Map<String, List<String>> routingData = new HashMap<>();
    List<String> allRealmAddresses;
    try {
        allRealmAddresses = zkClient.getChildren(MetadataStoreRoutingConstants.ROUTING_DATA_PATH);
    } catch (ZkNoNodeException e) {
        throw new MultiZkException("Routing data directory ZNode " + MetadataStoreRoutingConstants.ROUTING_DATA_PATH + " does not exist. Routing ZooKeeper address: " + endpoint);
    }
    if (allRealmAddresses != null) {
        for (String realmAddress : allRealmAddresses) {
            ZNRecord record = zkClient.readData(MetadataStoreRoutingConstants.ROUTING_DATA_PATH + "/" + realmAddress, true);
            if (record != null) {
                List<String> shardingKeys = record.getListField(MetadataStoreRoutingConstants.ZNRECORD_LIST_FIELD_KEY);
                routingData.put(realmAddress, shardingKeys != null ? shardingKeys : Collections.emptyList());
            }
        }
    }
    zkClient.close();
    return routingData;
}
Also used : ZkClient(org.apache.helix.zookeeper.impl.client.ZkClient) ZkNoNodeException(org.apache.helix.zookeeper.zkclient.exception.ZkNoNodeException) HashMap(java.util.HashMap) List(java.util.List) MultiZkException(org.apache.helix.zookeeper.exception.MultiZkException) ZNRecord(org.apache.helix.zookeeper.datamodel.ZNRecord) ZNRecordSerializer(org.apache.helix.zookeeper.datamodel.serializer.ZNRecordSerializer)

Aggregations

ZNRecord (org.apache.helix.zookeeper.datamodel.ZNRecord)2 MultiZkException (org.apache.helix.zookeeper.exception.MultiZkException)2 HashMap (java.util.HashMap)1 List (java.util.List)1 RealmAwareZkClient (org.apache.helix.zookeeper.api.client.RealmAwareZkClient)1 ZNRecordSerializer (org.apache.helix.zookeeper.datamodel.serializer.ZNRecordSerializer)1 ZkClient (org.apache.helix.zookeeper.impl.client.ZkClient)1 ZkNoNodeException (org.apache.helix.zookeeper.zkclient.exception.ZkNoNodeException)1 Test (org.testng.annotations.Test)1