Search in sources :

Example 1 with ZookeeperPersistentStoreProvider

use of org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider in project drill by axbaretto.

the class TestPStoreProviders method zkBackwardCompatabilityTest.

/**
 * DRILL-5809
 * Note: If this test breaks you are probably breaking backward and forward compatibility. Verify with the community
 * that breaking compatibility is acceptable and planned for.
 * @throws Exception
 */
@Test
public void zkBackwardCompatabilityTest() throws Exception {
    final String oldName = "myOldOption";
    try (CuratorFramework curator = createCurator()) {
        curator.start();
        PersistentStoreConfig<PersistedOptionValue> storeConfig = PersistentStoreConfig.newJacksonBuilder(new ObjectMapper(), PersistedOptionValue.class).name("sys.test").build();
        try (ZookeeperClient zkClient = new ZookeeperClient(curator, PathUtils.join("/", storeConfig.getName()), CreateMode.PERSISTENT)) {
            zkClient.start();
            String oldOptionJson = DrillFileUtils.getResourceAsString("/options/old_booleanopt.json");
            zkClient.put(oldName, oldOptionJson.getBytes(), null);
        }
        try (ZookeeperPersistentStoreProvider provider = new ZookeeperPersistentStoreProvider(zkHelper.getConfig(), curator)) {
            PersistentStore<PersistedOptionValue> store = provider.getOrCreateStore(storeConfig);
            assertTrue(store instanceof ZookeeperPersistentStore);
            PersistedOptionValue oldOptionValue = ((ZookeeperPersistentStore<PersistedOptionValue>) store).get(oldName, null);
            PersistedOptionValue expectedValue = new PersistedOptionValue("true");
            Assert.assertEquals(expectedValue, oldOptionValue);
        }
    }
}
Also used : ZookeeperClient(org.apache.drill.exec.coord.zk.ZookeeperClient) CuratorFramework(org.apache.curator.framework.CuratorFramework) ZookeeperPersistentStore(org.apache.drill.exec.store.sys.store.ZookeeperPersistentStore) PersistedOptionValue(org.apache.drill.exec.server.options.PersistedOptionValue) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) ZookeeperPersistentStoreProvider(org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider) SlowTest(org.apache.drill.categories.SlowTest) Test(org.junit.Test)

Example 2 with ZookeeperPersistentStoreProvider

use of org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider in project drill by axbaretto.

the class TestPStoreProviders method verifyZkStore.

@Test
public void verifyZkStore() throws Exception {
    try (CuratorFramework curator = createCurator()) {
        curator.start();
        ZookeeperPersistentStoreProvider provider = new ZookeeperPersistentStoreProvider(zkHelper.getConfig(), curator);
        PStoreTestUtil.test(provider);
    }
}
Also used : CuratorFramework(org.apache.curator.framework.CuratorFramework) ZookeeperPersistentStoreProvider(org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider) SlowTest(org.apache.drill.categories.SlowTest) Test(org.junit.Test)

Example 3 with ZookeeperPersistentStoreProvider

use of org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider in project drill by apache.

the class TestPStoreProviders method zkBackwardCompatabilityTest.

/**
 * DRILL-5809
 * Note: If this test breaks you are probably breaking backward and forward compatibility. Verify with the community
 * that breaking compatibility is acceptable and planned for.
 * @throws Exception
 */
@Test
public void zkBackwardCompatabilityTest() throws Exception {
    final String oldName = "myOldOption";
    try (CuratorFramework curator = createCurator()) {
        curator.start();
        PersistentStoreConfig<PersistedOptionValue> storeConfig = PersistentStoreConfig.newJacksonBuilder(new ObjectMapper(), PersistedOptionValue.class).name("sys.test").build();
        try (ZookeeperClient zkClient = new ZookeeperClient(curator, PathUtils.join("/", storeConfig.getName()), CreateMode.PERSISTENT)) {
            zkClient.start();
            String oldOptionJson = DrillFileUtils.getResourceAsString("/options/old_booleanopt.json");
            zkClient.put(oldName, oldOptionJson.getBytes(), null);
        }
        try (ZookeeperPersistentStoreProvider provider = new ZookeeperPersistentStoreProvider(zkHelper.getConfig(), curator)) {
            PersistentStore<PersistedOptionValue> store = provider.getOrCreateStore(storeConfig);
            assertTrue(store instanceof ZookeeperPersistentStore);
            PersistedOptionValue oldOptionValue = ((ZookeeperPersistentStore<PersistedOptionValue>) store).get(oldName, null);
            PersistedOptionValue expectedValue = new PersistedOptionValue("true");
            Assert.assertEquals(expectedValue, oldOptionValue);
        }
    }
}
Also used : ZookeeperClient(org.apache.drill.exec.coord.zk.ZookeeperClient) CuratorFramework(org.apache.curator.framework.CuratorFramework) ZookeeperPersistentStore(org.apache.drill.exec.store.sys.store.ZookeeperPersistentStore) PersistedOptionValue(org.apache.drill.exec.server.options.PersistedOptionValue) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) ZookeeperPersistentStoreProvider(org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider) SlowTest(org.apache.drill.categories.SlowTest) Test(org.junit.Test) FlakyTest(org.apache.drill.categories.FlakyTest)

Example 4 with ZookeeperPersistentStoreProvider

use of org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider in project drill by apache.

the class TestPStoreProviders method verifyZkStore.

@Test
public void verifyZkStore() throws Exception {
    try (CuratorFramework curator = createCurator()) {
        curator.start();
        PersistentStoreProvider provider = new ZookeeperPersistentStoreProvider(zkHelper.getConfig(), curator);
        PStoreTestUtil.test(provider);
    }
}
Also used : CuratorFramework(org.apache.curator.framework.CuratorFramework) LocalPersistentStoreProvider(org.apache.drill.exec.store.sys.store.provider.LocalPersistentStoreProvider) ZookeeperPersistentStoreProvider(org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider) ZookeeperPersistentStoreProvider(org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider) SlowTest(org.apache.drill.categories.SlowTest) Test(org.junit.Test) FlakyTest(org.apache.drill.categories.FlakyTest)

Example 5 with ZookeeperPersistentStoreProvider

use of org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider in project drill by apache.

the class TestPcapWithPersistentStore method pcapPluginBackwardCompatabilityTest.

/**
 * DRILL-7828
 * Note: If this test breaks you are probably breaking backward and forward compatibility. Verify with the community
 * that breaking compatibility is acceptable and planned for.
 */
@Test
public void pcapPluginBackwardCompatabilityTest() throws Exception {
    final String oldPlugin = "oldFormatPlugin";
    try (CuratorFramework curator = createCurator()) {
        curator.start();
        ObjectMapper objectMapper = new ObjectMapper();
        objectMapper.registerSubtypes(PcapFormatConfig.class, PcapngFormatConfig.class);
        PersistentStoreConfig<FileSystemConfig> storeConfig = PersistentStoreConfig.newJacksonBuilder(objectMapper, FileSystemConfig.class).name("type").build();
        try (ZookeeperClient zkClient = new ZookeeperClient(curator, PathUtils.join("/", storeConfig.getName()), CreateMode.PERSISTENT)) {
            zkClient.start();
            String oldFormatPlugin = DrillFileUtils.getResourceAsString("/config/oldPcapPlugins.json");
            zkClient.put(oldPlugin, oldFormatPlugin.getBytes(), null);
        }
        try (ZookeeperPersistentStoreProvider provider = new ZookeeperPersistentStoreProvider(zkHelper.getConfig(), curator)) {
            PersistentStore<FileSystemConfig> store = provider.getOrCreateStore(storeConfig);
            assertTrue(store instanceof ZookeeperPersistentStore);
            FileSystemConfig oldPluginConfig = ((ZookeeperPersistentStore<FileSystemConfig>) store).get(oldPlugin, null);
            Map<String, FormatPluginConfig> formats = oldPluginConfig.getFormats();
            Assert.assertEquals(formats.keySet(), ImmutableSet.of("pcap", "pcapng"));
            PcapFormatConfig pcap = (PcapFormatConfig) formats.get("pcap");
            PcapngFormatConfig pcapng = (PcapngFormatConfig) formats.get("pcapng");
            Assert.assertEquals(pcap.getExtensions(), ImmutableList.of("pcap"));
            assertTrue(pcapng.getStat());
        }
    }
}
Also used : ZookeeperClient(org.apache.drill.exec.coord.zk.ZookeeperClient) ZookeeperPersistentStore(org.apache.drill.exec.store.sys.store.ZookeeperPersistentStore) CuratorFramework(org.apache.curator.framework.CuratorFramework) PcapFormatConfig(org.apache.drill.exec.store.pcap.plugin.PcapFormatConfig) FormatPluginConfig(org.apache.drill.common.logical.FormatPluginConfig) PcapngFormatConfig(org.apache.drill.exec.store.pcap.plugin.PcapngFormatConfig) FileSystemConfig(org.apache.drill.exec.store.dfs.FileSystemConfig) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) ZookeeperPersistentStoreProvider(org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider) Test(org.junit.Test)

Aggregations

CuratorFramework (org.apache.curator.framework.CuratorFramework)5 ZookeeperPersistentStoreProvider (org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider)5 Test (org.junit.Test)5 SlowTest (org.apache.drill.categories.SlowTest)4 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)3 ZookeeperClient (org.apache.drill.exec.coord.zk.ZookeeperClient)3 ZookeeperPersistentStore (org.apache.drill.exec.store.sys.store.ZookeeperPersistentStore)3 FlakyTest (org.apache.drill.categories.FlakyTest)2 PersistedOptionValue (org.apache.drill.exec.server.options.PersistedOptionValue)2 FormatPluginConfig (org.apache.drill.common.logical.FormatPluginConfig)1 FileSystemConfig (org.apache.drill.exec.store.dfs.FileSystemConfig)1 PcapFormatConfig (org.apache.drill.exec.store.pcap.plugin.PcapFormatConfig)1 PcapngFormatConfig (org.apache.drill.exec.store.pcap.plugin.PcapngFormatConfig)1 LocalPersistentStoreProvider (org.apache.drill.exec.store.sys.store.provider.LocalPersistentStoreProvider)1