Search in sources :

Example 6 with VersionMismatchException

use of org.apache.drill.exec.exception.VersionMismatchException in project drill by apache.

the class TestDynamicUDFSupport method testExceedRetryAttemptsDuringRegistration.

@Test
public void testExceedRetryAttemptsDuringRegistration() throws Exception {
    RemoteFunctionRegistry remoteFunctionRegistry = spyRemoteFunctionRegistry();
    copyDefaultJarsToStagingArea();
    doThrow(new VersionMismatchException("Version mismatch detected", 1)).when(remoteFunctionRegistry).updateRegistry(any(Registry.class), any(DataChangeVersion.class));
    String summary = "Failed to update remote function registry. Exceeded retry attempts limit.";
    testBuilder().sqlQuery("create function using jar '%s'", default_binary_name).unOrdered().baselineColumns("ok", "summary").baselineValues(false, summary).go();
    verify(remoteFunctionRegistry, times(remoteFunctionRegistry.getRetryAttempts() + 1)).updateRegistry(any(Registry.class), any(DataChangeVersion.class));
    FileSystem fs = remoteFunctionRegistry.getFs();
    assertTrue("Binary should be present in staging area", fs.exists(new Path(remoteFunctionRegistry.getStagingArea(), default_binary_name)));
    assertTrue("Source should be present in staging area", fs.exists(new Path(remoteFunctionRegistry.getStagingArea(), default_source_name)));
    assertFalse("Registry area should be empty", fs.listFiles(remoteFunctionRegistry.getRegistryArea(), false).hasNext());
    assertFalse("Temporary area should be empty", fs.listFiles(remoteFunctionRegistry.getTmpArea(), false).hasNext());
    assertEquals("Registry should be empty", remoteFunctionRegistry.getRegistry(new DataChangeVersion()).getJarList().size(), 0);
}
Also used : RemoteFunctionRegistry(org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry) Path(org.apache.hadoop.fs.Path) FileSystem(org.apache.hadoop.fs.FileSystem) LocalFunctionRegistry(org.apache.drill.exec.expr.fn.registry.LocalFunctionRegistry) FunctionImplementationRegistry(org.apache.drill.exec.expr.fn.FunctionImplementationRegistry) RemoteFunctionRegistry(org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry) Registry(org.apache.drill.exec.proto.UserBitShared.Registry) Matchers.anyString(org.mockito.Matchers.anyString) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) DataChangeVersion(org.apache.drill.exec.store.sys.store.DataChangeVersion) VersionMismatchException(org.apache.drill.exec.exception.VersionMismatchException) Test(org.junit.Test)

Example 7 with VersionMismatchException

use of org.apache.drill.exec.exception.VersionMismatchException in project drill by apache.

the class TestDynamicUDFSupport method testSuccessfulUnregistrationAfterSeveralRetryAttempts.

@Test
public void testSuccessfulUnregistrationAfterSeveralRetryAttempts() throws Exception {
    RemoteFunctionRegistry remoteFunctionRegistry = spyRemoteFunctionRegistry();
    copyDefaultJarsToStagingArea();
    test("create function using jar '%s'", default_binary_name);
    reset(remoteFunctionRegistry);
    doThrow(new VersionMismatchException("Version mismatch detected", 1)).doThrow(new VersionMismatchException("Version mismatch detected", 1)).doCallRealMethod().when(remoteFunctionRegistry).updateRegistry(any(Registry.class), any(DataChangeVersion.class));
    String summary = "The following UDFs in jar %s have been unregistered:\n" + "[custom_lower(VARCHAR-REQUIRED)]";
    testBuilder().sqlQuery("drop function using jar '%s'", default_binary_name).unOrdered().baselineColumns("ok", "summary").baselineValues(true, String.format(summary, default_binary_name)).go();
    verify(remoteFunctionRegistry, times(3)).updateRegistry(any(Registry.class), any(DataChangeVersion.class));
    FileSystem fs = remoteFunctionRegistry.getFs();
    assertFalse("Registry area should be empty", fs.listFiles(remoteFunctionRegistry.getRegistryArea(), false).hasNext());
    assertEquals("Registry should be empty", remoteFunctionRegistry.getRegistry(new DataChangeVersion()).getJarList().size(), 0);
}
Also used : RemoteFunctionRegistry(org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry) FileSystem(org.apache.hadoop.fs.FileSystem) LocalFunctionRegistry(org.apache.drill.exec.expr.fn.registry.LocalFunctionRegistry) FunctionImplementationRegistry(org.apache.drill.exec.expr.fn.FunctionImplementationRegistry) RemoteFunctionRegistry(org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry) Registry(org.apache.drill.exec.proto.UserBitShared.Registry) Matchers.anyString(org.mockito.Matchers.anyString) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) DataChangeVersion(org.apache.drill.exec.store.sys.store.DataChangeVersion) VersionMismatchException(org.apache.drill.exec.exception.VersionMismatchException) Test(org.junit.Test)

Example 8 with VersionMismatchException

use of org.apache.drill.exec.exception.VersionMismatchException in project drill by apache.

the class TestDynamicUDFSupport method testSuccessfulRegistrationAfterSeveralRetryAttempts.

@Test
public void testSuccessfulRegistrationAfterSeveralRetryAttempts() throws Exception {
    RemoteFunctionRegistry remoteFunctionRegistry = spyRemoteFunctionRegistry();
    copyDefaultJarsToStagingArea();
    doThrow(new VersionMismatchException("Version mismatch detected", 1)).doThrow(new VersionMismatchException("Version mismatch detected", 1)).doCallRealMethod().when(remoteFunctionRegistry).updateRegistry(any(Registry.class), any(DataChangeVersion.class));
    String summary = "The following UDFs in jar %s have been registered:\n" + "[custom_lower(VARCHAR-REQUIRED)]";
    testBuilder().sqlQuery("create function using jar '%s'", default_binary_name).unOrdered().baselineColumns("ok", "summary").baselineValues(true, String.format(summary, default_binary_name)).go();
    verify(remoteFunctionRegistry, times(3)).updateRegistry(any(Registry.class), any(DataChangeVersion.class));
    FileSystem fs = remoteFunctionRegistry.getFs();
    assertFalse("Staging area should be empty", fs.listFiles(remoteFunctionRegistry.getStagingArea(), false).hasNext());
    assertFalse("Temporary area should be empty", fs.listFiles(remoteFunctionRegistry.getTmpArea(), false).hasNext());
    assertTrue("Binary should be present in registry area", fs.exists(new Path(remoteFunctionRegistry.getRegistryArea(), default_binary_name)));
    assertTrue("Source should be present in registry area", fs.exists(new Path(remoteFunctionRegistry.getRegistryArea(), default_source_name)));
    Registry registry = remoteFunctionRegistry.getRegistry(new DataChangeVersion());
    assertEquals("Registry should contain one jar", registry.getJarList().size(), 1);
    assertEquals(registry.getJar(0).getName(), default_binary_name);
}
Also used : RemoteFunctionRegistry(org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry) Path(org.apache.hadoop.fs.Path) FileSystem(org.apache.hadoop.fs.FileSystem) LocalFunctionRegistry(org.apache.drill.exec.expr.fn.registry.LocalFunctionRegistry) FunctionImplementationRegistry(org.apache.drill.exec.expr.fn.FunctionImplementationRegistry) RemoteFunctionRegistry(org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry) Registry(org.apache.drill.exec.proto.UserBitShared.Registry) Matchers.anyString(org.mockito.Matchers.anyString) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) DataChangeVersion(org.apache.drill.exec.store.sys.store.DataChangeVersion) VersionMismatchException(org.apache.drill.exec.exception.VersionMismatchException) Test(org.junit.Test)

Example 9 with VersionMismatchException

use of org.apache.drill.exec.exception.VersionMismatchException in project drill by apache.

the class LocalPersistentStore method put.

@Override
public void put(String key, V value, DataChangeVersion dataChangeVersion) {
    try (AutoCloseableLock lock = writeLock.open()) {
        if (dataChangeVersion != null && dataChangeVersion.getVersion() != version) {
            throw new VersionMismatchException("Version mismatch detected", dataChangeVersion.getVersion());
        }
        try (OutputStream os = fs.create(makePath(key))) {
            IOUtils.write(config.getSerializer().serialize(value), os);
            version++;
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }
}
Also used : AutoCloseableLock(org.apache.drill.common.concurrent.AutoCloseableLock) OutputStream(java.io.OutputStream) IOException(java.io.IOException) VersionMismatchException(org.apache.drill.exec.exception.VersionMismatchException)

Example 10 with VersionMismatchException

use of org.apache.drill.exec.exception.VersionMismatchException in project drill by apache.

the class NoWriteLocalStore method put.

@Override
public void put(final String key, final V value, final DataChangeVersion dataChangeVersion) {
    try (AutoCloseableLock lock = writeLock.open()) {
        if (dataChangeVersion != null && dataChangeVersion.getVersion() != version) {
            throw new VersionMismatchException("Version mismatch detected", dataChangeVersion.getVersion());
        }
        store.put(key, value);
        version++;
    }
}
Also used : AutoCloseableLock(org.apache.drill.common.concurrent.AutoCloseableLock) VersionMismatchException(org.apache.drill.exec.exception.VersionMismatchException)

Aggregations

VersionMismatchException (org.apache.drill.exec.exception.VersionMismatchException)10 RemoteFunctionRegistry (org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry)6 Registry (org.apache.drill.exec.proto.UserBitShared.Registry)6 DataChangeVersion (org.apache.drill.exec.store.sys.store.DataChangeVersion)6 FunctionImplementationRegistry (org.apache.drill.exec.expr.fn.FunctionImplementationRegistry)5 LocalFunctionRegistry (org.apache.drill.exec.expr.fn.registry.LocalFunctionRegistry)4 FileSystem (org.apache.hadoop.fs.FileSystem)4 CoreMatchers.containsString (org.hamcrest.CoreMatchers.containsString)4 Test (org.junit.Test)4 Matchers.anyString (org.mockito.Matchers.anyString)4 AutoCloseableLock (org.apache.drill.common.concurrent.AutoCloseableLock)3 DrillRuntimeException (org.apache.drill.common.exceptions.DrillRuntimeException)3 Path (org.apache.hadoop.fs.Path)3 IOException (java.io.IOException)2 Jar (org.apache.drill.exec.proto.UserBitShared.Jar)2 OutputStream (java.io.OutputStream)1 UserException (org.apache.drill.common.exceptions.UserException)1 FunctionValidationException (org.apache.drill.exec.exception.FunctionValidationException)1 JarValidationException (org.apache.drill.exec.exception.JarValidationException)1 ForemanSetupException (org.apache.drill.exec.work.foreman.ForemanSetupException)1