Search in sources :

Example 1 with LocalConfig

use of io.hetu.core.filesystem.LocalConfig in project hetu-core by openlookeng.

the class TestOrderByOperator method setUp.

@BeforeMethod
public void setUp() throws IOException {
    executor = newCachedThreadPool(daemonThreadsNamed("test-executor-%s"));
    scheduledExecutor = newScheduledThreadPool(2, daemonThreadsNamed("test-scheduledExecutor-%s"));
    spillerFactory = new DummySpillerFactory();
    when(fileSystemClientManager.getFileSystemClient(any(Path.class))).thenReturn(new HetuLocalFileSystemClient(new LocalConfig(new Properties()), Paths.get("/tmp/hetu/snapshot/")));
}
Also used : Path(java.nio.file.Path) LocalConfig(io.hetu.core.filesystem.LocalConfig) Properties(java.util.Properties) HetuLocalFileSystemClient(io.hetu.core.filesystem.HetuLocalFileSystemClient) BeforeMethod(org.testng.annotations.BeforeMethod)

Example 2 with LocalConfig

use of io.hetu.core.filesystem.LocalConfig in project hetu-core by openlookeng.

the class TestWindowOperator method setUp.

@BeforeMethod
public void setUp() throws IOException {
    executor = newCachedThreadPool(daemonThreadsNamed("test-executor-%s"));
    scheduledExecutor = newScheduledThreadPool(2, daemonThreadsNamed("test-scheduledExecutor-%s"));
    spillerFactory = new DummySpillerFactory();
    when(fileSystemClientManager.getFileSystemClient(any(Path.class))).thenReturn(new HetuLocalFileSystemClient(new LocalConfig(new Properties()), Paths.get("/tmp/hetu/snapshot/")));
}
Also used : Path(java.nio.file.Path) LocalConfig(io.hetu.core.filesystem.LocalConfig) Properties(java.util.Properties) HetuLocalFileSystemClient(io.hetu.core.filesystem.HetuLocalFileSystemClient) BeforeMethod(org.testng.annotations.BeforeMethod)

Example 3 with LocalConfig

use of io.hetu.core.filesystem.LocalConfig in project hetu-core by openlookeng.

the class TestFileSingleStreamSpillerFactory method setUp.

@BeforeMethod
public void setUp() throws IOException {
    closer = Closer.create();
    executor = MoreExecutors.listeningDecorator(Executors.newCachedThreadPool());
    closer.register(() -> executor.shutdownNow());
    spillPath1 = createTempDirectory(getClass().getName()).toFile();
    closer.register(() -> deleteRecursively(spillPath1.toPath(), ALLOW_INSECURE));
    spillPath2 = createTempDirectory(getClass().getName()).toFile();
    closer.register(() -> deleteRecursively(spillPath2.toPath(), ALLOW_INSECURE));
    when(fileSystemClientManager.getFileSystemClient(spillPath1.toPath())).thenReturn(new HetuLocalFileSystemClient(new LocalConfig(new Properties()), Paths.get(spillPath1.getCanonicalPath())));
    when(fileSystemClientManager.getFileSystemClient(spillPath2.toPath())).thenReturn(new HetuLocalFileSystemClient(new LocalConfig(new Properties()), Paths.get(spillPath2.getCanonicalPath())));
    when(fileSystemClientManager.getFileSystemClient("hdfs", spillPath1.toPath())).thenReturn(getLocalHdfs(spillPath1.toString()));
    when(fileSystemClientManager.getFileSystemClient("hdfs", spillPath2.toPath())).thenReturn(getLocalHdfs(spillPath2.toString()));
}
Also used : LocalConfig(io.hetu.core.filesystem.LocalConfig) Properties(java.util.Properties) HetuLocalFileSystemClient(io.hetu.core.filesystem.HetuLocalFileSystemClient) BeforeMethod(org.testng.annotations.BeforeMethod)

Example 4 with LocalConfig

use of io.hetu.core.filesystem.LocalConfig in project hetu-core by openlookeng.

the class TestGenericPartitioningSpiller method setUp.

@BeforeClass
public void setUp() throws Exception {
    tempDirectory = createTempDirectory(getClass().getSimpleName());
    when(fileSystemClientManager.getFileSystemClient(tempDirectory)).thenReturn(new HetuLocalFileSystemClient(new LocalConfig(new Properties()), tempDirectory));
    FeaturesConfig featuresConfig = new FeaturesConfig();
    featuresConfig.setSpillerSpillPaths(tempDirectory.toString());
    featuresConfig.setSpillerThreads(8);
    featuresConfig.setSpillMaxUsedSpaceThreshold(1.0);
    SingleStreamSpillerFactory singleStreamSpillerFactory = new FileSingleStreamSpillerFactory(createTestMetadataManager(), new SpillerStats(), featuresConfig, new NodeSpillConfig(), fileSystemClientManager);
    factory = new GenericPartitioningSpillerFactory(singleStreamSpillerFactory);
    scheduledExecutor = newSingleThreadScheduledExecutor();
}
Also used : LocalConfig(io.hetu.core.filesystem.LocalConfig) FeaturesConfig(io.prestosql.sql.analyzer.FeaturesConfig) Properties(java.util.Properties) HetuLocalFileSystemClient(io.hetu.core.filesystem.HetuLocalFileSystemClient) BeforeClass(org.testng.annotations.BeforeClass)

Example 5 with LocalConfig

use of io.hetu.core.filesystem.LocalConfig in project hetu-core by openlookeng.

the class TestTaskSnapshotManager method setup.

@BeforeMethod
public void setup() throws Exception {
    // Set up snapshot config
    snapshotConfig = new SnapshotConfig();
    // Set up mock file system client manager
    fileSystemClientManager = mock(FileSystemClientManager.class);
    when(fileSystemClientManager.getFileSystemClient(any(Path.class))).thenReturn(new HetuLocalFileSystemClient(new LocalConfig(new Properties()), Paths.get(SNAPSHOT_FILE_SYSTEM_DIR)));
    snapshotUtils = new SnapshotUtils(fileSystemClientManager, snapshotConfig, new InMemoryNodeManager());
    snapshotUtils.rootPath = SNAPSHOT_FILE_SYSTEM_DIR;
    snapshotUtils.initialize();
}
Also used : Path(java.nio.file.Path) LocalConfig(io.hetu.core.filesystem.LocalConfig) Properties(java.util.Properties) HetuLocalFileSystemClient(io.hetu.core.filesystem.HetuLocalFileSystemClient) FileSystemClientManager(io.prestosql.filesystem.FileSystemClientManager) InMemoryNodeManager(io.prestosql.metadata.InMemoryNodeManager) BeforeMethod(org.testng.annotations.BeforeMethod)

Aggregations

HetuLocalFileSystemClient (io.hetu.core.filesystem.HetuLocalFileSystemClient)19 LocalConfig (io.hetu.core.filesystem.LocalConfig)19 Properties (java.util.Properties)15 BeforeMethod (org.testng.annotations.BeforeMethod)8 Path (java.nio.file.Path)7 FileSystemClientManager (io.prestosql.filesystem.FileSystemClientManager)6 BeforeClass (org.testng.annotations.BeforeClass)5 Test (org.testng.annotations.Test)5 HetuMetastore (io.prestosql.spi.metastore.HetuMetastore)4 IOException (java.io.IOException)4 ImmutableMap (com.google.common.collect.ImmutableMap)3 Injector (com.google.inject.Injector)3 Bootstrap (io.airlift.bootstrap.Bootstrap)3 MBeanServerModule (io.prestosql.plugin.base.jmx.MBeanServerModule)3 PrestoException (io.prestosql.spi.PrestoException)3 StateStore (io.prestosql.spi.statestore.StateStore)3 MBeanModule (org.weakref.jmx.guice.MBeanModule)3 JsonProcessingException (com.fasterxml.jackson.core.JsonProcessingException)2 TempFolder (io.hetu.core.common.filesystem.TempFolder)2 HetuFsMetastore (io.hetu.core.metastore.hetufilesystem.HetuFsMetastore)2