Search in sources :

Example 6 with FileSystemViewStorageConfig

use of org.apache.hudi.common.table.view.FileSystemViewStorageConfig in project hudi by apache.

the class TestStreamerUtil method testDumpRemoteViewStorageConfig.

@Test
void testDumpRemoteViewStorageConfig() throws IOException {
    Configuration conf = TestConfigurations.getDefaultConf(tempFile.getAbsolutePath());
    StreamerUtil.createWriteClient(conf);
    FileSystemViewStorageConfig storageConfig = ViewStorageProperties.loadFromProperties(conf.getString(FlinkOptions.PATH));
    assertThat(storageConfig.getStorageType(), is(FileSystemViewStorageType.REMOTE_FIRST));
}
Also used : FileSystemViewStorageConfig(org.apache.hudi.common.table.view.FileSystemViewStorageConfig) Configuration(org.apache.flink.configuration.Configuration) Test(org.junit.jupiter.api.Test)

Example 7 with FileSystemViewStorageConfig

use of org.apache.hudi.common.table.view.FileSystemViewStorageConfig in project hudi by apache.

the class TestViewStorageProperties method testReadWriteProperties.

@Test
void testReadWriteProperties() throws IOException {
    String basePath = tempFile.getAbsolutePath();
    FileSystemViewStorageConfig config = FileSystemViewStorageConfig.newBuilder().withStorageType(FileSystemViewStorageType.SPILLABLE_DISK).withRemoteServerHost("host1").withRemoteServerPort(1234).build();
    ViewStorageProperties.createProperties(basePath, config);
    ViewStorageProperties.createProperties(basePath, config);
    ViewStorageProperties.createProperties(basePath, config);
    FileSystemViewStorageConfig readConfig = ViewStorageProperties.loadFromProperties(basePath);
    assertThat(readConfig.getStorageType(), is(FileSystemViewStorageType.SPILLABLE_DISK));
    assertThat(readConfig.getRemoteViewServerHost(), is("host1"));
    assertThat(readConfig.getRemoteViewServerPort(), is(1234));
}
Also used : FileSystemViewStorageConfig(org.apache.hudi.common.table.view.FileSystemViewStorageConfig) Test(org.junit.jupiter.api.Test)

Aggregations

FileSystemViewStorageConfig (org.apache.hudi.common.table.view.FileSystemViewStorageConfig)7 Test (org.junit.jupiter.api.Test)3 Configuration (org.apache.hadoop.conf.Configuration)2 HoodieFlinkWriteClient (org.apache.hudi.client.HoodieFlinkWriteClient)2 HoodieMetadataConfig (org.apache.hudi.common.config.HoodieMetadataConfig)2 HoodieLocalEngineContext (org.apache.hudi.common.engine.HoodieLocalEngineContext)2 HoodieWriteConfig (org.apache.hudi.config.HoodieWriteConfig)2 TimelineService (org.apache.hudi.timeline.service.TimelineService)2 IOException (java.io.IOException)1 Configuration (org.apache.flink.configuration.Configuration)1 Path (org.apache.hadoop.fs.Path)1 HoodieSparkEngineContext (org.apache.hudi.client.common.HoodieSparkEngineContext)1 HoodieCommonConfig (org.apache.hudi.common.config.HoodieCommonConfig)1 RemoteHoodieTableFileSystemView (org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView)1 JavaSparkContext (org.apache.spark.api.java.JavaSparkContext)1 BeforeEach (org.junit.jupiter.api.BeforeEach)1