Search in sources :

Example 6 with HoodieConfig

use of org.apache.hudi.common.config.HoodieConfig in project hudi by apache.

the class TestHoodieAWSCredentialsProviderFactory method testGetAWSCredentials.

@Test
public void testGetAWSCredentials() {
    HoodieConfig cfg = new HoodieConfig();
    cfg.setValue(HoodieAWSConfig.AWS_ACCESS_KEY, "random-access-key");
    cfg.setValue(HoodieAWSConfig.AWS_SECRET_KEY, "random-secret-key");
    cfg.setValue(HoodieAWSConfig.AWS_SESSION_TOKEN, "random-session-token");
    BasicSessionCredentials credentials = (BasicSessionCredentials) org.apache.hudi.aws.credentials.HoodieAWSCredentialsProviderFactory.getAwsCredentialsProvider(cfg.getProps()).getCredentials();
    assertEquals("random-access-key", credentials.getAWSAccessKeyId());
    assertEquals("random-secret-key", credentials.getAWSSecretKey());
    assertEquals("random-session-token", credentials.getSessionToken());
}
Also used : HoodieConfig(org.apache.hudi.common.config.HoodieConfig) BasicSessionCredentials(com.amazonaws.auth.BasicSessionCredentials) Test(org.junit.jupiter.api.Test)

Example 7 with HoodieConfig

use of org.apache.hudi.common.config.HoodieConfig in project hudi by apache.

the class TestUpgradeDowngrade method assertTableVersionFromPropertyFile.

private void assertTableVersionFromPropertyFile(HoodieTableVersion expectedVersion) throws IOException {
    Path propertyFile = new Path(metaClient.getMetaPath() + "/" + HoodieTableConfig.HOODIE_PROPERTIES_FILE);
    // Load the properties and verify
    FSDataInputStream fsDataInputStream = metaClient.getFs().open(propertyFile);
    HoodieConfig hoodieConfig = HoodieConfig.create(fsDataInputStream);
    fsDataInputStream.close();
    assertEquals(Integer.toString(expectedVersion.versionCode()), hoodieConfig.getString(HoodieTableConfig.VERSION));
}
Also used : Path(org.apache.hadoop.fs.Path) HoodieConfig(org.apache.hudi.common.config.HoodieConfig) FSDataInputStream(org.apache.hadoop.fs.FSDataInputStream)

Example 8 with HoodieConfig

use of org.apache.hudi.common.config.HoodieConfig in project hudi by apache.

the class TestUpgradeDowngradeCommand method assertTableVersionFromPropertyFile.

private void assertTableVersionFromPropertyFile() throws IOException {
    Path propertyFile = new Path(metaClient.getMetaPath() + "/" + HoodieTableConfig.HOODIE_PROPERTIES_FILE);
    // Load the properties and verify
    FSDataInputStream fsDataInputStream = metaClient.getFs().open(propertyFile);
    HoodieConfig hoodieConfig = HoodieConfig.create(fsDataInputStream);
    fsDataInputStream.close();
    assertEquals(Integer.toString(HoodieTableVersion.ZERO.versionCode()), hoodieConfig.getString(HoodieTableConfig.VERSION));
}
Also used : Path(org.apache.hadoop.fs.Path) HoodieConfig(org.apache.hudi.common.config.HoodieConfig) FSDataInputStream(org.apache.hadoop.fs.FSDataInputStream)

Aggregations

HoodieConfig (org.apache.hudi.common.config.HoodieConfig)8 Path (org.apache.hadoop.fs.Path)6 FSDataInputStream (org.apache.hadoop.fs.FSDataInputStream)3 FileSystem (org.apache.hadoop.fs.FileSystem)2 Test (org.junit.jupiter.api.Test)2 BasicSessionCredentials (com.amazonaws.auth.BasicSessionCredentials)1 Properties (java.util.Properties)1 FSDataOutputStream (org.apache.hadoop.fs.FSDataOutputStream)1 TypedProperties (org.apache.hudi.common.config.TypedProperties)1 HoodieRetryWrapperFileSystem (org.apache.hudi.common.fs.HoodieRetryWrapperFileSystem)1 HoodieWrapperFileSystem (org.apache.hudi.common.fs.HoodieWrapperFileSystem)1 HoodieDeltaStreamer (org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer)1 SqlQueryBasedTransformer (org.apache.hudi.utilities.transform.SqlQueryBasedTransformer)1 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)1