Examples with DFSPropertiesConfiguration org.apache.hudi.common.config.DFSPropertiesConfiguration used on opensource projects

Example 6 with DFSPropertiesConfiguration

use of org.apache.hudi.common.config.DFSPropertiesConfiguration in project hudi by apache.

the class UtilHelpers method readConfig.

public static DFSPropertiesConfiguration readConfig(Configuration hadoopConfig, Path cfgPath, List<String> overriddenProps) {
    DFSPropertiesConfiguration conf = new DFSPropertiesConfiguration(hadoopConfig, cfgPath);
    try {
        if (!overriddenProps.isEmpty()) {
            LOG.info("Adding overridden properties to file properties.");
            conf.addPropsFromStream(new BufferedReader(new StringReader(String.join("\n", overriddenProps))));
        }
    } catch (IOException ioe) {
        throw new HoodieIOException("Unexpected error adding config overrides", ioe);
    }
    return conf;
}

Example 7 with DFSPropertiesConfiguration

use of org.apache.hudi.common.config.DFSPropertiesConfiguration in project hudi by apache.

the class UtilHelpers method getConfig.

public static DFSPropertiesConfiguration getConfig(List<String> overriddenProps) {
    DFSPropertiesConfiguration conf = new DFSPropertiesConfiguration();
    try {
        if (!overriddenProps.isEmpty()) {
            LOG.info("Adding overridden properties to file properties.");
            conf.addPropsFromStream(new BufferedReader(new StringReader(String.join("\n", overriddenProps))));
        }
    } catch (IOException ioe) {
        throw new HoodieIOException("Unexpected error adding config overrides", ioe);
    }
    return conf;
}

Example 8 with DFSPropertiesConfiguration

use of org.apache.hudi.common.config.DFSPropertiesConfiguration in project hudi by apache.

the class StreamerUtil method readConfig.

/**
 * Read config from properties file (`--props` option) and cmd line (`--hoodie-conf` option).
 */
public static DFSPropertiesConfiguration readConfig(org.apache.hadoop.conf.Configuration hadoopConfig, Path cfgPath, List<String> overriddenProps) {
    DFSPropertiesConfiguration conf = new DFSPropertiesConfiguration(hadoopConfig, cfgPath);
    try {
        if (!overriddenProps.isEmpty()) {
            LOG.info("Adding overridden properties to file properties.");
            conf.addPropsFromStream(new BufferedReader(new StringReader(String.join("\n", overriddenProps))));
        }
    } catch (IOException ioe) {
        throw new HoodieIOException("Unexpected error adding config overrides", ioe);
    }
    return conf;
}