Examples with ConfigSource org.embulk.config.ConfigSource used on opensource projects

Example 16 with ConfigSource

use of org.embulk.config.ConfigSource in project embulk by embulk.

the class TestRenameFilterPlugin method checkTruncateRuleDefault.

@Test
public void checkTruncateRuleDefault() {
    final String[] original = { "123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890" };
    final String[] expected = { "12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678" };
    ConfigSource config = Exec.newConfigSource().set("rules", ImmutableList.of(ImmutableMap.of("rule", "truncate")));
    renameAndCheckSchema(config, original, expected);
}

Example 17 with ConfigSource

use of org.embulk.config.ConfigSource in project embulk by embulk.

the class TestRenameFilterPlugin method checkTruncateRule.

@Test
public void checkTruncateRule() {
    final String[] original = { "foo", "bar", "gj", "foobar", "foobarbaz" };
    final String[] expected = { "foo", "bar", "gj", "foo", "foo" };
    ConfigSource config = Exec.newConfigSource().set("rules", ImmutableList.of(ImmutableMap.of("rule", "truncate", "max_length", "3")));
    renameAndCheckSchema(config, original, expected);
}

Example 18 with ConfigSource

use of org.embulk.config.ConfigSource in project embulk by embulk.

the class TestTimestampFormatterParser method testDefaultDate.

@Test
public void testDefaultDate() throws Exception {
    ConfigSource config = Exec.newConfigSource().set("default_timestamp_format", "%H:%M:%S %Z").set("default_date", "2016-02-03");
    ParserTestTask ptask = config.loadConfig(ParserTestTask.class);
    TimestampParser parser = TimestampParserLegacy.createTimestampParserForTesting(ptask);
    assertEquals(Timestamp.ofEpochSecond(1454467589, 0), parser.parse("02:46:29 +0000"));
}

Example 19 with ConfigSource

use of org.embulk.config.ConfigSource in project embulk by embulk.

the class EmbulkSetup method setup.

public static EmbulkRunner setup(final Map<String, Object> systemConfigGiven) {
    final HashMap<String, Object> systemConfigModified = new HashMap<String, Object>(systemConfigGiven);
    // Calling ObjectMapper simply just as a formatter here so that it does not depend much on Jackson.
    // Do not leak the Jackson object outside.
    final ObjectMapper jacksonObjectMapper = new ObjectMapper();
    final String systemConfigJson;
    try {
        systemConfigJson = jacksonObjectMapper.writeValueAsString(systemConfigModified);
    } catch (JsonProcessingException ex) {
        throw new RuntimeException(ex);
    }
    final EmbulkEmbed.Bootstrap bootstrap = new org.embulk.EmbulkEmbed.Bootstrap();
    final ConfigSource systemConfig = bootstrap.getSystemConfigLoader().fromJsonString(systemConfigJson);
    bootstrap.setSystemConfig(systemConfig);
    // see embulk-core/src/main/java/org/embulk/jruby/JRubyScriptingModule.
    final EmbulkEmbed embed = bootstrap.initialize();
    return new EmbulkRunner(embed);
}

Example 20 with ConfigSource

use of org.embulk.config.ConfigSource in project embulk by embulk.

the class GuessExecutor method doGuess.

private ConfigDiff doGuess(ConfigSource config) {
    ConfigSource inputConfig = config.getNested("in");
    ConfigSource execConfig = config.getNestedOrGetEmpty("exec");
    InputPlugin input = newInputPlugin(inputConfig);
    ConfigDiff inputGuessed;
    if (input instanceof ConfigurableGuessInputPlugin) {
        inputGuessed = ((ConfigurableGuessInputPlugin) input).guess(execConfig, inputConfig);
    } else {
        try {
            inputGuessed = input.guess(inputConfig);
        } catch (AbstractMethodError ex) {
            // for backward compatibility with embulk v0.4 interface
            throw new UnsupportedOperationException(input.getClass().getSimpleName() + ".guess(ConfigSource) is not implemented. This input plugin does not support guessing.");
        }
    }
    ConfigDiff wrapped = Exec.newConfigDiff();
    wrapped.getNestedOrSetEmpty("in").merge(inputGuessed);
    return wrapped;
}