Search in sources :

Example 1 with ConstructorAndPublicMethodsCliObjectFactory

use of org.apache.gobblin.runtime.cli.ConstructorAndPublicMethodsCliObjectFactory in project incubator-gobblin by apache.

the class ConfigClientCLI method run.

@Override
public void run(String[] args) throws Exception {
    CliObjectFactory<Command> factory = new ConstructorAndPublicMethodsCliObjectFactory<>(Command.class);
    Command command = factory.buildObject(args, 1, true, args[0]);
    ConfigClient configClient = ConfigClient.createConfigClient(VersionStabilityPolicy.READ_FRESHEST);
    if (command.resolvedConfig) {
        Config resolvedConfig = configClient.getConfig(command.uri);
        System.out.println(resolvedConfig.root().render(ConfigRenderOptions.defaults()));
    }
}
Also used : Config(com.typesafe.config.Config) ConstructorAndPublicMethodsCliObjectFactory(org.apache.gobblin.runtime.cli.ConstructorAndPublicMethodsCliObjectFactory)

Example 2 with ConstructorAndPublicMethodsCliObjectFactory

use of org.apache.gobblin.runtime.cli.ConstructorAndPublicMethodsCliObjectFactory in project incubator-gobblin by apache.

the class StateStoreMigrationCli method run.

@Override
public void run(String[] args) throws Exception {
    CliObjectFactory<Command> factory = new ConstructorAndPublicMethodsCliObjectFactory<>(Command.class);
    Command command = factory.buildObject(args, 1, true, args[0]);
    FileSystem fs = FileSystem.get(new Configuration());
    FSDataInputStream inputStream = fs.open(command.path);
    Config config = ConfigFactory.parseReader(new InputStreamReader(inputStream, Charset.defaultCharset()));
    Preconditions.checkNotNull(config.getObject(SOURCE_KEY));
    Preconditions.checkNotNull(config.getObject(DESTINATION_KEY));
    DatasetStateStore dstDatasetStateStore = DatasetStateStore.buildDatasetStateStore(config.getConfig(DESTINATION_KEY));
    DatasetStateStore srcDatasetStateStore = DatasetStateStore.buildDatasetStateStore(config.getConfig(SOURCE_KEY));
    Map<String, JobState.DatasetState> map;
    // if migrating state for all jobs then list the store names (job names) and copy the current jst files
    if (ConfigUtils.getBoolean(config, MIGRATE_ALL_JOBS, Boolean.valueOf(DEFAULT_MIGRATE_ALL_JOBS))) {
        List<String> jobNames = srcDatasetStateStore.getStoreNames(Predicates.alwaysTrue());
        for (String jobName : jobNames) {
            migrateStateForJob(srcDatasetStateStore, dstDatasetStateStore, jobName, command.deleteSourceStateStore);
        }
    } else {
        Preconditions.checkNotNull(config.getString(JOB_NAME_KEY));
        migrateStateForJob(srcDatasetStateStore, dstDatasetStateStore, config.getString(JOB_NAME_KEY), command.deleteSourceStateStore);
    }
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) InputStreamReader(java.io.InputStreamReader) Config(com.typesafe.config.Config) DatasetStateStore(org.apache.gobblin.metastore.DatasetStateStore) FileSystem(org.apache.hadoop.fs.FileSystem) FSDataInputStream(org.apache.hadoop.fs.FSDataInputStream) ConstructorAndPublicMethodsCliObjectFactory(org.apache.gobblin.runtime.cli.ConstructorAndPublicMethodsCliObjectFactory)

Aggregations

Config (com.typesafe.config.Config)2 ConstructorAndPublicMethodsCliObjectFactory (org.apache.gobblin.runtime.cli.ConstructorAndPublicMethodsCliObjectFactory)2 InputStreamReader (java.io.InputStreamReader)1 DatasetStateStore (org.apache.gobblin.metastore.DatasetStateStore)1 Configuration (org.apache.hadoop.conf.Configuration)1 FSDataInputStream (org.apache.hadoop.fs.FSDataInputStream)1 FileSystem (org.apache.hadoop.fs.FileSystem)1