Search in sources :

Example 1 with ClientHelper

use of org.apache.samza.job.yarn.ClientHelper in project samza by apache.

the class YarnJobValidationTool method main.

public static void main(String[] args) throws Exception {
    CommandLine cmdline = new CommandLine();
    OptionParser parser = cmdline.parser();
    OptionSpec<String> validatorOpt = parser.accepts("metrics-validator", "The metrics validator class.").withOptionalArg().ofType(String.class).describedAs("com.foo.bar.ClassName");
    OptionSet options = cmdline.parser().parse(args);
    Config config = cmdline.loadConfig(options);
    MetricsValidator validator = null;
    if (options.has(validatorOpt)) {
        String validatorClass = options.valueOf(validatorOpt);
        validator = ReflectionUtil.getObj(validatorClass, MetricsValidator.class);
    }
    YarnConfiguration hadoopConfig = new YarnConfiguration();
    hadoopConfig.set("fs.http.impl", HttpFileSystem.class.getName());
    hadoopConfig.set("fs.https.impl", HttpFileSystem.class.getName());
    ClientHelper clientHelper = new ClientHelper(hadoopConfig);
    new YarnJobValidationTool(new JobConfig(config), clientHelper.yarnClient(), validator).run();
}
Also used : CommandLine(org.apache.samza.util.CommandLine) ClientHelper(org.apache.samza.job.yarn.ClientHelper) YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) JobConfig(org.apache.samza.config.JobConfig) SetConfig(org.apache.samza.coordinator.stream.messages.SetConfig) Config(org.apache.samza.config.Config) HttpFileSystem(org.apache.samza.util.hadoop.HttpFileSystem) OptionSet(joptsimple.OptionSet) OptionParser(joptsimple.OptionParser) JobConfig(org.apache.samza.config.JobConfig) MetricsValidator(org.apache.samza.metrics.MetricsValidator)

Aggregations

OptionParser (joptsimple.OptionParser)1 OptionSet (joptsimple.OptionSet)1 YarnConfiguration (org.apache.hadoop.yarn.conf.YarnConfiguration)1 Config (org.apache.samza.config.Config)1 JobConfig (org.apache.samza.config.JobConfig)1 SetConfig (org.apache.samza.coordinator.stream.messages.SetConfig)1 ClientHelper (org.apache.samza.job.yarn.ClientHelper)1 MetricsValidator (org.apache.samza.metrics.MetricsValidator)1 CommandLine (org.apache.samza.util.CommandLine)1 HttpFileSystem (org.apache.samza.util.hadoop.HttpFileSystem)1