Search in sources :

Example 1 with StramAppLauncher

use of com.datatorrent.stram.client.StramAppLauncher in project apex-core by apache.

the class ApexCli method getStramAppLauncher.

@SuppressWarnings("unused")
private StramAppLauncher getStramAppLauncher(String jarfileUri, Configuration config, boolean ignorePom) throws Exception {
    URI uri = new URI(jarfileUri);
    String scheme = uri.getScheme();
    StramAppLauncher appLauncher = null;
    if (scheme == null || scheme.equals("file")) {
        File jf = new File(uri.getPath());
        appLauncher = new StramAppLauncher(jf, config);
    } else {
        try (FileSystem tmpFs = FileSystem.newInstance(uri, conf)) {
            Path path = new Path(uri.getPath());
            appLauncher = new StramAppLauncher(tmpFs, path, config);
        }
    }
    if (appLauncher != null) {
        if (verboseLevel > 0) {
            System.err.print(appLauncher.getMvnBuildClasspathOutput());
        }
        return appLauncher;
    } else {
        throw new CliException("Scheme " + scheme + " not supported.");
    }
}
Also used : Path(org.apache.hadoop.fs.Path) StramAppLauncher(com.datatorrent.stram.client.StramAppLauncher) FileSystem(org.apache.hadoop.fs.FileSystem) URI(java.net.URI) File(java.io.File)

Example 2 with StramAppLauncher

use of com.datatorrent.stram.client.StramAppLauncher in project apex-core by apache.

the class YarnAppLauncherImpl method launchApp.

@Override
public YarnAppHandleImpl launchApp(final StreamingApplication app, Configuration conf, Attribute.AttributeMap launchParameters) throws LauncherException {
    if (launchParameters != null) {
        for (Map.Entry<Attribute<?>, Object> entry : launchParameters.entrySet()) {
            String property = propMapping.get(entry.getKey());
            if (property != null) {
                setConfiguration(conf, property, entry.getValue());
            }
        }
    }
    try {
        String name = app.getClass().getName();
        StramAppLauncher appLauncher = new StramAppLauncher(name, conf);
        appLauncher.loadDependencies();
        StreamingAppFactory appFactory = new StreamingAppFactory(name, app.getClass()) {

            @Override
            public LogicalPlan createApp(LogicalPlanConfiguration planConfig) {
                return super.createApp(app, planConfig);
            }
        };
        ApplicationId appId = appLauncher.launchApp(appFactory);
        appLauncher.resetContextClassLoader();
        return new YarnAppHandleImpl(appId, conf);
    } catch (Exception ex) {
        throw new LauncherException(ex);
    }
}
Also used : LogicalPlanConfiguration(com.datatorrent.stram.plan.logical.LogicalPlanConfiguration) StramAppLauncher(com.datatorrent.stram.client.StramAppLauncher) Attribute(com.datatorrent.api.Attribute) StreamingAppFactory(org.apache.apex.engine.util.StreamingAppFactory) ApplicationId(org.apache.hadoop.yarn.api.records.ApplicationId) HashMap(java.util.HashMap) Map(java.util.Map) IOException(java.io.IOException) YarnException(org.apache.hadoop.yarn.exceptions.YarnException)

Aggregations

StramAppLauncher (com.datatorrent.stram.client.StramAppLauncher)2 Attribute (com.datatorrent.api.Attribute)1 LogicalPlanConfiguration (com.datatorrent.stram.plan.logical.LogicalPlanConfiguration)1 File (java.io.File)1 IOException (java.io.IOException)1 URI (java.net.URI)1 HashMap (java.util.HashMap)1 Map (java.util.Map)1 StreamingAppFactory (org.apache.apex.engine.util.StreamingAppFactory)1 FileSystem (org.apache.hadoop.fs.FileSystem)1 Path (org.apache.hadoop.fs.Path)1 ApplicationId (org.apache.hadoop.yarn.api.records.ApplicationId)1 YarnException (org.apache.hadoop.yarn.exceptions.YarnException)1