Search in sources :

Example 1 with RecoverableRpcProxy

use of com.datatorrent.stram.RecoverableRpcProxy in project apex-core by apache.

the class StreamingContainer method main.

/**
   * Initialize container. Establishes heartbeat connection to the master
   * distribute through the callback address provided on the command line. Deploys
   * initial modules, then enters the heartbeat loop, which will only terminate
   * once container receives shutdown request from the master. On shutdown,
   * after exiting heartbeat loop, shutdown all modules and terminate
   * processing threads.
   *
   * @param args
   * @throws Throwable
   */
public static void main(String[] args) throws Throwable {
    StdOutErrLog.tieSystemOutAndErrToLog();
    logger.debug("PID: " + System.getenv().get("JVM_PID"));
    logger.info("Child starting with classpath: {}", System.getProperty("java.class.path"));
    String appPath = System.getProperty(PROP_APP_PATH);
    if (appPath == null) {
        logger.error("{} not set in container environment.", PROP_APP_PATH);
        System.exit(1);
    }
    // interpreted as unrecoverable container failure
    int exitStatus = 1;
    RecoverableRpcProxy rpcProxy = null;
    StreamingContainerUmbilicalProtocol umbilical = null;
    final String childId = System.getProperty(StreamingApplication.DT_PREFIX + "cid");
    try {
        rpcProxy = new RecoverableRpcProxy(appPath, new Configuration());
        umbilical = rpcProxy.getProxy();
        StreamingContainerContext ctx = umbilical.getInitContext(childId);
        StreamingContainer stramChild = new StreamingContainer(childId, umbilical);
        logger.debug("Container Context = {}", ctx);
        stramChild.setup(ctx);
        try {
            /* main thread enters heartbeat loop */
            stramChild.heartbeatLoop();
            exitStatus = 0;
        } finally {
            stramChild.teardown();
        }
    } catch (Error | Exception e) {
        LogFileInformation logFileInfo = LoggerUtil.getLogFileInformation();
        logger.error("Fatal {} in container!", (e instanceof Error) ? "Error" : "Exception", e);
        /* Report back any failures, for diagnostic purposes */
        try {
            umbilical.reportError(childId, null, ExceptionUtils.getStackTrace(e), logFileInfo);
        } catch (Exception ex) {
            logger.debug("Fail to log", ex);
        }
    } finally {
        if (rpcProxy != null) {
            rpcProxy.close();
        }
        DefaultMetricsSystem.shutdown();
        logger.info("Exit status for container: {}", exitStatus);
        LogManager.shutdown();
        if (exitStatus != 0) {
            System.exit(exitStatus);
        }
    }
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) BusConfiguration(net.engio.mbassy.bus.config.BusConfiguration) LogFileInformation(org.apache.apex.log.LogFileInformation) StreamingContainerContext(com.datatorrent.stram.api.StreamingContainerUmbilicalProtocol.StreamingContainerContext) RecoverableRpcProxy(com.datatorrent.stram.RecoverableRpcProxy) StreamingContainerUmbilicalProtocol(com.datatorrent.stram.api.StreamingContainerUmbilicalProtocol) Checkpoint(com.datatorrent.stram.api.Checkpoint) IOException(java.io.IOException) UnknownHostException(java.net.UnknownHostException)

Aggregations

RecoverableRpcProxy (com.datatorrent.stram.RecoverableRpcProxy)1 Checkpoint (com.datatorrent.stram.api.Checkpoint)1 StreamingContainerUmbilicalProtocol (com.datatorrent.stram.api.StreamingContainerUmbilicalProtocol)1 StreamingContainerContext (com.datatorrent.stram.api.StreamingContainerUmbilicalProtocol.StreamingContainerContext)1 IOException (java.io.IOException)1 UnknownHostException (java.net.UnknownHostException)1 BusConfiguration (net.engio.mbassy.bus.config.BusConfiguration)1 LogFileInformation (org.apache.apex.log.LogFileInformation)1 Configuration (org.apache.hadoop.conf.Configuration)1 YarnConfiguration (org.apache.hadoop.yarn.conf.YarnConfiguration)1