Search in sources :

Example 6 with LocalState

use of backtype.storm.utils.LocalState in project jstorm by alibaba.

the class WorkerHeartbeatRunable method doHeartbeat.

/**
     * do hearbeat, update LocalState
     * 
     * @throws IOException
     */
public void doHeartbeat() throws IOException {
    int currtime = TimeUtils.current_time_secs();
    WorkerHeartbeat hb = new WorkerHeartbeat(currtime, topologyId, task_ids, port);
    LOG.debug("Doing heartbeat:" + worker_id + ",port:" + port + ",hb" + hb.toString());
    LocalState state = getWorkerState();
    state.put(Common.LS_WORKER_HEARTBEAT, hb);
}
Also used : WorkerHeartbeat(com.alibaba.jstorm.daemon.worker.WorkerHeartbeat) LocalState(backtype.storm.utils.LocalState)

Example 7 with LocalState

use of backtype.storm.utils.LocalState in project storm-mesos by nathanmarz.

the class MesosSupervisor method prepare.

@Override
public void prepare(Map conf, String localDir) {
    try {
        _state = new LocalState(localDir);
    } catch (IOException e) {
        throw new RuntimeException(e);
    }
    Semaphore initter = new Semaphore(0);
    _executor = new StormExecutor(initter);
    _driver = new MesosExecutorDriver(_executor);
    _driver.start();
    LOG.info("Waiting for executor to initialize...");
    try {
        initter.acquire();
    } catch (InterruptedException e) {
        throw new RuntimeException(e);
    }
    LOG.info("Executor initialized...");
    Thread suicide = new SuicideDetector(conf);
    suicide.setDaemon(true);
    suicide.start();
}
Also used : MesosExecutorDriver(org.apache.mesos.MesosExecutorDriver) LocalState(backtype.storm.utils.LocalState) IOException(java.io.IOException) Semaphore(java.util.concurrent.Semaphore)

Example 8 with LocalState

use of backtype.storm.utils.LocalState in project storm-mesos by nathanmarz.

the class MesosNimbus method prepare.

@Override
public void prepare(Map conf, String localDir) {
    try {
        _conf = conf;
        _state = new LocalState(localDir);
        String id = (String) _state.get(FRAMEWORK_ID);
        _allowedHosts = listIntoSet((List) _conf.get(CONF_MESOS_ALLOWED_HOSTS));
        _disallowedHosts = listIntoSet((List) _conf.get(CONF_MESOS_DISALLOWED_HOSTS));
        Semaphore initter = new Semaphore(0);
        _scheduler = new NimbusScheduler(initter);
        Number failoverTimeout = (Number) conf.get(CONF_MASTER_FAILOVER_TIMEOUT_SECS);
        if (failoverTimeout == null)
            failoverTimeout = 3600;
        FrameworkInfo.Builder finfo = FrameworkInfo.newBuilder().setName("Storm-0.8.0").setFailoverTimeout(failoverTimeout.doubleValue()).setUser("");
        if (id != null) {
            finfo.setId(FrameworkID.newBuilder().setValue(id).build());
        }
        MesosSchedulerDriver driver = new MesosSchedulerDriver(_scheduler, finfo.build(), (String) conf.get(CONF_MASTER_URL));
        driver.start();
        LOG.info("Waiting for scheduler to initialize...");
        initter.acquire();
        LOG.info("Scheduler initialized...");
    } catch (IOException e) {
        throw new RuntimeException(e);
    } catch (InterruptedException e) {
        throw new RuntimeException(e);
    }
}
Also used : FrameworkInfo(org.apache.mesos.Protos.FrameworkInfo) LocalState(backtype.storm.utils.LocalState) ByteString(com.google.protobuf.ByteString) Semaphore(java.util.concurrent.Semaphore) IOException(java.io.IOException) MesosSchedulerDriver(org.apache.mesos.MesosSchedulerDriver)

Example 9 with LocalState

use of backtype.storm.utils.LocalState in project jstorm by alibaba.

the class StormConfig method supervisorState.

public static LocalState supervisorState(Map conf) throws IOException {
    LocalState localState = null;
    try {
        String localstateDir = supervisor_local_dir(conf) + FILE_SEPERATEOR + "localstate";
        FileUtils.forceMkdir(new File(localstateDir));
        localState = new LocalState(localstateDir);
    } catch (IOException e) {
        LOG.error("Failed to create supervisor LocalState", e);
        throw e;
    }
    return localState;
}
Also used : LocalState(backtype.storm.utils.LocalState) IOException(java.io.IOException) File(java.io.File)

Aggregations

LocalState (backtype.storm.utils.LocalState)9 IOException (java.io.IOException)3 File (java.io.File)2 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)2 Semaphore (java.util.concurrent.Semaphore)2 AsyncLoopThread (com.alibaba.jstorm.callback.AsyncLoopThread)1 StormClusterState (com.alibaba.jstorm.cluster.StormClusterState)1 SupervisorRefreshConfig (com.alibaba.jstorm.config.SupervisorRefreshConfig)1 WorkerHeartbeat (com.alibaba.jstorm.daemon.worker.WorkerHeartbeat)1 WorkerReportError (com.alibaba.jstorm.daemon.worker.WorkerReportError)1 EventManagerImp (com.alibaba.jstorm.event.EventManagerImp)1 EventManagerPusher (com.alibaba.jstorm.event.EventManagerPusher)1 Pair (com.alibaba.jstorm.utils.Pair)1 ByteString (com.google.protobuf.ByteString)1 HashMap (java.util.HashMap)1 HashSet (java.util.HashSet)1 Map (java.util.Map)1 Vector (java.util.Vector)1 MesosExecutorDriver (org.apache.mesos.MesosExecutorDriver)1 MesosSchedulerDriver (org.apache.mesos.MesosSchedulerDriver)1