Search in sources :

Example 1 with RuntimeJobDetail

use of io.cdap.cdap.runtime.spi.runtimejob.RuntimeJobDetail in project cdap by caskdata.

the class AbstractDataprocProvisioner method deleteClusterWithStatus.

@Override
public final ClusterStatus deleteClusterWithStatus(ProvisionerContext context, Cluster cluster) throws Exception {
    Map<String, String> properties = createContextProperties(context);
    DataprocConf conf = DataprocConf.create(properties);
    RuntimeJobManager jobManager = getRuntimeJobManager(context).orElse(null);
    // Also cleanup files created by the job run.
    if (jobManager != null) {
        try {
            RuntimeJobDetail jobDetail = jobManager.getDetail(context.getProgramRunInfo()).orElse(null);
            if (jobDetail != null && !jobDetail.getStatus().isTerminated()) {
                return ClusterStatus.RUNNING;
            }
        } finally {
            jobManager.close();
        }
        Storage storageClient = StorageOptions.newBuilder().setProjectId(conf.getProjectId()).setCredentials(conf.getDataprocCredentials()).build().getService();
        DataprocUtils.deleteGCSPath(storageClient, properties.get(BUCKET), DataprocUtils.CDAP_GCS_ROOT + "/" + context.getProgramRunInfo().getRun());
    }
    doDeleteCluster(context, cluster, conf);
    return ClusterStatus.DELETING;
}
Also used : Storage(com.google.cloud.storage.Storage) RuntimeJobManager(io.cdap.cdap.runtime.spi.runtimejob.RuntimeJobManager) DataprocRuntimeJobManager(io.cdap.cdap.runtime.spi.runtimejob.DataprocRuntimeJobManager) RuntimeJobDetail(io.cdap.cdap.runtime.spi.runtimejob.RuntimeJobDetail)

Example 2 with RuntimeJobDetail

use of io.cdap.cdap.runtime.spi.runtimejob.RuntimeJobDetail in project cdap by caskdata.

the class TetheringRuntimeJobManager method stop.

@Override
public void stop(ProgramRunInfo programRunInfo) throws Exception {
    RuntimeJobDetail jobDetail = getDetail(programRunInfo).orElse(null);
    if (jobDetail == null) {
        return;
    }
    RuntimeJobStatus status = jobDetail.getStatus();
    if (status.isTerminated() || status == RuntimeJobStatus.STOPPING) {
        return;
    }
    LOG.debug("Stopping run {} with following configurations: tethered instance name {}, tethered namespace {}.", programRunInfo.getRun(), tetheredInstanceName, tetheredNamespace);
    byte[] payload = Bytes.toBytes(GSON.toJson(programRunInfo));
    TetheringControlMessage message = new TetheringControlMessage(TetheringControlMessage.Type.STOP_PIPELINE, payload);
    publishToControlChannel(message);
}
Also used : RuntimeJobStatus(io.cdap.cdap.runtime.spi.runtimejob.RuntimeJobStatus) TetheringControlMessage(io.cdap.cdap.internal.tethering.TetheringControlMessage) RuntimeJobDetail(io.cdap.cdap.runtime.spi.runtimejob.RuntimeJobDetail)

Aggregations

RuntimeJobDetail (io.cdap.cdap.runtime.spi.runtimejob.RuntimeJobDetail)2 Storage (com.google.cloud.storage.Storage)1 TetheringControlMessage (io.cdap.cdap.internal.tethering.TetheringControlMessage)1 DataprocRuntimeJobManager (io.cdap.cdap.runtime.spi.runtimejob.DataprocRuntimeJobManager)1 RuntimeJobManager (io.cdap.cdap.runtime.spi.runtimejob.RuntimeJobManager)1 RuntimeJobStatus (io.cdap.cdap.runtime.spi.runtimejob.RuntimeJobStatus)1