Search in sources :

Example 1 with PipelineID

use of org.apache.hadoop.hdds.scm.pipeline.PipelineID in project ozone by apache.

the class XceiverServerRatis method triggerPipelineClose.

private void triggerPipelineClose(RaftGroupId groupId, String detail, ClosePipelineInfo.Reason reasonCode, boolean triggerHB) {
    PipelineID pipelineID = PipelineID.valueOf(groupId.getUuid());
    ClosePipelineInfo.Builder closePipelineInfo = ClosePipelineInfo.newBuilder().setPipelineID(pipelineID.getProtobuf()).setReason(reasonCode).setDetailedReason(detail);
    PipelineAction action = PipelineAction.newBuilder().setClosePipeline(closePipelineInfo).setAction(PipelineAction.Action.CLOSE).build();
    context.addPipelineActionIfAbsent(action);
    // wait for the next HB timeout or right away?
    if (triggerHB) {
        context.getParent().triggerHeartbeat();
    }
    LOG.error("pipeline Action {} on pipeline {}.Reason : {}", action.getAction(), pipelineID, action.getClosePipeline().getDetailedReason());
}
Also used : ClosePipelineInfo(org.apache.hadoop.hdds.protocol.proto.StorageContainerDatanodeProtocolProtos.ClosePipelineInfo) PipelineID(org.apache.hadoop.hdds.scm.pipeline.PipelineID) PipelineAction(org.apache.hadoop.hdds.protocol.proto.StorageContainerDatanodeProtocolProtos.PipelineAction)

Example 2 with PipelineID

use of org.apache.hadoop.hdds.scm.pipeline.PipelineID in project ozone by apache.

the class XceiverServerRatis method getPipelineIds.

@VisibleForTesting
public List<PipelineID> getPipelineIds() {
    Iterable<RaftGroupId> gids = server.getGroupIds();
    List<PipelineID> pipelineIDs = new ArrayList<>();
    for (RaftGroupId groupId : gids) {
        pipelineIDs.add(PipelineID.valueOf(groupId.getUuid()));
        LOG.info("pipeline id {}", PipelineID.valueOf(groupId.getUuid()));
    }
    return pipelineIDs;
}
Also used : ArrayList(java.util.ArrayList) PipelineID(org.apache.hadoop.hdds.scm.pipeline.PipelineID) RaftGroupId(org.apache.ratis.protocol.RaftGroupId) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 3 with PipelineID

use of org.apache.hadoop.hdds.scm.pipeline.PipelineID in project ozone by apache.

the class ClosePipelineCommandHandler method handle.

/**
 * Handles a given SCM command.
 *
 * @param command           - SCM Command
 * @param ozoneContainer    - Ozone Container.
 * @param context           - Current Context.
 * @param connectionManager - The SCMs that we are talking to.
 */
@Override
public void handle(SCMCommand command, OzoneContainer ozoneContainer, StateContext context, SCMConnectionManager connectionManager) {
    invocationCount.incrementAndGet();
    final long startTime = Time.monotonicNow();
    final DatanodeDetails dn = context.getParent().getDatanodeDetails();
    ClosePipelineCommand closePipelineCommand = (ClosePipelineCommand) command;
    final PipelineID pipelineID = closePipelineCommand.getPipelineID();
    final HddsProtos.PipelineID pipelineIdProto = pipelineID.getProtobuf();
    try {
        XceiverServerSpi server = ozoneContainer.getWriteChannel();
        if (server.isExist(pipelineIdProto)) {
            server.removeGroup(pipelineIdProto);
            LOG.info("Close Pipeline {} command on datanode {}.", pipelineID, dn.getUuidString());
        } else {
            LOG.debug("Ignoring close pipeline command for pipeline {} " + "as it does not exist", pipelineID);
        }
    } catch (IOException e) {
        LOG.error("Can't close pipeline {}", pipelineID, e);
    } finally {
        long endTime = Time.monotonicNow();
        totalTime += endTime - startTime;
    }
}
Also used : ClosePipelineCommand(org.apache.hadoop.ozone.protocol.commands.ClosePipelineCommand) HddsProtos(org.apache.hadoop.hdds.protocol.proto.HddsProtos) DatanodeDetails(org.apache.hadoop.hdds.protocol.DatanodeDetails) PipelineID(org.apache.hadoop.hdds.scm.pipeline.PipelineID) IOException(java.io.IOException) XceiverServerSpi(org.apache.hadoop.ozone.container.common.transport.server.XceiverServerSpi)

Example 4 with PipelineID

use of org.apache.hadoop.hdds.scm.pipeline.PipelineID in project ozone by apache.

the class TestFreonWithPipelineDestroy method destroyPipeline.

private void destroyPipeline() throws Exception {
    XceiverServerSpi server = cluster.getHddsDatanodes().get(0).getDatanodeStateMachine().getContainer().getWriteChannel();
    StorageContainerDatanodeProtocolProtos.PipelineReport report = server.getPipelineReport().get(0);
    PipelineID id = PipelineID.getFromProtobuf(report.getPipelineID());
    PipelineManager pipelineManager = cluster.getStorageContainerManager().getPipelineManager();
    Pipeline pipeline = pipelineManager.getPipeline(id);
    pipelineManager.closePipeline(pipeline, false);
}
Also used : PipelineManager(org.apache.hadoop.hdds.scm.pipeline.PipelineManager) PipelineID(org.apache.hadoop.hdds.scm.pipeline.PipelineID) StorageContainerDatanodeProtocolProtos(org.apache.hadoop.hdds.protocol.proto.StorageContainerDatanodeProtocolProtos) XceiverServerSpi(org.apache.hadoop.ozone.container.common.transport.server.XceiverServerSpi) Pipeline(org.apache.hadoop.hdds.scm.pipeline.Pipeline)

Example 5 with PipelineID

use of org.apache.hadoop.hdds.scm.pipeline.PipelineID in project ozone by apache.

the class SCMNodeManager method getPeerList.

@Override
public Collection<DatanodeDetails> getPeerList(DatanodeDetails dn) {
    HashSet<DatanodeDetails> dns = new HashSet<>();
    Preconditions.checkNotNull(dn);
    Set<PipelineID> pipelines = nodeStateManager.getPipelineByDnID(dn.getUuid());
    PipelineManager pipelineManager = scmContext.getScm().getPipelineManager();
    if (!pipelines.isEmpty()) {
        pipelines.forEach(id -> {
            try {
                Pipeline pipeline = pipelineManager.getPipeline(id);
                List<DatanodeDetails> peers = pipeline.getNodes();
                dns.addAll(peers);
            } catch (PipelineNotFoundException pnfe) {
            // ignore the pipeline not found exception here
            }
        });
    }
    // renove self node from the set
    dns.remove(dn);
    return dns;
}
Also used : DatanodeDetails(org.apache.hadoop.hdds.protocol.DatanodeDetails) PipelineManager(org.apache.hadoop.hdds.scm.pipeline.PipelineManager) PipelineID(org.apache.hadoop.hdds.scm.pipeline.PipelineID) PipelineNotFoundException(org.apache.hadoop.hdds.scm.pipeline.PipelineNotFoundException) HashSet(java.util.HashSet) Pipeline(org.apache.hadoop.hdds.scm.pipeline.Pipeline)

Aggregations

PipelineID (org.apache.hadoop.hdds.scm.pipeline.PipelineID)35 Pipeline (org.apache.hadoop.hdds.scm.pipeline.Pipeline)15 Test (org.junit.Test)13 IOException (java.io.IOException)12 DatanodeDetails (org.apache.hadoop.hdds.protocol.DatanodeDetails)11 HddsProtos (org.apache.hadoop.hdds.protocol.proto.HddsProtos)6 PipelineNotFoundException (org.apache.hadoop.hdds.scm.pipeline.PipelineNotFoundException)6 XceiverServerSpi (org.apache.hadoop.ozone.container.common.transport.server.XceiverServerSpi)5 ArrayList (java.util.ArrayList)4 HashMap (java.util.HashMap)4 UUID (java.util.UUID)4 OzoneConfiguration (org.apache.hadoop.hdds.conf.OzoneConfiguration)4 List (java.util.List)3 ContainerID (org.apache.hadoop.hdds.scm.container.ContainerID)3 RaftGroup (org.apache.ratis.protocol.RaftGroup)3 VisibleForTesting (com.google.common.annotations.VisibleForTesting)2 ConfigurationSource (org.apache.hadoop.hdds.conf.ConfigurationSource)2 MockDatanodeDetails (org.apache.hadoop.hdds.protocol.MockDatanodeDetails)2 ClosePipelineInfo (org.apache.hadoop.hdds.protocol.proto.StorageContainerDatanodeProtocolProtos.ClosePipelineInfo)2 CreatePipelineCommandProto (org.apache.hadoop.hdds.protocol.proto.StorageContainerDatanodeProtocolProtos.CreatePipelineCommandProto)2