Search in sources :

Example 1 with ReservationAllocationStateProto

use of org.apache.hadoop.yarn.proto.YarnProtos.ReservationAllocationStateProto in project hadoop by apache.

the class LeveldbRMStateStore method loadReservationState.

private void loadReservationState(RMState rmState) throws IOException {
    int numReservations = 0;
    LeveldbIterator iter = null;
    try {
        iter = new LeveldbIterator(db);
        iter.seek(bytes(RM_RESERVATION_KEY_PREFIX));
        while (iter.hasNext()) {
            Entry<byte[], byte[]> entry = iter.next();
            String key = asString(entry.getKey());
            String planReservationString = key.substring(RM_RESERVATION_KEY_PREFIX.length());
            String[] parts = planReservationString.split(SEPARATOR);
            if (parts.length != 2) {
                LOG.warn("Incorrect reservation state key " + key);
                continue;
            }
            String planName = parts[0];
            String reservationName = parts[1];
            ReservationAllocationStateProto allocationState = ReservationAllocationStateProto.parseFrom(entry.getValue());
            if (!rmState.getReservationState().containsKey(planName)) {
                rmState.getReservationState().put(planName, new HashMap<ReservationId, ReservationAllocationStateProto>());
            }
            ReservationId reservationId = ReservationId.parseReservationId(reservationName);
            rmState.getReservationState().get(planName).put(reservationId, allocationState);
            numReservations++;
        }
    } catch (DBException e) {
        throw new IOException(e);
    } finally {
        if (iter != null) {
            iter.close();
        }
    }
    LOG.info("Recovered " + numReservations + " reservations");
}
Also used : DBException(org.iq80.leveldb.DBException) LeveldbIterator(org.apache.hadoop.yarn.server.utils.LeveldbIterator) ReservationId(org.apache.hadoop.yarn.api.records.ReservationId) JniDBFactory.asString(org.fusesource.leveldbjni.JniDBFactory.asString) IOException(java.io.IOException) ReservationAllocationStateProto(org.apache.hadoop.yarn.proto.YarnProtos.ReservationAllocationStateProto)

Example 2 with ReservationAllocationStateProto

use of org.apache.hadoop.yarn.proto.YarnProtos.ReservationAllocationStateProto in project hadoop by apache.

the class MemoryRMStateStore method storeReservationState.

@Override
protected synchronized void storeReservationState(ReservationAllocationStateProto reservationAllocation, String planName, String reservationIdName) throws Exception {
    LOG.info("Storing reservationallocation for " + reservationIdName + " " + "for plan " + planName);
    Map<ReservationId, ReservationAllocationStateProto> planState = state.getReservationState().get(planName);
    if (planState == null) {
        planState = new HashMap<>();
        state.getReservationState().put(planName, planState);
    }
    ReservationId reservationId = ReservationId.parseReservationId(reservationIdName);
    planState.put(reservationId, reservationAllocation);
}
Also used : ReservationId(org.apache.hadoop.yarn.api.records.ReservationId) ReservationAllocationStateProto(org.apache.hadoop.yarn.proto.YarnProtos.ReservationAllocationStateProto)

Example 3 with ReservationAllocationStateProto

use of org.apache.hadoop.yarn.proto.YarnProtos.ReservationAllocationStateProto in project hadoop by apache.

the class ReservationListResponsePBImpl method initReservations.

private void initReservations() {
    if (this.reservations != null) {
        return;
    }
    ReservationListResponseProtoOrBuilder p = viaProto ? proto : builder;
    List<ReservationAllocationStateProto> reservationProtos = p.getReservationsList();
    reservations = new ArrayList<>();
    for (ReservationAllocationStateProto r : reservationProtos) {
        reservations.add(convertFromProtoFormat(r));
    }
}
Also used : ReservationListResponseProtoOrBuilder(org.apache.hadoop.yarn.proto.YarnServiceProtos.ReservationListResponseProtoOrBuilder) ReservationAllocationStateProto(org.apache.hadoop.yarn.proto.YarnProtos.ReservationAllocationStateProto)

Example 4 with ReservationAllocationStateProto

use of org.apache.hadoop.yarn.proto.YarnProtos.ReservationAllocationStateProto in project hadoop by apache.

the class AbstractReservationSystem method loadPlan.

private void loadPlan(String planName, Map<ReservationId, ReservationAllocationStateProto> reservations) throws PlanningException {
    Plan plan = plans.get(planName);
    Resource minAllocation = getMinAllocation();
    ResourceCalculator rescCalculator = getResourceCalculator();
    for (Entry<ReservationId, ReservationAllocationStateProto> currentReservation : reservations.entrySet()) {
        plan.addReservation(ReservationSystemUtil.toInMemoryAllocation(planName, currentReservation.getKey(), currentReservation.getValue(), minAllocation, rescCalculator), true);
        resQMap.put(currentReservation.getKey(), planName);
    }
    LOG.info("Recovered reservations for Plan: {}", planName);
}
Also used : ResourceCalculator(org.apache.hadoop.yarn.util.resource.ResourceCalculator) ReservationId(org.apache.hadoop.yarn.api.records.ReservationId) Resource(org.apache.hadoop.yarn.api.records.Resource) ReservationAllocationStateProto(org.apache.hadoop.yarn.proto.YarnProtos.ReservationAllocationStateProto)

Example 5 with ReservationAllocationStateProto

use of org.apache.hadoop.yarn.proto.YarnProtos.ReservationAllocationStateProto in project hadoop by apache.

the class ReservationSystemUtil method buildStateProto.

public static ReservationAllocationStateProto buildStateProto(ReservationAllocation allocation) {
    ReservationAllocationStateProto.Builder builder = ReservationAllocationStateProto.newBuilder();
    builder.setAcceptanceTime(allocation.getAcceptanceTime());
    builder.setContainsGangs(allocation.containsGangs());
    builder.setStartTime(allocation.getStartTime());
    builder.setEndTime(allocation.getEndTime());
    builder.setUser(allocation.getUser());
    ReservationDefinitionProto definitionProto = convertToProtoFormat(allocation.getReservationDefinition());
    builder.setReservationDefinition(definitionProto);
    for (Map.Entry<ReservationInterval, Resource> entry : allocation.getAllocationRequests().entrySet()) {
        ResourceAllocationRequestProto p = ResourceAllocationRequestProto.newBuilder().setStartTime(entry.getKey().getStartTime()).setEndTime(entry.getKey().getEndTime()).setResource(convertToProtoFormat(entry.getValue())).build();
        builder.addAllocationRequests(p);
    }
    ReservationAllocationStateProto allocationProto = builder.build();
    return allocationProto;
}
Also used : Resource(org.apache.hadoop.yarn.api.records.Resource) ResourceAllocationRequestProto(org.apache.hadoop.yarn.proto.YarnProtos.ResourceAllocationRequestProto) ReservationDefinitionProto(org.apache.hadoop.yarn.proto.YarnProtos.ReservationDefinitionProto) HashMap(java.util.HashMap) Map(java.util.Map) ReservationAllocationStateProto(org.apache.hadoop.yarn.proto.YarnProtos.ReservationAllocationStateProto)

Aggregations

ReservationAllocationStateProto (org.apache.hadoop.yarn.proto.YarnProtos.ReservationAllocationStateProto)11 ReservationId (org.apache.hadoop.yarn.api.records.ReservationId)9 RMState (org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState)4 HashMap (java.util.HashMap)3 Map (java.util.Map)3 Resource (org.apache.hadoop.yarn.api.records.Resource)3 ReservationSubmissionRequest (org.apache.hadoop.yarn.api.protocolrecords.ReservationSubmissionRequest)2 ReservationSubmissionResponse (org.apache.hadoop.yarn.api.protocolrecords.ReservationSubmissionResponse)2 ReservationDefinition (org.apache.hadoop.yarn.api.records.ReservationDefinition)2 ResourceCalculator (org.apache.hadoop.yarn.util.resource.ResourceCalculator)2 Test (org.junit.Test)2 IOException (java.io.IOException)1 ReservationUpdateRequest (org.apache.hadoop.yarn.api.protocolrecords.ReservationUpdateRequest)1 YarnRuntimeException (org.apache.hadoop.yarn.exceptions.YarnRuntimeException)1 ReservationDefinitionProto (org.apache.hadoop.yarn.proto.YarnProtos.ReservationDefinitionProto)1 ResourceAllocationRequestProto (org.apache.hadoop.yarn.proto.YarnProtos.ResourceAllocationRequestProto)1 ReservationListResponseProtoOrBuilder (org.apache.hadoop.yarn.proto.YarnServiceProtos.ReservationListResponseProtoOrBuilder)1 RMContext (org.apache.hadoop.yarn.server.resourcemanager.RMContext)1 InMemoryReservationAllocation (org.apache.hadoop.yarn.server.resourcemanager.reservation.InMemoryReservationAllocation)1 ReservationAllocation (org.apache.hadoop.yarn.server.resourcemanager.reservation.ReservationAllocation)1