Examples with ContainerPlacementMetadata org.apache.samza.clustermanager.container.placement.ContainerPlacementMetadata used on opensource projects

Example 11 with ContainerPlacementMetadata

use of org.apache.samza.clustermanager.container.placement.ContainerPlacementMetadata in project samza by apache.

the class ContainerManager method handleContainerStop.

/**
 * Handles the action to be taken after the container has been stopped. If stop was issued due to existing control
 * action, mark the container as stopped, otherwise
 *
 * Case 1. When standby is enabled, refer to {@link StandbyContainerManager#handleContainerStop} to check constraints.
 * Case 2. When standby is disabled there are two cases according to host-affinity being enabled
 *    Case 2.1. When host-affinity is enabled resources are requested on host where container was last seen
 *    Case 2.2. When host-affinity is disabled resources are requested for ANY_HOST
 *
 * @param processorId logical id of the container eg 1,2,3
 * @param containerId last known id of the container deployed
 * @param preferredHost host on which container was last deployed
 * @param exitStatus exit code returned by the container
 * @param preferredHostRetryDelay delay to be incurred before requesting resources
 * @param containerAllocator allocator for requesting resources
 */
void handleContainerStop(String processorId, String containerId, String preferredHost, int exitStatus, Duration preferredHostRetryDelay, ContainerAllocator containerAllocator) {
    if (hasActiveContainerPlacementAction(processorId)) {
        ContainerPlacementMetadata metadata = getPlacementActionMetadata(processorId).get();
        LOG.info("Setting the container state with Processor ID: {} to be stopped because of existing ContainerPlacement action: {}", processorId, metadata);
        metadata.setContainerStatus(ContainerPlacementMetadata.ContainerStatus.STOPPED);
    } else if (standbyContainerManager.isPresent()) {
        standbyContainerManager.get().handleContainerStop(processorId, containerId, preferredHost, exitStatus, containerAllocator, preferredHostRetryDelay);
    } else {
        // If StandbyTasks are not enabled, we simply make a request for the preferredHost
        containerAllocator.requestResourceWithDelay(processorId, preferredHost, preferredHostRetryDelay);
    }
}

Example 12 with ContainerPlacementMetadata

use of org.apache.samza.clustermanager.container.placement.ContainerPlacementMetadata in project samza by apache.

the class ContainerManager method handleContainerLaunchFail.

/**
 * Handle the container launch failure for active containers and standby (if enabled).
 *
 * Case 1. If this launch was issued due to an existing container placement action update the metadata to report failure and issue
 *         a request for source host where the container was last seen and mark the container placement failed
 * Case 2. When standby is enabled, refer to {@link StandbyContainerManager#handleContainerLaunchFail} to check behavior
 * Case 3. When standby is disabled the allocator issues a request for ANY_HOST resources
 *
 * @param processorId logical id of the container eg 1,2,3
 * @param containerId last known id of the container deployed
 * @param preferredHost host on which container is requested to be deployed
 * @param containerAllocator allocator for requesting resources
 */
void handleContainerLaunchFail(String processorId, String containerId, String preferredHost, ContainerAllocator containerAllocator) {
    if (processorId != null && hasActiveContainerPlacementAction(processorId)) {
        ContainerPlacementMetadata metaData = getPlacementActionMetadata(processorId).get();
        // Issue a request to start the container on source host
        String sourceHost = hostAffinityEnabled ? metaData.getSourceHost() : ResourceRequestState.ANY_HOST;
        markContainerPlacementActionFailed(metaData, String.format("failed to start container on destination host %s, attempting to start on source host %s", preferredHost, sourceHost));
        containerAllocator.requestResource(processorId, sourceHost);
    } else if (processorId != null && standbyContainerManager.isPresent()) {
        standbyContainerManager.get().handleContainerLaunchFail(processorId, containerId, containerAllocator);
    } else if (processorId != null) {
        LOG.info("Falling back to ANY_HOST for Processor ID: {} since launch failed for Container ID: {} on host: {}", processorId, containerId, preferredHost);
        containerAllocator.requestResource(processorId, ResourceRequestState.ANY_HOST);
    } else {
        LOG.warn("Did not find a pending Processor ID for Container ID: {} on host: {}. " + "Ignoring invalid/redundant notification.", containerId, preferredHost);
    }
}

Example 13 with ContainerPlacementMetadata

use of org.apache.samza.clustermanager.container.placement.ContainerPlacementMetadata in project samza by apache.

the class ContainerManager method handleExpiredResource.

/**
 * Handles expired allocated resource by requesting the same resource again and release the expired allocated resource
 *
 * @param request pending request for the preferred host
 * @param resource resource allocated from {@link ClusterResourceManager} which has expired
 * @param preferredHost host on which container is requested to be deployed
 * @param resourceRequestState state of request in {@link ContainerAllocator}
 * @param allocator allocator for requesting resources
 */
void handleExpiredResource(SamzaResourceRequest request, SamzaResource resource, String preferredHost, ResourceRequestState resourceRequestState, ContainerAllocator allocator) {
    LOG.info("Allocated resource {} has expired for Processor ID: {} request: {}. Re-requesting resource again", resource, request.getProcessorId(), request);
    resourceRequestState.releaseUnstartableContainer(resource, preferredHost);
    resourceRequestState.cancelResourceRequest(request);
    SamzaResourceRequest newResourceRequest = allocator.getResourceRequest(request.getProcessorId(), request.getPreferredHost());
    if (hasActiveContainerPlacementAction(newResourceRequest.getProcessorId())) {
        ContainerPlacementMetadata metadata = getPlacementActionMetadata(request.getProcessorId()).get();
        metadata.recordResourceRequest(newResourceRequest);
    }
    allocator.issueResourceRequest(newResourceRequest);
}