Search in sources :

Example 1 with CancelJobSupervisorOperation

use of com.hazelcast.mapreduce.impl.operation.CancelJobSupervisorOperation in project hazelcast by hazelcast.

the class JobSupervisor method asyncCancelRemoteOperations.

private void asyncCancelRemoteOperations(final Set<Address> addresses) {
    final NodeEngine nodeEngine = mapReduceService.getNodeEngine();
    TaskScheduler taskScheduler = nodeEngine.getExecutionService().getGlobalTaskScheduler();
    taskScheduler.execute(new Runnable() {

        @Override
        public void run() {
            String name = getConfiguration().getName();
            String jobId = getConfiguration().getJobId();
            for (Address address : addresses) {
                try {
                    CancelJobSupervisorOperation operation = new CancelJobSupervisorOperation(name, jobId);
                    mapReduceService.processRequest(address, operation);
                } catch (Exception ignore) {
                    // We can ignore this exception since we just want to cancel the job
                    // and the member may be crashed or unreachable in some way
                    ILogger logger = nodeEngine.getLogger(JobSupervisor.class);
                    logger.finest("Remote node may already be down", ignore);
                }
            }
        }
    });
}
Also used : NodeEngine(com.hazelcast.spi.NodeEngine) Address(com.hazelcast.nio.Address) ILogger(com.hazelcast.logging.ILogger) TaskScheduler(com.hazelcast.spi.TaskScheduler) CancelJobSupervisorOperation(com.hazelcast.mapreduce.impl.operation.CancelJobSupervisorOperation)

Example 2 with CancelJobSupervisorOperation

use of com.hazelcast.mapreduce.impl.operation.CancelJobSupervisorOperation in project hazelcast by hazelcast.

the class MapReduceService method registerJobSupervisorCancellation.

public boolean registerJobSupervisorCancellation(String name, String jobId, Address jobOwner) {
    NodeJobTracker jobTracker = (NodeJobTracker) createDistributedObject(name);
    if (jobTracker.registerJobSupervisorCancellation(jobId) && getLocalAddress().equals(jobOwner)) {
        for (Member member : clusterService.getMembers()) {
            if (!member.getAddress().equals(jobOwner)) {
                try {
                    ProcessingOperation operation = new CancelJobSupervisorOperation(name, jobId);
                    processRequest(member.getAddress(), operation);
                } catch (Exception ignore) {
                    LOGGER.finest("Member might be already unavailable", ignore);
                }
            }
        }
        return true;
    }
    return false;
}
Also used : ProcessingOperation(com.hazelcast.mapreduce.impl.operation.ProcessingOperation) Member(com.hazelcast.core.Member) CancelJobSupervisorOperation(com.hazelcast.mapreduce.impl.operation.CancelJobSupervisorOperation) ExecutionException(java.util.concurrent.ExecutionException)

Example 3 with CancelJobSupervisorOperation

use of com.hazelcast.mapreduce.impl.operation.CancelJobSupervisorOperation in project hazelcast by hazelcast.

the class MapReduceDataSerializerHook method createFactory.

@Override
public DataSerializableFactory createFactory() {
    ConstructorFunction<Integer, IdentifiedDataSerializable>[] constructors = new ConstructorFunction[LEN];
    constructors[KEY_VALUE_SOURCE_MAP] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new MapKeyValueSource();
        }
    };
    constructors[KEY_VALUE_SOURCE_MULTIMAP] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new MultiMapKeyValueSource();
        }
    };
    constructors[REDUCER_CHUNK_MESSAGE] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new IntermediateChunkNotification();
        }
    };
    constructors[REDUCER_LAST_CHUNK_MESSAGE] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new LastChunkNotification();
        }
    };
    constructors[TRACKED_JOB_OPERATION] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new KeyValueJobOperation();
        }
    };
    constructors[REQUEST_PARTITION_MAPPING] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new RequestPartitionMapping();
        }
    };
    constructors[REQUEST_PARTITION_REDUCING] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new RequestPartitionReducing();
        }
    };
    constructors[REQUEST_PARTITION_PROCESSED] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new RequestPartitionProcessed();
        }
    };
    constructors[GET_RESULT_OPERATION] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new GetResultOperation();
        }
    };
    constructors[START_PROCESSING_OPERATION] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new StartProcessingJobOperation();
        }
    };
    constructors[REQUEST_PARTITION_RESULT] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new RequestPartitionResult();
        }
    };
    constructors[REDUCING_FINISHED_MESSAGE] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new ReducingFinishedNotification();
        }
    };
    constructors[FIRE_NOTIFICATION_OPERATION] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new FireNotificationOperation();
        }
    };
    constructors[REQUEST_MEMBERID_ASSIGNMENT] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new RequestMemberIdAssignment();
        }
    };
    constructors[PROCESS_STATS_UPDATE_OPERATION] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new ProcessStatsUpdateOperation();
        }
    };
    constructors[NOTIFY_REMOTE_EXCEPTION_OPERATION] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new NotifyRemoteExceptionOperation();
        }
    };
    constructors[CANCEL_JOB_SUPERVISOR_OPERATION] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new CancelJobSupervisorOperation();
        }
    };
    constructors[KEY_VALUE_SOURCE_LIST] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new ListKeyValueSource();
        }
    };
    constructors[KEY_VALUE_SOURCE_SET] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new SetKeyValueSource();
        }
    };
    constructors[KEYS_ASSIGNMENT_RESULT] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new KeysAssignmentResult();
        }
    };
    constructors[KEYS_ASSIGNMENT_OPERATION] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new KeysAssignmentOperation();
        }
    };
    constructors[POSTPONE_PARTITION_PROCESSING_OPERATION] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new PostPonePartitionProcessing();
        }
    };
    constructors[HASH_MAP_ADAPTER] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new HashMapAdapter();
        }
    };
    constructors[COMBINER_RESULT_LIST] = new ConstructorFunction<Integer, IdentifiedDataSerializable>() {

        @Override
        public IdentifiedDataSerializable createNew(Integer arg) {
            return new CombinerResultList();
        }
    };
    return new ArrayDataSerializableFactory(constructors);
}
Also used : GetResultOperation(com.hazelcast.mapreduce.impl.operation.GetResultOperation) KeyValueJobOperation(com.hazelcast.mapreduce.impl.operation.KeyValueJobOperation) ProcessStatsUpdateOperation(com.hazelcast.mapreduce.impl.operation.ProcessStatsUpdateOperation) LastChunkNotification(com.hazelcast.mapreduce.impl.notification.LastChunkNotification) KeysAssignmentResult(com.hazelcast.mapreduce.impl.operation.KeysAssignmentResult) ReducingFinishedNotification(com.hazelcast.mapreduce.impl.notification.ReducingFinishedNotification) RequestPartitionProcessed(com.hazelcast.mapreduce.impl.operation.RequestPartitionProcessed) IdentifiedDataSerializable(com.hazelcast.nio.serialization.IdentifiedDataSerializable) KeysAssignmentOperation(com.hazelcast.mapreduce.impl.operation.KeysAssignmentOperation) PostPonePartitionProcessing(com.hazelcast.mapreduce.impl.operation.PostPonePartitionProcessing) IntermediateChunkNotification(com.hazelcast.mapreduce.impl.notification.IntermediateChunkNotification) ConstructorFunction(com.hazelcast.util.ConstructorFunction) RequestPartitionReducing(com.hazelcast.mapreduce.impl.operation.RequestPartitionReducing) FireNotificationOperation(com.hazelcast.mapreduce.impl.operation.FireNotificationOperation) RequestPartitionResult(com.hazelcast.mapreduce.impl.operation.RequestPartitionResult) RequestPartitionMapping(com.hazelcast.mapreduce.impl.operation.RequestPartitionMapping) StartProcessingJobOperation(com.hazelcast.mapreduce.impl.operation.StartProcessingJobOperation) RequestMemberIdAssignment(com.hazelcast.mapreduce.impl.operation.RequestMemberIdAssignment) ArrayDataSerializableFactory(com.hazelcast.internal.serialization.impl.ArrayDataSerializableFactory) CancelJobSupervisorOperation(com.hazelcast.mapreduce.impl.operation.CancelJobSupervisorOperation) NotifyRemoteExceptionOperation(com.hazelcast.mapreduce.impl.operation.NotifyRemoteExceptionOperation)

Aggregations

CancelJobSupervisorOperation (com.hazelcast.mapreduce.impl.operation.CancelJobSupervisorOperation)3 Member (com.hazelcast.core.Member)1 ArrayDataSerializableFactory (com.hazelcast.internal.serialization.impl.ArrayDataSerializableFactory)1 ILogger (com.hazelcast.logging.ILogger)1 IntermediateChunkNotification (com.hazelcast.mapreduce.impl.notification.IntermediateChunkNotification)1 LastChunkNotification (com.hazelcast.mapreduce.impl.notification.LastChunkNotification)1 ReducingFinishedNotification (com.hazelcast.mapreduce.impl.notification.ReducingFinishedNotification)1 FireNotificationOperation (com.hazelcast.mapreduce.impl.operation.FireNotificationOperation)1 GetResultOperation (com.hazelcast.mapreduce.impl.operation.GetResultOperation)1 KeyValueJobOperation (com.hazelcast.mapreduce.impl.operation.KeyValueJobOperation)1 KeysAssignmentOperation (com.hazelcast.mapreduce.impl.operation.KeysAssignmentOperation)1 KeysAssignmentResult (com.hazelcast.mapreduce.impl.operation.KeysAssignmentResult)1 NotifyRemoteExceptionOperation (com.hazelcast.mapreduce.impl.operation.NotifyRemoteExceptionOperation)1 PostPonePartitionProcessing (com.hazelcast.mapreduce.impl.operation.PostPonePartitionProcessing)1 ProcessStatsUpdateOperation (com.hazelcast.mapreduce.impl.operation.ProcessStatsUpdateOperation)1 ProcessingOperation (com.hazelcast.mapreduce.impl.operation.ProcessingOperation)1 RequestMemberIdAssignment (com.hazelcast.mapreduce.impl.operation.RequestMemberIdAssignment)1 RequestPartitionMapping (com.hazelcast.mapreduce.impl.operation.RequestPartitionMapping)1 RequestPartitionProcessed (com.hazelcast.mapreduce.impl.operation.RequestPartitionProcessed)1 RequestPartitionReducing (com.hazelcast.mapreduce.impl.operation.RequestPartitionReducing)1