Search in sources :

Example 11 with HadoopTaskContext

use of org.apache.ignite.internal.processors.hadoop.HadoopTaskContext in project ignite by apache.

the class HadoopV2Job method cleanupTaskEnvironment.

/**
 * {@inheritDoc}
 */
@Override
public void cleanupTaskEnvironment(HadoopTaskInfo info) throws IgniteCheckedException {
    HadoopTaskContext ctx = ctxs.remove(new T2<>(info.type(), info.taskNumber())).get();
    taskCtxClsPool.add(ctx.getClass());
    File locDir = taskLocalDir(igniteWorkDirectory(), locNodeId, info);
    if (locDir.exists())
        U.delete(locDir);
}
Also used : HadoopTaskContext(org.apache.ignite.internal.processors.hadoop.HadoopTaskContext) File(java.io.File) T2(org.apache.ignite.internal.util.typedef.T2)

Aggregations

HadoopTaskContext (org.apache.ignite.internal.processors.hadoop.HadoopTaskContext)11 Random (java.util.Random)5 IntWritable (org.apache.hadoop.io.IntWritable)5 HadoopMultimap (org.apache.ignite.internal.processors.hadoop.shuffle.collections.HadoopMultimap)5 GridUnsafeMemory (org.apache.ignite.internal.util.offheap.unsafe.GridUnsafeMemory)5 HadoopJobInfo (org.apache.ignite.internal.processors.hadoop.HadoopJobInfo)4 GridRandom (org.apache.ignite.internal.util.GridRandom)4 ByteArrayInputStream (java.io.ByteArrayInputStream)2 DataInputStream (java.io.DataInputStream)2 IOException (java.io.IOException)2 Collection (java.util.Collection)2 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)2 ConcurrentLinkedQueue (java.util.concurrent.ConcurrentLinkedQueue)2 IgniteCheckedException (org.apache.ignite.IgniteCheckedException)2 HadoopSerialization (org.apache.ignite.internal.processors.hadoop.HadoopSerialization)2 HadoopTaskInfo (org.apache.ignite.internal.processors.hadoop.HadoopTaskInfo)2 HadoopTaskInput (org.apache.ignite.internal.processors.hadoop.HadoopTaskInput)2 HadoopPerformanceCounter (org.apache.ignite.internal.processors.hadoop.counter.HadoopPerformanceCounter)2 HadoopConcurrentHashMultimap (org.apache.ignite.internal.processors.hadoop.shuffle.collections.HadoopConcurrentHashMultimap)2 HadoopSkipList (org.apache.ignite.internal.processors.hadoop.shuffle.collections.HadoopSkipList)2