Search in sources :

Example 1 with MaybeOffloaded

use of org.apache.flink.runtime.deployment.TaskDeploymentDescriptor.MaybeOffloaded in project flink by apache.

the class IntermediateResult method clearCachedInformationForPartitionGroup.

public void clearCachedInformationForPartitionGroup(ConsumedPartitionGroup consumedPartitionGroup) {
    // When a ConsumedPartitionGroup changes, the cache of ShuffleDescriptors for this
    // partition group is no longer valid and needs to be removed.
    // 
    // Currently, there are two scenarios:
    // 1. The ConsumedPartitionGroup is released
    // 2. Its producer encounters a failover
    // Remove the cache for the ConsumedPartitionGroup and notify the BLOB writer to delete the
    // cache if it is offloaded
    final MaybeOffloaded<ShuffleDescriptor[]> cache = this.shuffleDescriptorCache.remove(consumedPartitionGroup);
    if (cache instanceof Offloaded) {
        PermanentBlobKey blobKey = ((Offloaded<ShuffleDescriptor[]>) cache).serializedValueKey;
        this.producer.getGraph().deleteBlobs(Collections.singletonList(blobKey));
    }
}
Also used : Offloaded(org.apache.flink.runtime.deployment.TaskDeploymentDescriptor.Offloaded) MaybeOffloaded(org.apache.flink.runtime.deployment.TaskDeploymentDescriptor.MaybeOffloaded) PermanentBlobKey(org.apache.flink.runtime.blob.PermanentBlobKey) ShuffleDescriptor(org.apache.flink.runtime.shuffle.ShuffleDescriptor)

Aggregations

PermanentBlobKey (org.apache.flink.runtime.blob.PermanentBlobKey)1 MaybeOffloaded (org.apache.flink.runtime.deployment.TaskDeploymentDescriptor.MaybeOffloaded)1 Offloaded (org.apache.flink.runtime.deployment.TaskDeploymentDescriptor.Offloaded)1 ShuffleDescriptor (org.apache.flink.runtime.shuffle.ShuffleDescriptor)1