Search in sources :

Example 6 with EventAnnouncement

use of org.apache.flink.runtime.io.network.api.EventAnnouncement in project flink by apache.

the class RemoteInputChannel method announce.

private SequenceBuffer announce(SequenceBuffer sequenceBuffer) throws IOException {
    checkState(!sequenceBuffer.buffer.isBuffer(), "Only a CheckpointBarrier can be announced but found %s", sequenceBuffer.buffer);
    checkAnnouncedOnlyOnce(sequenceBuffer);
    AbstractEvent event = EventSerializer.fromBuffer(sequenceBuffer.buffer, getClass().getClassLoader());
    checkState(event instanceof CheckpointBarrier, "Only a CheckpointBarrier can be announced but found %s", sequenceBuffer.buffer);
    CheckpointBarrier barrier = (CheckpointBarrier) event;
    return new SequenceBuffer(EventSerializer.toBuffer(new EventAnnouncement(barrier, sequenceBuffer.sequenceNumber), true), sequenceBuffer.sequenceNumber);
}
Also used : CheckpointBarrier(org.apache.flink.runtime.io.network.api.CheckpointBarrier) EventAnnouncement(org.apache.flink.runtime.io.network.api.EventAnnouncement) AbstractEvent(org.apache.flink.runtime.event.AbstractEvent)

Example 7 with EventAnnouncement

use of org.apache.flink.runtime.io.network.api.EventAnnouncement in project flink by apache.

the class ChannelStatePersister method checkForBarrier.

protected OptionalLong checkForBarrier(Buffer buffer) throws IOException {
    AbstractEvent event = parseEvent(buffer);
    if (event instanceof CheckpointBarrier) {
        long barrierId = ((CheckpointBarrier) event).getId();
        long expectedBarrierId = checkpointStatus == CheckpointStatus.COMPLETED ? lastSeenBarrier + 1 : lastSeenBarrier;
        if (barrierId >= expectedBarrierId) {
            logEvent("found barrier", barrierId);
            checkpointStatus = CheckpointStatus.BARRIER_RECEIVED;
            lastSeenBarrier = barrierId;
            return OptionalLong.of(lastSeenBarrier);
        } else {
            logEvent("ignoring barrier", barrierId);
        }
    }
    if (event instanceof EventAnnouncement) {
        // NOTE: only remote channels
        EventAnnouncement announcement = (EventAnnouncement) event;
        if (announcement.getAnnouncedEvent() instanceof CheckpointBarrier) {
            long barrierId = ((CheckpointBarrier) announcement.getAnnouncedEvent()).getId();
            logEvent("found announcement for barrier", barrierId);
            return OptionalLong.of(barrierId);
        }
    }
    return OptionalLong.empty();
}
Also used : CheckpointBarrier(org.apache.flink.runtime.io.network.api.CheckpointBarrier) EventAnnouncement(org.apache.flink.runtime.io.network.api.EventAnnouncement) AbstractEvent(org.apache.flink.runtime.event.AbstractEvent)

Aggregations

EventAnnouncement (org.apache.flink.runtime.io.network.api.EventAnnouncement)7 CheckpointBarrier (org.apache.flink.runtime.io.network.api.CheckpointBarrier)6 AbstractEvent (org.apache.flink.runtime.event.AbstractEvent)4 CancelCheckpointMarker (org.apache.flink.runtime.io.network.api.CancelCheckpointMarker)3 IOException (java.io.IOException)2 SubtaskConnectionDescriptor (org.apache.flink.runtime.io.network.api.SubtaskConnectionDescriptor)2 EndOfChannelStateEvent (org.apache.flink.runtime.io.network.partition.consumer.EndOfChannelStateEvent)2 TestCheckpointedInputGateBuilder (org.apache.flink.streaming.util.TestCheckpointedInputGateBuilder)2 Test (org.junit.Test)2 ByteBuffer (java.nio.ByteBuffer)1 ByteOrder (java.nio.ByteOrder)1 DataInputDeserializer (org.apache.flink.core.memory.DataInputDeserializer)1 DataOutputSerializer (org.apache.flink.core.memory.DataOutputSerializer)1 EndOfData (org.apache.flink.runtime.io.network.api.EndOfData)1 EndOfPartitionEvent (org.apache.flink.runtime.io.network.api.EndOfPartitionEvent)1 EventSerializer.toBuffer (org.apache.flink.runtime.io.network.api.serialization.EventSerializer.toBuffer)1 Buffer (org.apache.flink.runtime.io.network.buffer.Buffer)1 TestInputChannel (org.apache.flink.runtime.io.network.partition.consumer.TestInputChannel)1 TestBufferFactory.createBuffer (org.apache.flink.runtime.io.network.util.TestBufferFactory.createBuffer)1