Examples with PulsarPartitionSplitState org.apache.flink.connector.pulsar.source.split.PulsarPartitionSplitState used on opensource projects

Example 1 with PulsarPartitionSplitState

use of org.apache.flink.connector.pulsar.source.split.PulsarPartitionSplitState in project flink by apache.

the class PulsarOrderedSourceReader method onSplitFinished.

@Override
protected void onSplitFinished(Map<String, PulsarPartitionSplitState> finishedSplitIds) {
    // We don't require new splits, all the splits are pre-assigned by source enumerator.
    if (LOG.isDebugEnabled()) {
        LOG.debug("onSplitFinished event: {}", finishedSplitIds);
    }
    for (Map.Entry<String, PulsarPartitionSplitState> entry : finishedSplitIds.entrySet()) {
        PulsarPartitionSplitState state = entry.getValue();
        MessageId latestConsumedId = state.getLatestConsumedId();
        if (latestConsumedId != null) {
            cursorsOfFinishedSplits.put(state.getPartition(), latestConsumedId);
        }
    }
}

Example 2 with PulsarPartitionSplitState

use of org.apache.flink.connector.pulsar.source.split.PulsarPartitionSplitState in project flink by apache.

the class PulsarUnorderedPartitionSplitReader method snapshotState.

public PulsarPartitionSplitState snapshotState(long checkpointId) {
    PulsarPartitionSplitState state = new PulsarPartitionSplitState(registeredSplit);
    // Avoiding NP problem when Pulsar don't get the message before Flink checkpoint.
    if (uncommittedTransaction != null) {
        TxnID txnID = uncommittedTransaction.getTxnID();
        this.uncommittedTransaction = newTransaction();
        state.setUncommittedTransactionId(txnID);
    }
    return state;
}

Example 3 with PulsarPartitionSplitState

use of org.apache.flink.connector.pulsar.source.split.PulsarPartitionSplitState in project flink by apache.

the class PulsarUnorderedSourceReader method onSplitFinished.

@Override
protected void onSplitFinished(Map<String, PulsarPartitionSplitState> finishedSplitIds) {
    // We don't require new splits, all the splits are pre-assigned by source enumerator.
    if (LOG.isDebugEnabled()) {
        LOG.debug("onSplitFinished event: {}", finishedSplitIds);
    }
    if (coordinatorClient != null) {
        // Commit the uncommitted transaction
        for (Map.Entry<String, PulsarPartitionSplitState> entry : finishedSplitIds.entrySet()) {
            PulsarPartitionSplitState state = entry.getValue();
            TxnID uncommittedTransactionId = state.getUncommittedTransactionId();
            if (uncommittedTransactionId != null) {
                transactionsOfFinishedSplits.add(uncommittedTransactionId);
            }
        }
    }
}