Search in sources :

Example 1 with MetaStoreHandlerNotExistsException

use of org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException.MetaStoreHandlerNotExistsException in project flink by apache.

the class PulsarCommitter method commit.

@Override
public void commit(Collection<CommitRequest<PulsarCommittable>> requests) throws IOException, InterruptedException {
    TransactionCoordinatorClient client = transactionCoordinatorClient();
    for (CommitRequest<PulsarCommittable> request : requests) {
        PulsarCommittable committable = request.getCommittable();
        TxnID txnID = committable.getTxnID();
        String topic = committable.getTopic();
        LOG.debug("Start committing the Pulsar transaction {} for topic {}", txnID, topic);
        try {
            client.commit(txnID);
        } catch (TransactionCoordinatorClientException e) {
            // This is a known bug for Pulsar Transaction.
            // We have to use instanceof instead of catching them.
            TransactionCoordinatorClientException ex = PulsarTransactionUtils.unwrap(e);
            if (ex instanceof CoordinatorNotFoundException) {
                LOG.error("We couldn't find the Transaction Coordinator from Pulsar broker {}. " + "Check your broker configuration.", committable, ex);
                request.signalFailedWithKnownReason(ex);
            } else if (ex instanceof InvalidTxnStatusException) {
                LOG.error("Unable to commit transaction ({}) because it's in an invalid state. " + "Most likely the transaction has been aborted for some reason. " + "Please check the Pulsar broker logs for more details.", committable, ex);
                request.signalAlreadyCommitted();
            } else if (ex instanceof TransactionNotFoundException) {
                if (request.getNumberOfRetries() == 0) {
                    LOG.error("Unable to commit transaction ({}) because it's not found on Pulsar broker. " + "Most likely the checkpoint interval exceed the transaction timeout.", committable, ex);
                    request.signalFailedWithKnownReason(ex);
                } else {
                    LOG.warn("We can't find the transaction {} after {} retry committing. " + "This may mean that the transaction have been committed in previous but failed with timeout. " + "So we just mark it as committed.", txnID, request.getNumberOfRetries());
                    request.signalAlreadyCommitted();
                }
            } else if (ex instanceof MetaStoreHandlerNotExistsException) {
                LOG.error("We can't find the meta store handler by the mostSigBits from TxnID {}. " + "Did you change the metadata for topic {}?", committable, TRANSACTION_COORDINATOR_ASSIGN, ex);
                request.signalFailedWithKnownReason(ex);
            } else {
                LOG.error("Encountered retriable exception while committing transaction {} for topic {}.", committable, topic, ex);
                int maxRecommitTimes = sinkConfiguration.getMaxRecommitTimes();
                if (request.getNumberOfRetries() < maxRecommitTimes) {
                    request.retryLater();
                } else {
                    String message = String.format("Failed to commit transaction %s after retrying %d times", txnID, maxRecommitTimes);
                    request.signalFailedWithKnownReason(new FlinkRuntimeException(message, ex));
                }
            }
        } catch (Exception e) {
            LOG.error("Transaction ({}) encountered unknown error and data could be potentially lost.", committable, e);
            request.signalFailedWithUnknownReason(e);
        }
    }
}
Also used : InvalidTxnStatusException(org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException.InvalidTxnStatusException) TransactionNotFoundException(org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException.TransactionNotFoundException) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) MetaStoreHandlerNotExistsException(org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException.MetaStoreHandlerNotExistsException) TransactionCoordinatorClientException(org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException) IOException(java.io.IOException) InvalidTxnStatusException(org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException.InvalidTxnStatusException) CoordinatorNotFoundException(org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException.CoordinatorNotFoundException) TransactionNotFoundException(org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException.TransactionNotFoundException) TxnID(org.apache.pulsar.client.api.transaction.TxnID) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) MetaStoreHandlerNotExistsException(org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException.MetaStoreHandlerNotExistsException) TransactionCoordinatorClientException(org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException) TransactionCoordinatorClient(org.apache.pulsar.client.api.transaction.TransactionCoordinatorClient) CoordinatorNotFoundException(org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException.CoordinatorNotFoundException)

Aggregations

IOException (java.io.IOException)1 FlinkRuntimeException (org.apache.flink.util.FlinkRuntimeException)1 TransactionCoordinatorClient (org.apache.pulsar.client.api.transaction.TransactionCoordinatorClient)1 TransactionCoordinatorClientException (org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException)1 CoordinatorNotFoundException (org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException.CoordinatorNotFoundException)1 InvalidTxnStatusException (org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException.InvalidTxnStatusException)1 MetaStoreHandlerNotExistsException (org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException.MetaStoreHandlerNotExistsException)1 TransactionNotFoundException (org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException.TransactionNotFoundException)1 TxnID (org.apache.pulsar.client.api.transaction.TxnID)1