Search in sources :

Example 1 with UnexpectedException

use of com.evanlennick.retry4j.exception.UnexpectedException in project bender by Nextdoor.

the class S3EventIterator method hasNext.

@Override
public boolean hasNext() {
    if (this.currentIndex < this.records.size()) {
        return true;
    }
    /*
     * Wrap has next row in retry logic. This is because there is intermittent socket timeouts when
     * reading from S3 that cause the function to hang/fail.
     */
    Callable<Boolean> callable = () -> {
        return this.lineIterator.hasNext();
    };
    boolean hasNext;
    try {
        CallResults<Object> results = new CallExecutor(this.config).execute(callable);
        hasNext = (boolean) results.getResult();
    } catch (RetriesExhaustedException ree) {
        throw new RuntimeException(ree.getCallResults().getLastExceptionThatCausedRetry());
    } catch (UnexpectedException ue) {
        throw ue;
    }
    /*
     * If there are no lines then the reader from which the lines came from should be closed.
     */
    if (!hasNext) {
        closeCurrentReader();
    }
    return hasNext;
}
Also used : CallExecutor(com.evanlennick.retry4j.CallExecutor) UnexpectedException(com.evanlennick.retry4j.exception.UnexpectedException) RetriesExhaustedException(com.evanlennick.retry4j.exception.RetriesExhaustedException) S3Object(com.amazonaws.services.s3.model.S3Object)

Example 2 with UnexpectedException

use of com.evanlennick.retry4j.exception.UnexpectedException in project bender by Nextdoor.

the class S3EventIterator method next.

@Override
public InternalEvent next() {
    updateCursor();
    /*
     * Wrap reading next row in retry logic. This is because there is intermittent socket timeouts
     * when reading from S3 that cause the function to hang/fail.
     */
    Callable<String> callable = () -> {
        return this.lineIterator.next();
    };
    String nextRow;
    try {
        CallResults<Object> results = new CallExecutor(this.config).execute(callable);
        nextRow = (String) results.getResult();
    } catch (RetriesExhaustedException ree) {
        throw new RuntimeException(ree.getCallResults().getLastExceptionThatCausedRetry());
    } catch (UnexpectedException ue) {
        throw ue;
    }
    /*
     * Construct the internal event
     */
    return new S3InternalEvent(nextRow, this.context, this.arrivalTime, currentS3Entity.getObject().getKey(), currentS3Entity.getBucket().getName(), currentS3Entity.getObject().getVersionId());
}
Also used : CallExecutor(com.evanlennick.retry4j.CallExecutor) UnexpectedException(com.evanlennick.retry4j.exception.UnexpectedException) RetriesExhaustedException(com.evanlennick.retry4j.exception.RetriesExhaustedException) S3Object(com.amazonaws.services.s3.model.S3Object)

Example 3 with UnexpectedException

use of com.evanlennick.retry4j.exception.UnexpectedException in project bender by Nextdoor.

the class HttpTransport method sendBatch.

public void sendBatch(byte[] raw) throws TransportException {
    /*
     * Wrap the call with retry logic to avoid intermittent ES issues.
     */
    Callable<HttpResponse> callable = () -> {
        HttpResponse resp;
        String responseString = null;
        HttpPost httpPost = new HttpPost(this.url);
        /*
       * Do the call, read response, release connection so it is available for use again, and
       * finally check the response.
       */
        try {
            if (this.useGzip) {
                resp = sendBatchCompressed(httpPost, raw);
            } else {
                resp = sendBatchUncompressed(httpPost, raw);
            }
            try {
                responseString = EntityUtils.toString(resp.getEntity());
            } catch (ParseException | IOException e) {
                throw new TransportException("http transport call failed because " + resp.getStatusLine().getReasonPhrase());
            }
        } finally {
            /*
         * Always release connection otherwise it blocks future requests.
         */
            httpPost.releaseConnection();
        }
        checkResponse(resp, responseString);
        return resp;
    };
    RetryConfig config = new RetryConfigBuilder().retryOnSpecificExceptions(TransportException.class).withMaxNumberOfTries(this.retries + 1).withDelayBetweenTries(this.retryDelayMs, ChronoUnit.MILLIS).withExponentialBackoff().build();
    try {
        new CallExecutor(config).execute(callable);
    } catch (RetriesExhaustedException ree) {
        logger.warn("transport failed after " + ree.getCallResults().getTotalTries() + " tries.");
        throw new TransportException(ree.getCallResults().getLastExceptionThatCausedRetry());
    } catch (UnexpectedException ue) {
        throw new TransportException(ue);
    }
}
Also used : RetryConfigBuilder(com.evanlennick.retry4j.RetryConfigBuilder) HttpPost(org.apache.http.client.methods.HttpPost) CallExecutor(com.evanlennick.retry4j.CallExecutor) UnexpectedException(com.evanlennick.retry4j.exception.UnexpectedException) RetryConfig(com.evanlennick.retry4j.RetryConfig) RetriesExhaustedException(com.evanlennick.retry4j.exception.RetriesExhaustedException) HttpResponse(org.apache.http.HttpResponse) TransportException(com.nextdoor.bender.ipc.TransportException)

Example 4 with UnexpectedException

use of com.evanlennick.retry4j.exception.UnexpectedException in project bender by Nextdoor.

the class TcpTransport method sendBatch.

@Override
public void sendBatch(TransportBuffer buffer) throws TransportException {
    Buffer internalBuffer = ((TcpTransportBuffer) buffer).getInternalBuffer();
    Callable<Boolean> write = () -> {
        try {
            sink.write(internalBuffer, internalBuffer.size());
            return true;
        } catch (IOException ex) {
            throw new TransportException("Error while sending in tcp transport", ex);
        }
    };
    try {
        new CallExecutor(retryConfig).execute(write);
    } catch (RetriesExhaustedException | UnexpectedException ue) {
        throw new TransportException(ue);
    }
}
Also used : Buffer(okio.Buffer) TransportBuffer(com.nextdoor.bender.ipc.TransportBuffer) CallExecutor(com.evanlennick.retry4j.CallExecutor) UnexpectedException(com.evanlennick.retry4j.exception.UnexpectedException) RetriesExhaustedException(com.evanlennick.retry4j.exception.RetriesExhaustedException) IOException(java.io.IOException) TransportException(com.nextdoor.bender.ipc.TransportException)

Aggregations

CallExecutor (com.evanlennick.retry4j.CallExecutor)4 RetriesExhaustedException (com.evanlennick.retry4j.exception.RetriesExhaustedException)4 UnexpectedException (com.evanlennick.retry4j.exception.UnexpectedException)4 S3Object (com.amazonaws.services.s3.model.S3Object)2 TransportException (com.nextdoor.bender.ipc.TransportException)2 RetryConfig (com.evanlennick.retry4j.RetryConfig)1 RetryConfigBuilder (com.evanlennick.retry4j.RetryConfigBuilder)1 TransportBuffer (com.nextdoor.bender.ipc.TransportBuffer)1 IOException (java.io.IOException)1 Buffer (okio.Buffer)1 HttpResponse (org.apache.http.HttpResponse)1 HttpPost (org.apache.http.client.methods.HttpPost)1