Examples with BufferedRecord org.apache.gobblin.async.BufferedRecord used on opensource projects

Example 6 with BufferedRecord

use of org.apache.gobblin.async.BufferedRecord in project incubator-gobblin by apache.

the class AsyncHttpJoinConverter method convertRecordAsync.

/**
 * Convert an input record to a future object where an output record will be filled in sometime later
 * Sequence:
 *    Convert input (DI) to an http request
 *    Send http request asynchronously, and registers an http callback
 *    Create an {@link CompletableFuture} object. When the callback is invoked, this future object is filled in by an output record which is converted from http response.
 *    Return the future object.
 */
@Override
public final CompletableFuture<DO> convertRecordAsync(SO outputSchema, DI inputRecord, WorkUnitState workUnitState) throws DataConversionException {
    // Convert DI to HttpOperation
    HttpOperation operation = generateHttpOperation(inputRecord, workUnitState);
    BufferedRecord<GenericRecord> bufferedRecord = new BufferedRecord<>(operation, WriteCallback.EMPTY);
    // Convert HttpOperation to RQ
    Queue<BufferedRecord<GenericRecord>> buffer = new LinkedBlockingDeque<>();
    buffer.add(bufferedRecord);
    AsyncRequest<GenericRecord, RQ> request = this.requestBuilder.buildRequest(buffer);
    RQ rawRequest = request.getRawRequest();
    // Execute query and get response
    AsyncHttpJoinConverterContext context = new AsyncHttpJoinConverterContext(this, outputSchema, inputRecord, request);
    try {
        httpClient.sendAsyncRequest(rawRequest, context.getCallback());
    } catch (IOException e) {
        throw new DataConversionException(e);
    }
    return context.future;
}