Examples with RetryHttpRequestInitializer org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer used on opensource projects

Example 1 with RetryHttpRequestInitializer

use of org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer in project beam by apache.

the class BigQueryServicesImpl method newBigQueryClient.

/**
 * Returns a BigQuery client builder using the specified {@link BigQueryOptions}.
 */
private static Bigquery.Builder newBigQueryClient(BigQueryOptions options) {
    // Do not log 404. It clutters the output and is possibly even required by the
    // caller.
    RetryHttpRequestInitializer httpRequestInitializer = new RetryHttpRequestInitializer(ImmutableList.of(404));
    httpRequestInitializer.setCustomErrors(createBigQueryClientCustomErrors());
    httpRequestInitializer.setWriteTimeout(options.getHTTPWriteTimeout());
    ImmutableList.Builder<HttpRequestInitializer> initBuilder = ImmutableList.builder();
    Credentials credential = options.getGcpCredential();
    initBuilder.add(credential == null ? new NullCredentialInitializer() : new HttpCredentialsAdapter(credential));
    initBuilder.add(new LatencyRecordingHttpRequestInitializer(API_METRIC_LABEL));
    initBuilder.add(httpRequestInitializer);
    HttpRequestInitializer chainInitializer = new ChainingHttpRequestInitializer(Iterables.toArray(initBuilder.build(), HttpRequestInitializer.class));
    return new Bigquery.Builder(Transport.getTransport(), Transport.getJsonFactory(), chainInitializer).setApplicationName(options.getAppName()).setGoogleClientRequestInitializer(options.getGoogleApiTrace());
}

Example 2 with RetryHttpRequestInitializer

use of org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer in project beam by apache.

the class V1TestUtil method getDatastore.

/**
 * Build a new datastore client.
 */
static Datastore getDatastore(PipelineOptions pipelineOptions, String projectId) {
    Credentials credential = pipelineOptions.as(GcpOptions.class).getGcpCredential();
    HttpRequestInitializer initializer;
    if (credential != null) {
        initializer = new ChainingHttpRequestInitializer(new HttpCredentialsAdapter(credential), new RetryHttpRequestInitializer());
    } else {
        initializer = new RetryHttpRequestInitializer();
    }
    DatastoreOptions.Builder builder = new DatastoreOptions.Builder().projectId(projectId).initializer(initializer);
    return DatastoreFactory.get().create(builder.build());
}

Example 3 with RetryHttpRequestInitializer

use of org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer in project beam by apache.

the class BigQueryServicesImplTest method setUp.

@Before
public void setUp() {
    MockitoAnnotations.initMocks(this);
    // Set up the MockHttpRequest for future inspection.
    request = new MockLowLevelHttpRequest() {

        int index = 0;

        @Override
        public LowLevelHttpResponse execute() throws IOException {
            Verify.verify(index < responses.length, "The number of HttpRequest invocation exceeded the number of prepared mock requests. Index: %d", index);
            return responses[index++];
        }
    };
    // A mock transport that lets us mock the API responses.
    MockHttpTransport transport = new MockHttpTransport.Builder().setLowLevelHttpRequest(request).build();
    // A sample BigQuery API client that uses default JsonFactory and RetryHttpInitializer.
    bigquery = new Bigquery.Builder(transport, Transport.getJsonFactory(), new RetryHttpRequestInitializer()).build();
    // Setup the ProcessWideContainer for testing metrics are set.
    MetricsContainerImpl container = new MetricsContainerImpl(null);
    MetricsEnvironment.setProcessWideContainer(container);
    MetricsEnvironment.setCurrentContainer(container);
}