Search in sources :

Example 56 with HttpTransport

use of com.google.api.client.http.HttpTransport in project data-transfer-project by google.

the class MicrosoftTransferExtension method initialize.

@Override
public void initialize(ExtensionContext context) {
    // times.
    if (initialized)
        return;
    TemporaryPerJobDataStore jobStore = context.getService(TemporaryPerJobDataStore.class);
    HttpTransport httpTransport = context.getService(HttpTransport.class);
    JsonFactory jsonFactory = context.getService(JsonFactory.class);
    TransformerService transformerService = new TransformerServiceImpl();
    OkHttpClient client = new OkHttpClient.Builder().build();
    ObjectMapper mapper = new ObjectMapper();
    AppCredentials appCredentials;
    try {
        appCredentials = context.getService(AppCredentialStore.class).getAppCredentials("MICROSOFT_KEY", "MICROSOFT_SECRET");
    } catch (IOException e) {
        Monitor monitor = context.getMonitor();
        monitor.info(() -> "Unable to retrieve Microsoft AppCredentials. Did you set MICROSOFT_KEY and MICROSOFT_SECRET?");
        return;
    }
    // Create the MicrosoftCredentialFactory with the given {@link AppCredentials}.
    MicrosoftCredentialFactory credentialFactory = new MicrosoftCredentialFactory(httpTransport, jsonFactory, appCredentials);
    Monitor monitor = context.getMonitor();
    ImmutableMap.Builder<String, Importer> importBuilder = ImmutableMap.builder();
    importBuilder.put(CONTACTS, new MicrosoftContactsImporter(BASE_GRAPH_URL, client, mapper, transformerService));
    importBuilder.put(CALENDAR, new MicrosoftCalendarImporter(BASE_GRAPH_URL, client, mapper, transformerService));
    importBuilder.put(PHOTOS, new MicrosoftPhotosImporter(BASE_GRAPH_URL, client, mapper, jobStore, monitor, credentialFactory));
    importerMap = importBuilder.build();
    ImmutableMap.Builder<String, Exporter> exporterBuilder = ImmutableMap.builder();
    exporterBuilder.put(CONTACTS, new MicrosoftContactsExporter(BASE_GRAPH_URL, client, mapper, transformerService));
    exporterBuilder.put(CALENDAR, new MicrosoftCalendarExporter(BASE_GRAPH_URL, client, mapper, transformerService));
    exporterBuilder.put(PHOTOS, new MicrosoftPhotosExporter(credentialFactory, jsonFactory, monitor));
    exporterBuilder.put(OFFLINE_DATA, new MicrosoftOfflineDataExporter(BASE_GRAPH_URL, client, mapper));
    exporterMap = exporterBuilder.build();
    initialized = true;
}
Also used : OkHttpClient(okhttp3.OkHttpClient) MicrosoftCalendarImporter(org.datatransferproject.transfer.microsoft.calendar.MicrosoftCalendarImporter) AppCredentials(org.datatransferproject.types.transfer.auth.AppCredentials) JsonFactory(com.google.api.client.json.JsonFactory) MicrosoftOfflineDataExporter(org.datatransferproject.transfer.microsoft.offline.MicrosoftOfflineDataExporter) MicrosoftPhotosExporter(org.datatransferproject.transfer.microsoft.photos.MicrosoftPhotosExporter) MicrosoftContactsExporter(org.datatransferproject.transfer.microsoft.contacts.MicrosoftContactsExporter) Exporter(org.datatransferproject.spi.transfer.provider.Exporter) MicrosoftCalendarExporter(org.datatransferproject.transfer.microsoft.calendar.MicrosoftCalendarExporter) MicrosoftCalendarExporter(org.datatransferproject.transfer.microsoft.calendar.MicrosoftCalendarExporter) Monitor(org.datatransferproject.api.launcher.Monitor) MicrosoftPhotosExporter(org.datatransferproject.transfer.microsoft.photos.MicrosoftPhotosExporter) TransformerServiceImpl(org.datatransferproject.transfer.microsoft.transformer.TransformerServiceImpl) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Importer(org.datatransferproject.spi.transfer.provider.Importer) MicrosoftPhotosImporter(org.datatransferproject.transfer.microsoft.photos.MicrosoftPhotosImporter) MicrosoftContactsImporter(org.datatransferproject.transfer.microsoft.contacts.MicrosoftContactsImporter) MicrosoftCalendarImporter(org.datatransferproject.transfer.microsoft.calendar.MicrosoftCalendarImporter) TemporaryPerJobDataStore(org.datatransferproject.spi.cloud.storage.TemporaryPerJobDataStore) MicrosoftContactsImporter(org.datatransferproject.transfer.microsoft.contacts.MicrosoftContactsImporter) MicrosoftOfflineDataExporter(org.datatransferproject.transfer.microsoft.offline.MicrosoftOfflineDataExporter) MicrosoftCredentialFactory(org.datatransferproject.transfer.microsoft.common.MicrosoftCredentialFactory) IOException(java.io.IOException) MicrosoftPhotosImporter(org.datatransferproject.transfer.microsoft.photos.MicrosoftPhotosImporter) ImmutableMap(com.google.common.collect.ImmutableMap) HttpTransport(com.google.api.client.http.HttpTransport) MicrosoftContactsExporter(org.datatransferproject.transfer.microsoft.contacts.MicrosoftContactsExporter) TransformerService(org.datatransferproject.transfer.microsoft.transformer.TransformerService)

Example 57 with HttpTransport

use of com.google.api.client.http.HttpTransport in project data-transfer-project by google.

the class KoofrTransferExtension method initialize.

@Override
public void initialize(ExtensionContext context) {
    // rather than throwing if called multiple times.
    if (initialized)
        return;
    JobStore jobStore = context.getService(JobStore.class);
    HttpTransport httpTransport = context.getService(HttpTransport.class);
    JsonFactory jsonFactory = context.getService(JsonFactory.class);
    OkHttpClient client = new OkHttpClient.Builder().build();
    ObjectMapper mapper = new ObjectMapper();
    AppCredentials appCredentials;
    try {
        appCredentials = context.getService(AppCredentialStore.class).getAppCredentials("KOOFR_KEY", "KOOFR_SECRET");
    } catch (IOException e) {
        Monitor monitor = context.getMonitor();
        monitor.info(() -> "Unable to retrieve Koofr AppCredentials. Did you set KOOFR_KEY and KOOFR_SECRET?");
        return;
    }
    // Create the KoofrCredentialFactory with the given {@link AppCredentials}.
    KoofrCredentialFactory credentialFactory = new KoofrCredentialFactory(httpTransport, jsonFactory, appCredentials);
    Monitor monitor = context.getMonitor();
    int fileUploadReadTimeout = context.getSetting("koofrFileUploadReadTimeout", 60000);
    int fileUploadWriteTimeout = context.getSetting("koofrFileUploadWriteTimeout", 60000);
    monitor.info(() -> format("Configuring Koofr HTTP file upload client with read timeout %d ms and write timeout %d ms", fileUploadReadTimeout, fileUploadWriteTimeout));
    OkHttpClient fileUploadClient = client.newBuilder().readTimeout(fileUploadReadTimeout, TimeUnit.MILLISECONDS).writeTimeout(fileUploadReadTimeout, TimeUnit.MILLISECONDS).build();
    KoofrClientFactory koofrClientFactory = new KoofrClientFactory(BASE_API_URL, client, fileUploadClient, mapper, monitor, credentialFactory);
    ImmutableMap.Builder<String, Importer> importBuilder = ImmutableMap.builder();
    importBuilder.put(PHOTOS, new KoofrPhotosImporter(koofrClientFactory, monitor, jobStore));
    importBuilder.put(VIDEOS, new KoofrVideosImporter(koofrClientFactory, monitor));
    importerMap = importBuilder.build();
    ImmutableMap.Builder<String, Exporter> exportBuilder = ImmutableMap.builder();
    exportBuilder.put(PHOTOS, new KoofrPhotosExporter(koofrClientFactory, monitor));
    exportBuilder.put(VIDEOS, new KoofrVideosExporter(koofrClientFactory, monitor));
    exporterMap = exportBuilder.build();
    initialized = true;
}
Also used : KoofrPhotosExporter(org.datatransferproject.transfer.koofr.photos.KoofrPhotosExporter) OkHttpClient(okhttp3.OkHttpClient) KoofrClientFactory(org.datatransferproject.transfer.koofr.common.KoofrClientFactory) AppCredentials(org.datatransferproject.types.transfer.auth.AppCredentials) KoofrCredentialFactory(org.datatransferproject.transfer.koofr.common.KoofrCredentialFactory) JsonFactory(com.google.api.client.json.JsonFactory) JobStore(org.datatransferproject.spi.cloud.storage.JobStore) KoofrVideosImporter(org.datatransferproject.transfer.koofr.videos.KoofrVideosImporter) IOException(java.io.IOException) Exporter(org.datatransferproject.spi.transfer.provider.Exporter) KoofrPhotosExporter(org.datatransferproject.transfer.koofr.photos.KoofrPhotosExporter) KoofrVideosExporter(org.datatransferproject.transfer.koofr.videos.KoofrVideosExporter) ImmutableMap(com.google.common.collect.ImmutableMap) HttpTransport(com.google.api.client.http.HttpTransport) KoofrPhotosImporter(org.datatransferproject.transfer.koofr.photos.KoofrPhotosImporter) Monitor(org.datatransferproject.api.launcher.Monitor) KoofrVideosExporter(org.datatransferproject.transfer.koofr.videos.KoofrVideosExporter) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) KoofrVideosImporter(org.datatransferproject.transfer.koofr.videos.KoofrVideosImporter) Importer(org.datatransferproject.spi.transfer.provider.Importer) KoofrPhotosImporter(org.datatransferproject.transfer.koofr.photos.KoofrPhotosImporter)

Example 58 with HttpTransport

use of com.google.api.client.http.HttpTransport in project druid by druid-io.

the class GoogleStorageDruidModule method getGoogleStorage.

@Provides
@LazySingleton
public GoogleStorage getGoogleStorage(final GoogleAccountConfig config) throws IOException, GeneralSecurityException {
    LOG.info("Building Cloud Storage Client...");
    HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
    JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
    GoogleCredential credential = GoogleCredential.getApplicationDefault(httpTransport, jsonFactory);
    if (credential.createScopedRequired()) {
        credential = credential.createScoped(StorageScopes.all());
    }
    Storage storage = new Storage.Builder(httpTransport, jsonFactory, credential).setApplicationName(APPLICATION_NAME).build();
    return new GoogleStorage(storage);
}
Also used : HttpTransport(com.google.api.client.http.HttpTransport) GoogleNetHttpTransport(com.google.api.client.googleapis.javanet.GoogleNetHttpTransport) Storage(com.google.api.services.storage.Storage) JsonFactory(com.google.api.client.json.JsonFactory) GoogleCredential(com.google.api.client.googleapis.auth.oauth2.GoogleCredential) LazySingleton(io.druid.guice.LazySingleton) Provides(com.google.inject.Provides)

Example 59 with HttpTransport

use of com.google.api.client.http.HttpTransport in project beam by apache.

the class RetryHttpRequestInitializerTest method setUp.

@Before
public void setUp() {
    MockitoAnnotations.initMocks(this);
    HttpTransport lowLevelTransport = new HttpTransport() {

        @Override
        protected LowLevelHttpRequest buildRequest(String method, String url) throws IOException {
            return mockLowLevelRequest;
        }
    };
    // Retry initializer will pass through to credential, since we can have
    // only a single HttpRequestInitializer, and we use multiple Credential
    // types in the SDK, not all of which allow for retry configuration.
    RetryHttpRequestInitializer initializer = new RetryHttpRequestInitializer(new MockNanoClock(), new Sleeper() {

        @Override
        public void sleep(long millis) throws InterruptedException {
        }
    }, Arrays.asList(418), mockHttpResponseInterceptor);
    storage = new Storage.Builder(lowLevelTransport, jsonFactory, initializer).setApplicationName("test").build();
}
Also used : MockHttpTransport(com.google.api.client.testing.http.MockHttpTransport) HttpTransport(com.google.api.client.http.HttpTransport) Storage(com.google.api.services.storage.Storage) Matchers.anyString(org.mockito.Matchers.anyString) Sleeper(com.google.api.client.util.Sleeper) Before(org.junit.Before)

Example 60 with HttpTransport

use of com.google.api.client.http.HttpTransport in project beam by apache.

the class BigqueryMatcher method newBigqueryClient.

@VisibleForTesting
Bigquery newBigqueryClient(String applicationName) {
    HttpTransport transport = Transport.getTransport();
    JsonFactory jsonFactory = Transport.getJsonFactory();
    Credentials credential = getDefaultCredential();
    return new Bigquery.Builder(transport, jsonFactory, new HttpCredentialsAdapter(credential)).setApplicationName(applicationName).build();
}
Also used : HttpTransport(com.google.api.client.http.HttpTransport) HttpCredentialsAdapter(com.google.auth.http.HttpCredentialsAdapter) JsonFactory(com.google.api.client.json.JsonFactory) GoogleCredentials(com.google.auth.oauth2.GoogleCredentials) Credentials(com.google.auth.Credentials) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Aggregations

HttpTransport (com.google.api.client.http.HttpTransport)106 MockHttpTransport (com.google.api.client.testing.http.MockHttpTransport)40 JsonFactory (com.google.api.client.json.JsonFactory)35 HttpRequest (com.google.api.client.http.HttpRequest)29 NetHttpTransport (com.google.api.client.http.javanet.NetHttpTransport)28 JacksonFactory (com.google.api.client.json.jackson2.JacksonFactory)28 GoogleCredential (com.google.api.client.googleapis.auth.oauth2.GoogleCredential)27 IOException (java.io.IOException)24 GoogleNetHttpTransport (com.google.api.client.googleapis.javanet.GoogleNetHttpTransport)21 HttpResponse (com.google.api.client.http.HttpResponse)21 GenericUrl (com.google.api.client.http.GenericUrl)15 MockLowLevelHttpRequest (com.google.api.client.testing.http.MockLowLevelHttpRequest)12 MockLowLevelHttpResponse (com.google.api.client.testing.http.MockLowLevelHttpResponse)11 Credential (com.google.api.client.auth.oauth2.Credential)10 HttpRequestFactory (com.google.api.client.http.HttpRequestFactory)10 Storage (com.google.api.services.storage.Storage)9 HttpRequestInitializer (com.google.api.client.http.HttpRequestInitializer)8 Test (org.junit.Test)8 Monitor (org.datatransferproject.api.launcher.Monitor)7 ErrorTransport (com.google.api.client.googleapis.json.GoogleJsonErrorTest.ErrorTransport)6