Search in sources :

Example 6 with TokensAndUrlAuthData

use of org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData in project data-transfer-project by google.

the class KoofrVideosImporterTest method setUp.

@Before
public void setUp() throws Exception {
    server = new MockWebServer();
    server.start();
    client = mock(KoofrClient.class);
    clientFactory = mock(KoofrClientFactory.class);
    when(clientFactory.create(any())).thenReturn(client);
    monitor = mock(Monitor.class);
    importer = new KoofrVideosImporter(clientFactory, monitor);
    executor = mock(IdempotentImportExecutor.class);
    when(executor.executeAndSwallowIOExceptions(any(), any(), any())).then((InvocationOnMock invocation) -> {
        Callable<String> callable = invocation.getArgument(2);
        return callable.call();
    });
    authData = new TokensAndUrlAuthData("acc", "refresh", "");
}
Also used : KoofrClientFactory(org.datatransferproject.transfer.koofr.common.KoofrClientFactory) Monitor(org.datatransferproject.api.launcher.Monitor) InvocationOnMock(org.mockito.invocation.InvocationOnMock) MockWebServer(okhttp3.mockwebserver.MockWebServer) KoofrClient(org.datatransferproject.transfer.koofr.common.KoofrClient) TokensAndUrlAuthData(org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData) IdempotentImportExecutor(org.datatransferproject.spi.transfer.idempotentexecutor.IdempotentImportExecutor) Before(org.junit.Before)

Example 7 with TokensAndUrlAuthData

use of org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData in project data-transfer-project by google.

the class MicrosoftCalendarExportTest method setUp.

@Before
public void setUp() {
    client = new OkHttpClient.Builder().build();
    mapper = new ObjectMapper();
    transformerService = new TransformerServiceImpl();
    token = new TokensAndUrlAuthData("token123", "refreshToken", "tokenUrl");
    server = new MockWebServer();
}
Also used : TokensAndUrlAuthData(org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData) MockWebServer(com.squareup.okhttp.mockwebserver.MockWebServer) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) TransformerServiceImpl(org.datatransferproject.transfer.microsoft.transformer.TransformerServiceImpl) Before(org.junit.Before)

Example 8 with TokensAndUrlAuthData

use of org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData in project data-transfer-project by google.

the class GoogleVideosImporter method importItem.

@Override
public ImportResult importItem(UUID jobId, IdempotentImportExecutor executor, TokensAndUrlAuthData authData, VideosContainerResource data) throws Exception {
    if (data == null) {
        // Nothing to do
        return ImportResult.OK;
    }
    PhotosLibraryClient client;
    if (clientsMap.containsKey(jobId)) {
        client = clientsMap.get(jobId);
    } else {
        PhotosLibrarySettings settings = PhotosLibrarySettings.newBuilder().setCredentialsProvider(FixedCredentialsProvider.create(UserCredentials.newBuilder().setClientId(appCredentials.getKey()).setClientSecret(appCredentials.getSecret()).setAccessToken(new AccessToken(authData.getAccessToken(), new Date())).setRefreshToken(authData.getRefreshToken()).build())).build();
        client = PhotosLibraryClient.initialize(settings);
        clientsMap.put(jobId, client);
    }
    long bytes = 0L;
    // Uploads videos
    final Collection<VideoModel> videos = data.getVideos();
    if (videos != null && videos.size() > 0) {
        Stream<VideoModel> stream = videos.stream().filter(video -> shouldImport(video, executor)).map(this::transformVideoName);
        // We partition into groups of 49 as 50 is the maximum number of items that can be created in
        // one call. (We use 49 to avoid potential off by one errors)
        // https://developers.google.com/photos/library/guides/upload-media#creating-media-item
        final UnmodifiableIterator<List<VideoModel>> batches = Iterators.partition(stream.iterator(), 49);
        while (batches.hasNext()) {
            long batchBytes = importVideoBatch(batches.next(), client, executor);
            bytes += batchBytes;
        }
    }
    final ImportResult result = ImportResult.OK;
    return result.copyWithBytes(bytes);
}
Also used : RandomAccessFile(java.io.RandomAccessFile) DestinationMemoryFullException(org.datatransferproject.spi.transfer.types.DestinationMemoryFullException) Error(com.google.photos.library.v1.upload.UploadMediaItemResponse.Error) ImportResult(org.datatransferproject.spi.transfer.provider.ImportResult) Date(java.util.Date) PhotosLibrarySettings(com.google.photos.library.v1.PhotosLibrarySettings) FixedCredentialsProvider(com.google.api.gax.core.FixedCredentialsProvider) HashMap(java.util.HashMap) AppCredentials(org.datatransferproject.types.transfer.auth.AppCredentials) Iterators(com.google.common.collect.Iterators) ArrayList(java.util.ArrayList) TemporaryPerJobDataStore(org.datatransferproject.spi.cloud.storage.TemporaryPerJobDataStore) Strings(com.google.common.base.Strings) NewMediaItem(com.google.photos.library.v1.proto.NewMediaItem) Pair(org.apache.commons.lang3.tuple.Pair) Map(java.util.Map) Importer(org.datatransferproject.spi.transfer.provider.Importer) UploadMediaItemRequest(com.google.photos.library.v1.upload.UploadMediaItemRequest) NewMediaItemFactory(com.google.photos.library.v1.util.NewMediaItemFactory) Code(com.google.rpc.Code) ImageStreamProvider(org.datatransferproject.transfer.ImageStreamProvider) PhotosLibraryClient(com.google.photos.library.v1.PhotosLibraryClient) IdempotentImportExecutor(org.datatransferproject.spi.transfer.idempotentexecutor.IdempotentImportExecutor) UploadErrorException(org.datatransferproject.spi.transfer.types.UploadErrorException) Collection(java.util.Collection) Status(com.google.rpc.Status) MediaObject(org.datatransferproject.types.common.models.MediaObject) IOException(java.io.IOException) NewMediaItemResult(com.google.photos.library.v1.proto.NewMediaItemResult) UUID(java.util.UUID) UserCredentials(com.google.auth.oauth2.UserCredentials) File(java.io.File) FileNotFoundException(java.io.FileNotFoundException) VideosContainerResource(org.datatransferproject.types.common.models.videos.VideosContainerResource) BatchCreateMediaItemsResponse(com.google.photos.library.v1.proto.BatchCreateMediaItemsResponse) List(java.util.List) Stream(java.util.stream.Stream) Monitor(org.datatransferproject.api.launcher.Monitor) TokensAndUrlAuthData(org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData) InvalidArgumentException(com.google.api.gax.rpc.InvalidArgumentException) VideoModel(org.datatransferproject.types.common.models.videos.VideoModel) Preconditions(com.google.common.base.Preconditions) UploadMediaItemResponse(com.google.photos.library.v1.upload.UploadMediaItemResponse) VisibleForTesting(com.google.common.annotations.VisibleForTesting) UnmodifiableIterator(com.google.common.collect.UnmodifiableIterator) AccessToken(com.google.auth.oauth2.AccessToken) InputStream(java.io.InputStream) ImportResult(org.datatransferproject.spi.transfer.provider.ImportResult) AccessToken(com.google.auth.oauth2.AccessToken) PhotosLibrarySettings(com.google.photos.library.v1.PhotosLibrarySettings) ArrayList(java.util.ArrayList) List(java.util.List) PhotosLibraryClient(com.google.photos.library.v1.PhotosLibraryClient) VideoModel(org.datatransferproject.types.common.models.videos.VideoModel) Date(java.util.Date)

Example 9 with TokensAndUrlAuthData

use of org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData in project data-transfer-project by google.

the class KoofrClientTest method setUp.

@Before
public void setUp() throws IOException {
    server = new MockWebServer();
    server.start();
    httpClient = new OkHttpClient.Builder().build();
    mapper = new ObjectMapper();
    monitor = mock(Monitor.class);
    credentialFactory = mock(KoofrCredentialFactory.class);
    credential = new Credential.Builder(BearerToken.authorizationHeaderAccessMethod()).build();
    credential.setAccessToken("acc");
    credential.setExpirationTimeMilliseconds(null);
    when(credentialFactory.createCredential(any())).thenReturn(credential);
    client = new KoofrClient(server.url("").toString(), httpClient, httpClient, mapper, monitor, credentialFactory);
    authData = new TokensAndUrlAuthData("acc", "refresh", "");
    client.getOrCreateCredential(authData);
}
Also used : Monitor(org.datatransferproject.api.launcher.Monitor) MockWebServer(okhttp3.mockwebserver.MockWebServer) TokensAndUrlAuthData(org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Before(org.junit.Before)

Example 10 with TokensAndUrlAuthData

use of org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData in project data-transfer-project by google.

the class KoofrPhotosExporterTest method setUp.

@Before
public void setUp() throws Exception {
    client = mock(KoofrClient.class);
    clientFactory = mock(KoofrClientFactory.class);
    when(clientFactory.create(any())).thenReturn(client);
    monitor = mock(Monitor.class);
    exporter = new KoofrPhotosExporter(clientFactory, monitor);
    authData = new TokensAndUrlAuthData("acc", "refresh", "");
}
Also used : KoofrClientFactory(org.datatransferproject.transfer.koofr.common.KoofrClientFactory) Monitor(org.datatransferproject.api.launcher.Monitor) KoofrClient(org.datatransferproject.transfer.koofr.common.KoofrClient) TokensAndUrlAuthData(org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData) Before(org.junit.Before)

Aggregations

TokensAndUrlAuthData (org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData)14 Monitor (org.datatransferproject.api.launcher.Monitor)7 Before (org.junit.Before)6 IdempotentImportExecutor (org.datatransferproject.spi.transfer.idempotentexecutor.IdempotentImportExecutor)5 IOException (java.io.IOException)4 UUID (java.util.UUID)4 ExportInformation (org.datatransferproject.types.common.ExportInformation)4 Test (org.junit.Test)4 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)3 Collection (java.util.Collection)3 Collectors (java.util.stream.Collectors)3 ImportResult (org.datatransferproject.spi.transfer.provider.ImportResult)3 Importer (org.datatransferproject.spi.transfer.provider.Importer)3 KoofrClient (org.datatransferproject.transfer.koofr.common.KoofrClient)3 KoofrClientFactory (org.datatransferproject.transfer.koofr.common.KoofrClientFactory)3 Credential (com.google.api.client.auth.oauth2.Credential)2 VisibleForTesting (com.google.common.annotations.VisibleForTesting)2 Preconditions (com.google.common.base.Preconditions)2 Strings (com.google.common.base.Strings)2 HashMap (java.util.HashMap)2