Search in sources :

Example 26 with ExportResult

use of org.datatransferproject.spi.transfer.provider.ExportResult in project data-transfer-project by google.

the class MicrosoftPhotosExporterTest method exportAlbumAndPhotoWithNextPage.

@Test
public void exportAlbumAndPhotoWithNextPage() throws IOException {
    // Setup
    MicrosoftDriveItem folderItem = setUpSingleAlbum();
    MicrosoftDriveItem photoItem = setUpSinglePhoto(IMAGE_URI, PHOTO_ID);
    when(driveItemsResponse.getDriveItems()).thenReturn(new MicrosoftDriveItem[] { folderItem, photoItem });
    when(driveItemsResponse.getNextPageLink()).thenReturn(DRIVE_PAGE_URL);
    // Run
    ExportResult<PhotosContainerResource> result = microsoftPhotosExporter.exportOneDrivePhotos(null, Optional.empty(), Optional.empty(), uuid);
    // Verify method calls
    verify(photosInterface).getDriveItemsFromSpecialFolder(MicrosoftSpecialFolder.FolderType.photos);
    verify(driveItemsResponse).getDriveItems();
    // Verify pagination token is set
    ContinuationData continuationData = result.getContinuationData();
    StringPaginationToken paginationToken = (StringPaginationToken) continuationData.getPaginationData();
    assertThat(paginationToken.getToken()).isEqualTo(DRIVE_TOKEN_PREFIX + DRIVE_PAGE_URL);
    // Verify one album is ready for import
    Collection<PhotoAlbum> actualAlbums = result.getExportedData().getAlbums();
    assertThat(actualAlbums.stream().map(PhotoAlbum::getId).collect(Collectors.toList())).containsExactly(FOLDER_ID);
    // Verify one photo should be present (in the root Photos special folder)
    Collection<PhotoModel> actualPhotos = result.getExportedData().getPhotos();
    assertThat(actualPhotos.stream().map(PhotoModel::getFetchableUrl).collect(Collectors.toList())).containsExactly(IMAGE_URI);
    assertThat(actualPhotos.stream().map(PhotoModel::getAlbumId).collect(Collectors.toList())).containsExactly(null);
    assertThat(actualPhotos.stream().map(PhotoModel::getTitle).collect(Collectors.toList())).containsExactly(FILENAME);
    // Verify there is one container ready for sub-processing
    List<ContainerResource> actualResources = continuationData.getContainerResources();
    assertThat(actualResources.stream().map(a -> ((IdOnlyContainerResource) a).getId()).collect(Collectors.toList())).containsExactly(FOLDER_ID);
}
Also used : ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) Matchers(org.mockito.Matchers) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) Before(org.junit.Before) JacksonFactory(com.google.api.client.json.jackson2.JacksonFactory) Collection(java.util.Collection) IOException(java.io.IOException) ContainerResource(org.datatransferproject.types.common.models.ContainerResource) Test(org.junit.Test) UUID(java.util.UUID) Truth.assertThat(com.google.common.truth.Truth.assertThat) Collectors(java.util.stream.Collectors) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) Mockito(org.mockito.Mockito) List(java.util.List) Monitor(org.datatransferproject.api.launcher.Monitor) TokensAndUrlAuthData(org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData) DRIVE_TOKEN_PREFIX(org.datatransferproject.transfer.microsoft.photos.MicrosoftPhotosExporter.DRIVE_TOKEN_PREFIX) Optional(java.util.Optional) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) MicrosoftCredentialFactory(org.datatransferproject.transfer.microsoft.common.MicrosoftCredentialFactory) org.datatransferproject.transfer.microsoft.driveModels(org.datatransferproject.transfer.microsoft.driveModels) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) ContainerResource(org.datatransferproject.types.common.models.ContainerResource) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) Test(org.junit.Test)

Example 27 with ExportResult

use of org.datatransferproject.spi.transfer.provider.ExportResult in project data-transfer-project by google.

the class MicrosoftOfflineDataExporter method export.

@Override
public ExportResult<MicrosoftOfflineData> export(UUID jobId, TokenAuthData authData, Optional<ExportInformation> exportInformation) {
    try {
        String data = getDataFile(jobId, offlineDataUrl, authData);
        MicrosoftOfflineData offlineData = new MicrosoftOfflineData(data);
        return new ExportResult<>(ExportResult.ResultType.END, offlineData);
    } catch (IOException e) {
        // FIXME log error
        e.printStackTrace();
        return new ExportResult<>(e);
    }
}
Also used : MicrosoftOfflineData(org.datatransferproject.transfer.microsoft.spi.types.MicrosoftOfflineData) IOException(java.io.IOException) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult)

Example 28 with ExportResult

use of org.datatransferproject.spi.transfer.provider.ExportResult in project data-transfer-project by google.

the class CallableExporter method call.

@Override
public ExportResult call() throws Exception {
    boolean success = false;
    Stopwatch stopwatch = Stopwatch.createStarted();
    try {
        ExportResult result = exporterProvider.get().export(jobId, authData, exportInformation);
        success = result.getType() != ExportResult.ResultType.ERROR;
        return result;
    } finally {
        metricRecorder.exportPageAttemptFinished(JobMetadata.getDataType(), JobMetadata.getExportService(), success, stopwatch.elapsed());
    }
}
Also used : Stopwatch(com.google.common.base.Stopwatch) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult)

Example 29 with ExportResult

use of org.datatransferproject.spi.transfer.provider.ExportResult in project data-transfer-project by google.

the class GoogleVideosExporter method exportVideos.

@VisibleForTesting
ExportResult<VideosContainerResource> exportVideos(TokensAndUrlAuthData authData, Optional<StringPaginationToken> paginationData) throws IOException {
    Optional<String> paginationToken = paginationData.map(StringPaginationToken::getToken);
    MediaItemSearchResponse mediaItemSearchResponse = getOrCreateVideosInterface(authData).listVideoItems(paginationToken);
    PaginationData nextPageData = null;
    if (!Strings.isNullOrEmpty(mediaItemSearchResponse.getNextPageToken())) {
        nextPageData = new StringPaginationToken(mediaItemSearchResponse.getNextPageToken());
    }
    ContinuationData continuationData = new ContinuationData(nextPageData);
    VideosContainerResource containerResource = null;
    GoogleMediaItem[] mediaItems = mediaItemSearchResponse.getMediaItems();
    if (mediaItems != null && mediaItems.length > 0) {
        List<VideoModel> videos = convertVideosList(mediaItems);
        containerResource = new VideosContainerResource(null, videos);
    }
    ResultType resultType = ResultType.CONTINUE;
    if (nextPageData == null) {
        resultType = ResultType.END;
    }
    return new ExportResult<>(resultType, containerResource, continuationData);
}
Also used : PaginationData(org.datatransferproject.types.common.PaginationData) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) ResultType(org.datatransferproject.spi.transfer.provider.ExportResult.ResultType) VideoModel(org.datatransferproject.types.common.models.videos.VideoModel) MediaItemSearchResponse(org.datatransferproject.datatransfer.google.mediaModels.MediaItemSearchResponse) VideosContainerResource(org.datatransferproject.types.common.models.videos.VideosContainerResource) GoogleMediaItem(org.datatransferproject.datatransfer.google.mediaModels.GoogleMediaItem) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 30 with ExportResult

use of org.datatransferproject.spi.transfer.provider.ExportResult in project data-transfer-project by google.

the class GoogleCalendarExporterTest method exportCalendarFirstSet.

@Test
public void exportCalendarFirstSet() throws IOException {
    setUpSingleCalendarResponse();
    // Looking at first page, with at least one page after it
    calendarListResponse.setNextPageToken(NEXT_TOKEN);
    // Run test
    ExportResult<CalendarContainerResource> result = googleCalendarExporter.export(JOB_ID, null, Optional.empty());
    // Check results
    // Verify correct methods were called
    verify(calendarClient).calendarList();
    verify(calendarCalendarList).list();
    verify(calendarListRequest).execute();
    // Check pagination token
    ContinuationData continuationData = (ContinuationData) result.getContinuationData();
    StringPaginationToken paginationToken = (StringPaginationToken) continuationData.getPaginationData();
    assertThat(paginationToken.getToken()).isEqualTo(CALENDAR_TOKEN_PREFIX + NEXT_TOKEN);
    // Check calendars
    Collection<CalendarModel> actualCalendars = result.getExportedData().getCalendars();
    assertThat(actualCalendars.stream().map(CalendarModel::getId).collect(Collectors.toList())).containsExactly(CALENDAR_ID);
    // Check events (should be empty, even though there is an event in the calendar)
    Collection<CalendarEventModel> actualEvents = result.getExportedData().getEvents();
    assertThat(actualEvents).isEmpty();
    // Should be one container in the resource list
    List<ContainerResource> actualResources = continuationData.getContainerResources();
    assertThat(actualResources.stream().map(a -> ((IdOnlyContainerResource) a).getId()).collect(Collectors.toList())).containsExactly(CALENDAR_ID);
}
Also used : MAX_ATTENDEES(org.datatransferproject.datatransfer.google.common.GoogleStaticObjects.MAX_ATTENDEES) CalendarContainerResource(org.datatransferproject.types.common.models.calendar.CalendarContainerResource) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult) CalendarEventModel(org.datatransferproject.types.common.models.calendar.CalendarEventModel) Mockito.verifyNoInteractions(org.mockito.Mockito.verifyNoInteractions) EVENT_TOKEN_PREFIX(org.datatransferproject.datatransfer.google.common.GoogleStaticObjects.EVENT_TOKEN_PREFIX) ExportInformation(org.datatransferproject.types.common.ExportInformation) Events(com.google.api.services.calendar.model.Events) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) Calendar(com.google.api.services.calendar.Calendar) GoogleCredentialFactory(org.datatransferproject.datatransfer.google.common.GoogleCredentialFactory) Before(org.junit.Before) Event(com.google.api.services.calendar.model.Event) InOrder(org.mockito.InOrder) CalendarList(com.google.api.services.calendar.model.CalendarList) Collection(java.util.Collection) IOException(java.io.IOException) ContainerResource(org.datatransferproject.types.common.models.ContainerResource) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) UUID(java.util.UUID) PaginationData(org.datatransferproject.types.common.PaginationData) Truth.assertThat(com.google.common.truth.Truth.assertThat) CALENDAR_TOKEN_PREFIX(org.datatransferproject.datatransfer.google.common.GoogleStaticObjects.CALENDAR_TOKEN_PREFIX) Collectors(java.util.stream.Collectors) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) Mockito.verify(org.mockito.Mockito.verify) CalendarModel(org.datatransferproject.types.common.models.calendar.CalendarModel) Mockito(org.mockito.Mockito) List(java.util.List) CalendarListEntry(com.google.api.services.calendar.model.CalendarListEntry) Optional(java.util.Optional) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) Collections(java.util.Collections) Mockito.mock(org.mockito.Mockito.mock) CalendarContainerResource(org.datatransferproject.types.common.models.calendar.CalendarContainerResource) ContainerResource(org.datatransferproject.types.common.models.ContainerResource) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) CalendarModel(org.datatransferproject.types.common.models.calendar.CalendarModel) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) CalendarEventModel(org.datatransferproject.types.common.models.calendar.CalendarEventModel) CalendarContainerResource(org.datatransferproject.types.common.models.calendar.CalendarContainerResource) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) Test(org.junit.Test)

Aggregations

ExportResult (org.datatransferproject.spi.transfer.provider.ExportResult)44 ContinuationData (org.datatransferproject.spi.transfer.types.ContinuationData)33 ArrayList (java.util.ArrayList)26 IOException (java.io.IOException)23 StringPaginationToken (org.datatransferproject.types.common.StringPaginationToken)22 PhotosContainerResource (org.datatransferproject.types.common.models.photos.PhotosContainerResource)21 PaginationData (org.datatransferproject.types.common.PaginationData)19 IdOnlyContainerResource (org.datatransferproject.types.common.models.IdOnlyContainerResource)18 PhotoAlbum (org.datatransferproject.types.common.models.photos.PhotoAlbum)16 PhotoModel (org.datatransferproject.types.common.models.photos.PhotoModel)16 ResultType (org.datatransferproject.spi.transfer.provider.ExportResult.ResultType)14 List (java.util.List)11 Optional (java.util.Optional)9 UUID (java.util.UUID)9 Collectors (java.util.stream.Collectors)8 VisibleForTesting (com.google.common.annotations.VisibleForTesting)7 ImmutableList (com.google.common.collect.ImmutableList)7 Collection (java.util.Collection)6 TokensAndUrlAuthData (org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData)6 Before (org.junit.Before)6