Search in sources :

Example 6 with PhotoModel

use of org.datatransferproject.types.common.models.photos.PhotoModel in project data-transfer-project by google.

the class MicrosoftPhotosExporterTest method exportAlbumWithoutNextPage.

@Test
public void exportAlbumWithoutNextPage() throws IOException {
    // Setup
    MicrosoftDriveItem folderItem = setUpSingleAlbum();
    when(driveItemsResponse.getDriveItems()).thenReturn(new MicrosoftDriveItem[] { folderItem });
    when(driveItemsResponse.getNextPageLink()).thenReturn(null);
    StringPaginationToken inputPaginationToken = new StringPaginationToken(DRIVE_TOKEN_PREFIX + DRIVE_PAGE_URL);
    // Run
    ExportResult<PhotosContainerResource> result = microsoftPhotosExporter.exportOneDrivePhotos(null, Optional.empty(), Optional.of(inputPaginationToken), uuid);
    // Verify method calls
    verify(photosInterface).getDriveItems(Optional.empty(), Optional.of(DRIVE_PAGE_URL));
    verify(driveItemsResponse).getDriveItems();
    // Verify next pagination token is absent
    ContinuationData continuationData = result.getContinuationData();
    StringPaginationToken paginationToken = (StringPaginationToken) continuationData.getPaginationData();
    assertThat(paginationToken).isEqualTo(null);
    // Verify one album is ready for import
    Collection<PhotoAlbum> actualAlbums = result.getExportedData().getAlbums();
    assertThat(actualAlbums.stream().map(PhotoAlbum::getId).collect(Collectors.toList())).containsExactly(FOLDER_ID);
    // Verify photos should be empty (in the root)
    Collection<PhotoModel> actualPhotos = result.getExportedData().getPhotos();
    assertThat(actualPhotos).isEmpty();
    // Verify there is one container ready for sub-processing
    List<ContainerResource> actualResources = continuationData.getContainerResources();
    assertThat(actualResources.stream().map(a -> ((IdOnlyContainerResource) a).getId()).collect(Collectors.toList())).containsExactly(FOLDER_ID);
}
Also used : ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) Matchers(org.mockito.Matchers) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) Before(org.junit.Before) JacksonFactory(com.google.api.client.json.jackson2.JacksonFactory) Collection(java.util.Collection) IOException(java.io.IOException) ContainerResource(org.datatransferproject.types.common.models.ContainerResource) Test(org.junit.Test) UUID(java.util.UUID) Truth.assertThat(com.google.common.truth.Truth.assertThat) Collectors(java.util.stream.Collectors) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) Mockito(org.mockito.Mockito) List(java.util.List) Monitor(org.datatransferproject.api.launcher.Monitor) TokensAndUrlAuthData(org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData) DRIVE_TOKEN_PREFIX(org.datatransferproject.transfer.microsoft.photos.MicrosoftPhotosExporter.DRIVE_TOKEN_PREFIX) Optional(java.util.Optional) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) MicrosoftCredentialFactory(org.datatransferproject.transfer.microsoft.common.MicrosoftCredentialFactory) org.datatransferproject.transfer.microsoft.driveModels(org.datatransferproject.transfer.microsoft.driveModels) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) ContainerResource(org.datatransferproject.types.common.models.ContainerResource) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) Test(org.junit.Test)

Example 7 with PhotoModel

use of org.datatransferproject.types.common.models.photos.PhotoModel in project data-transfer-project by google.

the class MicrosoftPhotosExporterTest method exportPhotoWithoutNextPage.

@Test
public void exportPhotoWithoutNextPage() throws IOException {
    // Setup
    when(driveItemsResponse.getNextPageLink()).thenReturn(null);
    MicrosoftDriveItem photoItem = setUpSinglePhoto(IMAGE_URI, PHOTO_ID);
    when(driveItemsResponse.getDriveItems()).thenReturn(new MicrosoftDriveItem[] { photoItem });
    when(driveItemsResponse.getNextPageLink()).thenReturn(null);
    StringPaginationToken inputPaginationToken = new StringPaginationToken(DRIVE_TOKEN_PREFIX + DRIVE_PAGE_URL);
    IdOnlyContainerResource idOnlyContainerResource = new IdOnlyContainerResource(FOLDER_ID);
    // Run
    ExportResult<PhotosContainerResource> result = microsoftPhotosExporter.exportOneDrivePhotos(null, Optional.of(idOnlyContainerResource), Optional.of(inputPaginationToken), uuid);
    // Verify method calls
    verify(photosInterface).getDriveItems(Optional.of(FOLDER_ID), Optional.of(DRIVE_PAGE_URL));
    verify(driveItemsResponse).getDriveItems();
    // Verify next pagination token is absent
    ContinuationData continuationData = result.getContinuationData();
    StringPaginationToken paginationToken = (StringPaginationToken) continuationData.getPaginationData();
    assertThat(paginationToken).isEqualTo(null);
    // Verify no albums are exported
    Collection<PhotoAlbum> actualAlbums = result.getExportedData().getAlbums();
    assertThat(actualAlbums).isEmpty();
    // Verify one photo (in an album) should be exported
    Collection<PhotoModel> actualPhotos = result.getExportedData().getPhotos();
    assertThat(actualPhotos.stream().map(PhotoModel::getFetchableUrl).collect(Collectors.toList())).containsExactly(IMAGE_URI);
    assertThat(actualPhotos.stream().map(PhotoModel::getAlbumId).collect(Collectors.toList())).containsExactly(FOLDER_ID);
    assertThat(actualPhotos.stream().map(PhotoModel::getTitle).collect(Collectors.toList())).containsExactly(FILENAME);
    // Verify there are no containers ready for sub-processing
    List<ContainerResource> actualResources = continuationData.getContainerResources();
    assertThat(actualResources).isEmpty();
}
Also used : PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) ContainerResource(org.datatransferproject.types.common.models.ContainerResource) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) Test(org.junit.Test)

Example 8 with PhotoModel

use of org.datatransferproject.types.common.models.photos.PhotoModel in project data-transfer-project by google.

the class MicrosoftPhotosExporter method tryConvertDriveItemToPhotoModel.

private PhotoModel tryConvertDriveItemToPhotoModel(Optional<String> albumId, MicrosoftDriveItem driveItem, UUID jobId) {
    if (driveItem.file != null && driveItem.file.mimeType != null && driveItem.file.mimeType.startsWith("image/")) {
        PhotoModel photo = new PhotoModel(driveItem.name, driveItem.downloadUrl, driveItem.description, driveItem.file.mimeType, driveItem.id, albumId.orElse(null), false);
        monitor.debug(() -> String.format("%s: Microsoft OneDrive exporting photo: %s", jobId, photo));
        return photo;
    }
    return null;
}
Also used : PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel)

Example 9 with PhotoModel

use of org.datatransferproject.types.common.models.photos.PhotoModel in project data-transfer-project by google.

the class MicrosoftPhotosExporter method exportOneDrivePhotos.

@VisibleForTesting
ExportResult<PhotosContainerResource> exportOneDrivePhotos(TokensAndUrlAuthData authData, Optional<IdOnlyContainerResource> albumData, Optional<PaginationData> paginationData, UUID jobId) throws IOException {
    Optional<String> albumId = Optional.empty();
    if (albumData.isPresent()) {
        albumId = Optional.of(albumData.get().getId());
    }
    Optional<String> paginationUrl = getDrivePaginationToken(paginationData);
    MicrosoftDriveItemsResponse driveItemsResponse;
    if (paginationData.isPresent() || albumData.isPresent()) {
        driveItemsResponse = getOrCreatePhotosInterface(authData).getDriveItems(albumId, paginationUrl);
    } else {
        driveItemsResponse = getOrCreatePhotosInterface(authData).getDriveItemsFromSpecialFolder(MicrosoftSpecialFolder.FolderType.photos);
    }
    PaginationData nextPageData = SetNextPageToken(driveItemsResponse);
    ContinuationData continuationData = new ContinuationData(nextPageData);
    PhotosContainerResource containerResource;
    MicrosoftDriveItem[] driveItems = driveItemsResponse.getDriveItems();
    List<PhotoAlbum> albums = new ArrayList<>();
    List<PhotoModel> photos = new ArrayList<>();
    if (driveItems != null && driveItems.length > 0) {
        for (MicrosoftDriveItem driveItem : driveItems) {
            PhotoAlbum album = tryConvertDriveItemToPhotoAlbum(driveItem, jobId);
            if (album != null) {
                albums.add(album);
                continuationData.addContainerResource(new IdOnlyContainerResource(driveItem.id));
            }
            PhotoModel photo = tryConvertDriveItemToPhotoModel(albumId, driveItem, jobId);
            if (photo != null) {
                photos.add(photo);
            }
        }
    }
    ExportResult.ResultType result = nextPageData == null ? ExportResult.ResultType.END : ExportResult.ResultType.CONTINUE;
    containerResource = new PhotosContainerResource(albums, photos);
    return new ExportResult<>(result, containerResource, continuationData);
}
Also used : MicrosoftDriveItem(org.datatransferproject.transfer.microsoft.driveModels.MicrosoftDriveItem) PaginationData(org.datatransferproject.types.common.PaginationData) MicrosoftDriveItemsResponse(org.datatransferproject.transfer.microsoft.driveModels.MicrosoftDriveItemsResponse) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) ArrayList(java.util.ArrayList) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 10 with PhotoModel

use of org.datatransferproject.types.common.models.photos.PhotoModel in project data-transfer-project by google.

the class GooglePhotosImporterTest method importTwoPhotosWithFailure.

@Test
public void importTwoPhotosWithFailure() throws Exception {
    PhotoModel photoModel1 = new PhotoModel(PHOTO_TITLE, IMG_URI, PHOTO_DESCRIPTION, JPEG_MEDIA_TYPE, "oldPhotoID1", OLD_ALBUM_ID, false);
    PhotoModel photoModel2 = new PhotoModel(PHOTO_TITLE, IMG_URI, PHOTO_DESCRIPTION, JPEG_MEDIA_TYPE, "oldPhotoID2", OLD_ALBUM_ID, false);
    Mockito.when(googlePhotosInterface.uploadPhotoContent(any())).thenReturn("token1", "token2");
    BatchMediaItemResponse batchMediaItemResponse = new BatchMediaItemResponse(new NewMediaItemResult[] { buildMediaItemResult("token1", Code.OK_VALUE), buildMediaItemResult("token2", Code.UNAUTHENTICATED_VALUE) });
    Mockito.when(googlePhotosInterface.createPhotos(any(NewMediaItemUpload.class))).thenReturn(batchMediaItemResponse);
    long length = googlePhotosImporter.importPhotoBatch(UUID.randomUUID(), Mockito.mock(TokensAndUrlAuthData.class), Lists.newArrayList(photoModel1, photoModel2), executor, NEW_ALBUM_ID);
    // Only one photo of 32L imported
    assertEquals(32L, length);
    assertTrue(executor.isKeyCached(String.format("%s-%s", OLD_ALBUM_ID, "oldPhotoID1")));
    String failedDataId = String.format("%s-%s", OLD_ALBUM_ID, "oldPhotoID2");
    assertFalse(executor.isKeyCached(failedDataId));
    ErrorDetail errorDetail = executor.getErrors().iterator().next();
    assertEquals(failedDataId, errorDetail.id());
    assertThat(errorDetail.exception(), CoreMatchers.containsString("Media item could not be created."));
}
Also used : ErrorDetail(org.datatransferproject.types.transfer.errors.ErrorDetail) BatchMediaItemResponse(org.datatransferproject.datatransfer.google.mediaModels.BatchMediaItemResponse) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) TokensAndUrlAuthData(org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData) NewMediaItemUpload(org.datatransferproject.datatransfer.google.mediaModels.NewMediaItemUpload) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) Test(org.junit.Test)

Aggregations

PhotoModel (org.datatransferproject.types.common.models.photos.PhotoModel)44 PhotosContainerResource (org.datatransferproject.types.common.models.photos.PhotosContainerResource)29 PhotoAlbum (org.datatransferproject.types.common.models.photos.PhotoAlbum)24 Test (org.junit.Test)22 ContinuationData (org.datatransferproject.spi.transfer.types.ContinuationData)17 ExportResult (org.datatransferproject.spi.transfer.provider.ExportResult)16 ArrayList (java.util.ArrayList)14 UUID (java.util.UUID)14 IOException (java.io.IOException)12 IdOnlyContainerResource (org.datatransferproject.types.common.models.IdOnlyContainerResource)12 StringPaginationToken (org.datatransferproject.types.common.StringPaginationToken)10 TokensAndUrlAuthData (org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData)8 ByteArrayInputStream (java.io.ByteArrayInputStream)7 InputStream (java.io.InputStream)6 PaginationData (org.datatransferproject.types.common.PaginationData)6 Collection (java.util.Collection)5 List (java.util.List)5 Optional (java.util.Optional)5 Collectors (java.util.stream.Collectors)5 BatchMediaItemResponse (org.datatransferproject.datatransfer.google.mediaModels.BatchMediaItemResponse)5