Search in sources :

Example 16 with PhotoModel

use of org.datatransferproject.types.common.models.photos.PhotoModel in project data-transfer-project by google.

the class FlickrPhotosExporterTest method toCommonPhoto.

@Test
public void toCommonPhoto() {
    Photo photo = FlickrTestUtils.initializePhoto(PHOTO_TITLE, FETCHABLE_URL, PHOTO_DESCRIPTION, MEDIA_TYPE);
    PhotoModel photoModel = FlickrPhotosExporter.toCommonPhoto(photo, ALBUM_ID);
    assertThat(photoModel.getAlbumId()).isEqualTo(ALBUM_ID);
    assertThat(photoModel.getFetchableUrl()).isEqualTo(FETCHABLE_URL);
    assertThat(photoModel.getTitle()).isEqualTo(PHOTO_TITLE);
    assertThat(photoModel.getDescription()).isEqualTo(PHOTO_DESCRIPTION);
    assertThat(photoModel.getMediaType()).isEqualTo("image/jpeg");
}
Also used : PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) Photo(com.flickr4java.flickr.photos.Photo) Test(org.junit.Test)

Example 17 with PhotoModel

use of org.datatransferproject.types.common.models.photos.PhotoModel in project data-transfer-project by google.

the class GooglePhotosExporter method convertPhotosList.

private List<PhotoModel> convertPhotosList(Optional<String> albumId, GoogleMediaItem[] mediaItems, UUID jobId) throws IOException {
    List<PhotoModel> photos = new ArrayList<>(mediaItems.length);
    TempPhotosData tempPhotosData = null;
    InputStream stream = jobStore.getStream(jobId, createCacheKey()).getStream();
    if (stream != null) {
        tempPhotosData = new ObjectMapper().readValue(stream, TempPhotosData.class);
        stream.close();
    }
    for (GoogleMediaItem mediaItem : mediaItems) {
        if (mediaItem.getMediaMetadata().getPhoto() != null) {
            // TODO: address videos
            boolean shouldUpload = albumId.isPresent();
            if (tempPhotosData != null) {
                shouldUpload = shouldUpload || !tempPhotosData.isContainedPhotoId(mediaItem.getId());
            }
            if (shouldUpload) {
                PhotoModel photoModel = convertToPhotoModel(albumId, mediaItem);
                photos.add(photoModel);
                monitor.debug(() -> String.format("%s: Google exporting photo: %s", jobId, photoModel.getDataId()));
            }
        }
    }
    return photos;
}
Also used : TempPhotosData(org.datatransferproject.spi.transfer.types.TempPhotosData) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) ArrayList(java.util.ArrayList) GoogleMediaItem(org.datatransferproject.datatransfer.google.mediaModels.GoogleMediaItem) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper)

Example 18 with PhotoModel

use of org.datatransferproject.types.common.models.photos.PhotoModel in project data-transfer-project by google.

the class GooglePhotosExporterTest method exportAlbumFirstSet.

@Test
public void exportAlbumFirstSet() throws IOException, InvalidTokenException, PermissionDeniedException {
    setUpSingleAlbum();
    when(albumListResponse.getNextPageToken()).thenReturn(ALBUM_TOKEN);
    // Run test
    ExportResult<PhotosContainerResource> result = googlePhotosExporter.exportAlbums(null, Optional.empty(), uuid);
    // Check results
    // Verify correct methods were called
    verify(photosInterface).listAlbums(Optional.empty());
    verify(albumListResponse).getAlbums();
    // Check pagination token
    ContinuationData continuationData = result.getContinuationData();
    StringPaginationToken paginationToken = (StringPaginationToken) continuationData.getPaginationData();
    assertThat(paginationToken.getToken()).isEqualTo(ALBUM_TOKEN_PREFIX + ALBUM_TOKEN);
    // Check albums field of container
    Collection<PhotoAlbum> actualAlbums = result.getExportedData().getAlbums();
    assertThat(actualAlbums.stream().map(PhotoAlbum::getId).collect(Collectors.toList())).containsExactly(ALBUM_ID);
    // Check photos field of container (should be empty, even though there is a photo in the
    // original album)
    Collection<PhotoModel> actualPhotos = result.getExportedData().getPhotos();
    assertThat(actualPhotos).isEmpty();
    // Should be one container in the resource list
    List<ContainerResource> actualResources = continuationData.getContainerResources();
    assertThat(actualResources.stream().map(a -> ((IdOnlyContainerResource) a).getId()).collect(Collectors.toList())).containsExactly(ALBUM_ID);
}
Also used : ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) ALBUM_TOKEN_PREFIX(org.datatransferproject.datatransfer.google.photos.GooglePhotosExporter.ALBUM_TOKEN_PREFIX) GoogleMediaItem(org.datatransferproject.datatransfer.google.mediaModels.GoogleMediaItem) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) ArgumentMatchers.eq(org.mockito.ArgumentMatchers.eq) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PermissionDeniedException(org.datatransferproject.spi.transfer.types.PermissionDeniedException) Mockito.verifyNoInteractions(org.mockito.Mockito.verifyNoInteractions) AlbumListResponse(org.datatransferproject.datatransfer.google.mediaModels.AlbumListResponse) MediaMetadata(org.datatransferproject.datatransfer.google.mediaModels.MediaMetadata) TemporaryPerJobDataStore(org.datatransferproject.spi.cloud.storage.TemporaryPerJobDataStore) ArgumentCaptor(org.mockito.ArgumentCaptor) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) InputStreamWrapper(org.datatransferproject.spi.cloud.storage.TemporaryPerJobDataStore.InputStreamWrapper) GoogleCredentialFactory(org.datatransferproject.datatransfer.google.common.GoogleCredentialFactory) Before(org.junit.Before) JacksonFactory(com.google.api.client.json.jackson2.JacksonFactory) Collection(java.util.Collection) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) IOException(java.io.IOException) ContainerResource(org.datatransferproject.types.common.models.ContainerResource) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) UUID(java.util.UUID) PaginationData(org.datatransferproject.types.common.PaginationData) Truth.assertThat(com.google.common.truth.Truth.assertThat) Photo(org.datatransferproject.datatransfer.google.mediaModels.Photo) Collectors(java.util.stream.Collectors) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) Mockito.verify(org.mockito.Mockito.verify) PHOTO_TOKEN_PREFIX(org.datatransferproject.datatransfer.google.photos.GooglePhotosExporter.PHOTO_TOKEN_PREFIX) List(java.util.List) Assert.assertNull(org.junit.Assert.assertNull) Monitor(org.datatransferproject.api.launcher.Monitor) MediaItemSearchResponse(org.datatransferproject.datatransfer.google.mediaModels.MediaItemSearchResponse) TempPhotosData(org.datatransferproject.spi.transfer.types.TempPhotosData) Optional(java.util.Optional) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) GoogleAlbum(org.datatransferproject.datatransfer.google.mediaModels.GoogleAlbum) InvalidTokenException(org.datatransferproject.spi.transfer.types.InvalidTokenException) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) Mockito.mock(org.mockito.Mockito.mock) InputStream(java.io.InputStream) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) ContainerResource(org.datatransferproject.types.common.models.ContainerResource) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) Test(org.junit.Test)

Example 19 with PhotoModel

use of org.datatransferproject.types.common.models.photos.PhotoModel in project data-transfer-project by google.

the class KoofrPhotosImporter method importItem.

@Override
public ImportResult importItem(UUID jobId, IdempotentImportExecutor idempotentImportExecutor, TokensAndUrlAuthData authData, PhotosContainerResource resource) throws Exception {
    KoofrClient koofrClient = koofrClientFactory.create(authData);
    monitor.debug(() -> String.format("%s: Importing %s albums and %s photos before transmogrification", jobId, resource.getAlbums().size(), resource.getPhotos().size()));
    // Make the data Koofr compatible
    resource.transmogrify(transmogrificationConfig);
    monitor.debug(() -> String.format("%s: Importing %s albums and %s photos after transmogrification", jobId, resource.getAlbums().size(), resource.getPhotos().size()));
    for (PhotoAlbum album : resource.getAlbums()) {
        // Create a Koofr folder and then save the id with the mapping data
        idempotentImportExecutor.executeAndSwallowIOExceptions(album.getId(), album.getName(), () -> createAlbumFolder(album, koofrClient));
    }
    for (PhotoModel photoModel : resource.getPhotos()) {
        idempotentImportExecutor.executeAndSwallowIOExceptions(IdempotentImportExecutorHelper.getPhotoIdempotentId(photoModel), photoModel.getTitle(), () -> importSinglePhoto(photoModel, jobId, idempotentImportExecutor, koofrClient));
    }
    return ImportResult.OK;
}
Also used : PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) KoofrClient(org.datatransferproject.transfer.koofr.common.KoofrClient) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum)

Example 20 with PhotoModel

use of org.datatransferproject.types.common.models.photos.PhotoModel in project data-transfer-project by google.

the class KoofrPhotosExporterTest method testExport.

@Test
public void testExport() throws Exception {
    when(client.getRootPath()).thenReturn("/Data transfer");
    when(client.listRecursive("/Data transfer")).thenReturn(Fixtures.listRecursiveItems);
    when(client.fileLink("/Data transfer/Album 1/Photo 1.jpg")).thenReturn("https://app-1.koofr.net/content/files/get/Photo+1.jpg?base=TESTBASE");
    when(client.fileLink("/Data transfer/Album 1/Photo 2.jpg")).thenReturn("https://app-1.koofr.net/content/files/get/Photo+2.jpg?base=TESTBASE");
    when(client.fileLink("/Data transfer/Album 2 :heart:/Photo 3.jpg")).thenReturn("https://app-1.koofr.net/content/files/get/Photo+3.jpg?base=TESTBASE");
    UUID jobId = UUID.randomUUID();
    ExportResult<PhotosContainerResource> result = exporter.export(jobId, authData, Optional.empty());
    assertEquals(ExportResult.ResultType.END, result.getType());
    assertNull(result.getContinuationData());
    PhotosContainerResource exportedData = result.getExportedData();
    List<PhotoAlbum> expectedAlbums = ImmutableList.of(new PhotoAlbum("/Album 1", "Album 1", null), new PhotoAlbum("/Album 2 :heart:", "Album 2 ❤️", "Album 2 description ❤️"));
    assertEquals(expectedAlbums, exportedData.getAlbums());
    List<PhotoModel> expectedPhotos = ImmutableList.of(new PhotoModel("Photo 1.jpg", "https://app-1.koofr.net/content/files/get/Photo+1.jpg?base=TESTBASE", "Photo 1 description", "image/jpeg", "/Album 1/Photo 1.jpg", "/Album 1", false, new Date(1324824491000L)), new PhotoModel("Photo 2.jpg", "https://app-1.koofr.net/content/files/get/Photo+2.jpg?base=TESTBASE", null, "image/jpeg", "/Album 1/Photo 2.jpg", "/Album 1", false, new Date(1368774569000L)), new PhotoModel("Photo 3.jpg", "https://app-1.koofr.net/content/files/get/Photo+3.jpg?base=TESTBASE", "Photo 3 description", "image/jpeg", "/Album 2 :heart:/Photo 3.jpg", "/Album 2 :heart:", false, new Date(1489345497000L)));
    assertEquals(expectedPhotos, exportedData.getPhotos());
}
Also used : PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) UUID(java.util.UUID) Date(java.util.Date) Test(org.junit.Test)

Aggregations

PhotoModel (org.datatransferproject.types.common.models.photos.PhotoModel)44 PhotosContainerResource (org.datatransferproject.types.common.models.photos.PhotosContainerResource)29 PhotoAlbum (org.datatransferproject.types.common.models.photos.PhotoAlbum)24 Test (org.junit.Test)22 ContinuationData (org.datatransferproject.spi.transfer.types.ContinuationData)17 ExportResult (org.datatransferproject.spi.transfer.provider.ExportResult)16 ArrayList (java.util.ArrayList)14 UUID (java.util.UUID)14 IOException (java.io.IOException)12 IdOnlyContainerResource (org.datatransferproject.types.common.models.IdOnlyContainerResource)12 StringPaginationToken (org.datatransferproject.types.common.StringPaginationToken)10 TokensAndUrlAuthData (org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData)8 ByteArrayInputStream (java.io.ByteArrayInputStream)7 InputStream (java.io.InputStream)6 PaginationData (org.datatransferproject.types.common.PaginationData)6 Collection (java.util.Collection)5 List (java.util.List)5 Optional (java.util.Optional)5 Collectors (java.util.stream.Collectors)5 BatchMediaItemResponse (org.datatransferproject.datatransfer.google.mediaModels.BatchMediaItemResponse)5