Search in sources :

Example 36 with PhotosContainerResource

use of org.datatransferproject.types.common.models.photos.PhotosContainerResource in project data-transfer-project by google.

the class GooglePhotosExporterTest method exportAlbumSubsequentSet.

@Test
public void exportAlbumSubsequentSet() throws IOException, InvalidTokenException, PermissionDeniedException {
    setUpSingleAlbum();
    when(albumListResponse.getNextPageToken()).thenReturn(null);
    StringPaginationToken inputPaginationToken = new StringPaginationToken(ALBUM_TOKEN_PREFIX + ALBUM_TOKEN);
    // Run test
    ExportResult<PhotosContainerResource> result = googlePhotosExporter.exportAlbums(null, Optional.of(inputPaginationToken), uuid);
    // Check results
    // Verify correct methods were called
    verify(photosInterface).listAlbums(Optional.of(ALBUM_TOKEN));
    verify(albumListResponse).getAlbums();
    // Check pagination token - should be absent
    ContinuationData continuationData = result.getContinuationData();
    StringPaginationToken paginationData = (StringPaginationToken) continuationData.getPaginationData();
    assertThat(paginationData.getToken()).isEqualTo(GooglePhotosExporter.PHOTO_TOKEN_PREFIX);
}
Also used : PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) Test(org.junit.Test)

Example 37 with PhotosContainerResource

use of org.datatransferproject.types.common.models.photos.PhotosContainerResource in project data-transfer-project by google.

the class GooglePhotosExporterTest method exportPhotoFirstSet.

@Test
public void exportPhotoFirstSet() throws IOException, InvalidTokenException, PermissionDeniedException {
    setUpSingleAlbum();
    when(albumListResponse.getNextPageToken()).thenReturn(null);
    GoogleMediaItem mediaItem = setUpSinglePhoto(IMG_URI, PHOTO_ID);
    when(mediaItemSearchResponse.getMediaItems()).thenReturn(new GoogleMediaItem[] { mediaItem });
    when(mediaItemSearchResponse.getNextPageToken()).thenReturn(PHOTO_TOKEN);
    IdOnlyContainerResource idOnlyContainerResource = new IdOnlyContainerResource(ALBUM_ID);
    ExportResult<PhotosContainerResource> result = googlePhotosExporter.exportPhotos(null, Optional.of(idOnlyContainerResource), Optional.empty(), uuid);
    // Check results
    // Verify correct methods were called
    verify(photosInterface).listMediaItems(Optional.of(ALBUM_ID), Optional.empty());
    verify(mediaItemSearchResponse).getMediaItems();
    // Check pagination
    ContinuationData continuationData = result.getContinuationData();
    StringPaginationToken paginationToken = (StringPaginationToken) continuationData.getPaginationData();
    assertThat(paginationToken.getToken()).isEqualTo(PHOTO_TOKEN_PREFIX + PHOTO_TOKEN);
    // Check albums field of container (should be empty)
    Collection<PhotoAlbum> actualAlbums = result.getExportedData().getAlbums();
    assertThat(actualAlbums).isEmpty();
    // Check photos field of container
    Collection<PhotoModel> actualPhotos = result.getExportedData().getPhotos();
    assertThat(actualPhotos.stream().map(PhotoModel::getFetchableUrl).collect(Collectors.toList())).containsExactly(// for download
    IMG_URI + "=d");
    assertThat(actualPhotos.stream().map(PhotoModel::getAlbumId).collect(Collectors.toList())).containsExactly(ALBUM_ID);
    assertThat(actualPhotos.stream().map(PhotoModel::getTitle).collect(Collectors.toList())).containsExactly(FILENAME);
}
Also used : PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) GoogleMediaItem(org.datatransferproject.datatransfer.google.mediaModels.GoogleMediaItem) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) Test(org.junit.Test)

Example 38 with PhotosContainerResource

use of org.datatransferproject.types.common.models.photos.PhotosContainerResource in project data-transfer-project by google.

the class GooglePhotosExporterTest method onlyExportAlbumlessPhoto.

@Test
public /* Tests that when there is no album information passed along to exportPhotos, only albumless
  photos are exported.
  */
void onlyExportAlbumlessPhoto() throws IOException, InvalidTokenException, PermissionDeniedException {
    // Set up - two photos will be returned by a media item search without an album id, but one of
    // them will have already been put into the list of contained photos
    String containedPhotoUri = "contained photo uri";
    String containedPhotoId = "contained photo id";
    GoogleMediaItem containedPhoto = setUpSinglePhoto(containedPhotoUri, containedPhotoId);
    String albumlessPhotoUri = "albumless photo uri";
    String albumlessPhotoId = "albumless photo id";
    GoogleMediaItem albumlessPhoto = setUpSinglePhoto(albumlessPhotoUri, albumlessPhotoId);
    MediaItemSearchResponse mediaItemSearchResponse = mock(MediaItemSearchResponse.class);
    when(photosInterface.listMediaItems(eq(Optional.empty()), eq(Optional.empty()))).thenReturn(mediaItemSearchResponse);
    when(mediaItemSearchResponse.getMediaItems()).thenReturn(new GoogleMediaItem[] { containedPhoto, albumlessPhoto });
    when(mediaItemSearchResponse.getNextPageToken()).thenReturn(null);
    TempPhotosData tempPhotosData = new TempPhotosData(uuid);
    tempPhotosData.addContainedPhotoId(containedPhotoId);
    InputStream stream = GooglePhotosExporter.convertJsonToInputStream(tempPhotosData);
    when(jobStore.getStream(uuid, "tempPhotosData")).thenReturn(new InputStreamWrapper(stream));
    // Run test
    ExportResult<PhotosContainerResource> result = googlePhotosExporter.exportPhotos(null, Optional.empty(), Optional.empty(), uuid);
    // Check results
    assertThat(result.getExportedData().getPhotos().stream().map(PhotoModel::getFetchableUrl).collect(Collectors.toList())).containsExactly(// download
    albumlessPhotoUri + "=d");
}
Also used : PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) TempPhotosData(org.datatransferproject.spi.transfer.types.TempPhotosData) InputStreamWrapper(org.datatransferproject.spi.cloud.storage.TemporaryPerJobDataStore.InputStreamWrapper) InputStream(java.io.InputStream) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) GoogleMediaItem(org.datatransferproject.datatransfer.google.mediaModels.GoogleMediaItem) MediaItemSearchResponse(org.datatransferproject.datatransfer.google.mediaModels.MediaItemSearchResponse) Test(org.junit.Test)

Example 39 with PhotosContainerResource

use of org.datatransferproject.types.common.models.photos.PhotosContainerResource in project data-transfer-project by google.

the class ImgurPhotosExporter method requestNonAlbumPhotos.

/**
 * Queries all photos for the account. Chooses photos which are not included to the collection of
 * photos from albums.
 *
 * @param authData authentication information
 * @param paginationData pagination information to use for subsequent calls.
 */
private ExportResult<PhotosContainerResource> requestNonAlbumPhotos(TokensAndUrlAuthData authData, PaginationData paginationData, UUID jobId) throws IOException {
    int page = paginationData == null ? 0 : ((IntPaginationToken) paginationData).getStart();
    String url = format(ALL_PHOTOS_URL_TEMPLATE, page);
    Set<PhotoAlbum> albums = new HashSet<>();
    List<PhotoModel> photos = new ArrayList<>();
    List<Map<String, Object>> items = requestData(authData, url);
    boolean hasMore = (items != null && items.size() != 0);
    for (Map<String, Object> item : items) {
        String photoId = (String) item.get("id");
        // Select photos which are not included to the collection of retrieved album photos
        if (!albumPhotos.contains(photoId)) {
            PhotoModel photoModel = new PhotoModel((String) item.get("name"), (String) item.get("link"), (String) item.get("description"), (String) item.get("type"), (String) item.get("id"), DEFAULT_ALBUM_ID, true);
            photos.add(photoModel);
            InputStream inputStream = getImageAsStream(photoModel.getFetchableUrl());
            jobStore.create(jobId, photoModel.getFetchableUrl(), inputStream);
        }
    }
    if (!containsNonAlbumPhotos && photos.size() > 0) {
        // Add album for non-album photos
        albums.add(new PhotoAlbum(DEFAULT_ALBUM_ID, "Non-album photos", "Contains non-album photos"));
        // Make sure album will not be added multiply times on subsequent calls
        containsNonAlbumPhotos = true;
    }
    PaginationData newPage = null;
    if (hasMore) {
        newPage = new IntPaginationToken(page + 1);
        monitor.info(() -> format("added non-album photos, size: %s", photos.size()));
    }
    PhotosContainerResource photosContainerResource = new PhotosContainerResource(albums, photos);
    ContinuationData continuationData = new ContinuationData(newPage);
    ExportResult.ResultType resultType = ExportResult.ResultType.CONTINUE;
    if (newPage == null) {
        resultType = ExportResult.ResultType.END;
    }
    return new ExportResult<>(resultType, photosContainerResource, continuationData);
}
Also used : IntPaginationToken(org.datatransferproject.types.common.IntPaginationToken) PaginationData(org.datatransferproject.types.common.PaginationData) InputStream(java.io.InputStream) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) ArrayList(java.util.ArrayList) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) Map(java.util.Map) HashSet(java.util.HashSet) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult)

Example 40 with PhotosContainerResource

use of org.datatransferproject.types.common.models.photos.PhotosContainerResource in project data-transfer-project by google.

the class ImgurPhotosExporter method requestPhotos.

/**
 * Exports photos from album.
 *
 * <p>This request doesn't support pages so it retrieves all photos at once for each album.
 *
 * @param authData authentication information
 * @param resource contains album id
 * @param paginationData pagination information to use for subsequent calls.
 */
private ExportResult<PhotosContainerResource> requestPhotos(TokensAndUrlAuthData authData, IdOnlyContainerResource resource, PaginationData paginationData, UUID jobId) throws IOException {
    String albumId = resource.getId();
    // Means all other albums with photos are imported, so non-album photos can be determined
    if (albumId.equals(DEFAULT_ALBUM_ID)) {
        return requestNonAlbumPhotos(authData, paginationData, jobId);
    }
    String url = format(ALBUM_PHOTOS_URL_TEMPLATE, albumId);
    List<PhotoModel> photos = new ArrayList<>();
    List<Map<String, Object>> items = requestData(authData, url);
    for (Map<String, Object> item : items) {
        PhotoModel photoModel = new PhotoModel((String) item.get("name"), (String) item.get("link"), (String) item.get("description"), (String) item.get("type"), (String) item.get("id"), albumId, true);
        photos.add(photoModel);
        InputStream inputStream = getImageAsStream(photoModel.getFetchableUrl());
        jobStore.create(jobId, photoModel.getFetchableUrl(), inputStream);
        // Save id of each album photo for finding non-album photos later
        albumPhotos.add((String) item.get("id"));
    }
    // This request doesn't support pages
    ExportResult.ResultType resultType = ExportResult.ResultType.END;
    if (photos.size() > 0) {
        monitor.info(() -> format("added albumPhotos, size: %s", photos.size()));
    }
    PhotosContainerResource photosContainerResource = new PhotosContainerResource(null, photos);
    return new ExportResult<>(resultType, photosContainerResource, new ContinuationData(null));
}
Also used : InputStream(java.io.InputStream) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) ArrayList(java.util.ArrayList) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) Map(java.util.Map) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult)

Aggregations

PhotosContainerResource (org.datatransferproject.types.common.models.photos.PhotosContainerResource)57 PhotoAlbum (org.datatransferproject.types.common.models.photos.PhotoAlbum)37 Test (org.junit.Test)37 IdOnlyContainerResource (org.datatransferproject.types.common.models.IdOnlyContainerResource)29 PhotoModel (org.datatransferproject.types.common.models.photos.PhotoModel)29 ContinuationData (org.datatransferproject.spi.transfer.types.ContinuationData)27 ExportResult (org.datatransferproject.spi.transfer.provider.ExportResult)21 ArrayList (java.util.ArrayList)18 StringPaginationToken (org.datatransferproject.types.common.StringPaginationToken)18 UUID (java.util.UUID)14 PaginationData (org.datatransferproject.types.common.PaginationData)12 IOException (java.io.IOException)10 ExportInformation (org.datatransferproject.types.common.ExportInformation)9 ImportResult (org.datatransferproject.spi.transfer.provider.ImportResult)8 IntPaginationToken (org.datatransferproject.types.common.IntPaginationToken)8 InputStreamWrapper (org.datatransferproject.spi.cloud.storage.TemporaryPerJobDataStore.InputStreamWrapper)6 ContainerResource (org.datatransferproject.types.common.models.ContainerResource)6 TokensAndUrlAuthData (org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData)6 Photoset (com.flickr4java.flickr.photosets.Photoset)5 ImmutableList (com.google.common.collect.ImmutableList)5