Search in sources :

Example 6 with PhotosContainerResource

use of org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource in project data-transfer-project by google.

the class FlickrPhotosExporter method getPhotos.

private ExportResult<PhotosContainerResource> getPhotos(IdOnlyContainerResource resource, PaginationData paginationData) {
    String photoSetId = resource.getId();
    int page = paginationData == null ? 1 : ((IntPaginationToken) paginationData).getStart();
    PhotoList<Photo> photoSetList;
    try {
        if (photoSetId == null) {
            RequestContext.getRequestContext().setExtras(EXTRAS);
            photoSetList = photosInterface.getNotInSet(PHOTO_PER_PAGE, page);
            RequestContext.getRequestContext().setExtras(ImmutableList.of());
        } else {
            photoSetList = photosetsInterface.getPhotos(photoSetId, ImmutableSet.copyOf(EXTRAS), 0, PHOTO_PER_PAGE, page);
        }
    } catch (FlickrException e) {
        return new ExportResult<>(ResultType.ERROR, "Error exporting Flickr photo: " + e.getErrorMessage());
    }
    boolean hasMore = photoSetList.getPage() != photoSetList.getPages() && !photoSetList.isEmpty();
    Collection<PhotoModel> photos = photoSetList.stream().map(p -> toCommonPhoto(p, photoSetId)).collect(Collectors.toList());
    PaginationData newPage = null;
    if (hasMore) {
        newPage = new IntPaginationToken(page + 1);
    }
    // Get result type
    ResultType resultType = ResultType.CONTINUE;
    if (newPage == null) {
        resultType = ResultType.END;
    }
    PhotosContainerResource photosContainerResource = new PhotosContainerResource(null, photos);
    return new ExportResult<>(resultType, photosContainerResource, new ContinuationData(newPage));
}
Also used : IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) PhotoAlbum(org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum) FlickrException(com.flickr4java.flickr.FlickrException) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult) LoggerFactory(org.slf4j.LoggerFactory) IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) Photoset(com.flickr4java.flickr.photosets.Photoset) REST(com.flickr4java.flickr.REST) ArrayList(java.util.ArrayList) Photo(com.flickr4java.flickr.photos.Photo) Strings(com.google.common.base.Strings) ImmutableList(com.google.common.collect.ImmutableList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) AuthData(org.dataportabilityproject.types.transfer.auth.AuthData) Flickr(com.flickr4java.flickr.Flickr) PhotoList(com.flickr4java.flickr.photos.PhotoList) Auth(com.flickr4java.flickr.auth.Auth) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) ExportInformation(org.dataportabilityproject.spi.transfer.types.ExportInformation) ImmutableSet(com.google.common.collect.ImmutableSet) Exporter(org.dataportabilityproject.spi.transfer.provider.Exporter) Logger(org.slf4j.Logger) PhotosInterface(com.flickr4java.flickr.photos.PhotosInterface) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) Collection(java.util.Collection) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) Photosets(com.flickr4java.flickr.photosets.Photosets) List(java.util.List) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) Preconditions(com.google.common.base.Preconditions) AppCredentials(org.dataportabilityproject.types.transfer.auth.AppCredentials) VisibleForTesting(com.google.common.annotations.VisibleForTesting) RequestContext(com.flickr4java.flickr.RequestContext) PhotosetsInterface(com.flickr4java.flickr.photosets.PhotosetsInterface) IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) FlickrException(com.flickr4java.flickr.FlickrException) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) Photo(com.flickr4java.flickr.photos.Photo) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 7 with PhotosContainerResource

use of org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource in project data-transfer-project by google.

the class FlickrPhotosExporterTest method exportPhotosFromPhotoset.

@Test
public void exportPhotosFromPhotoset() throws FlickrException {
    // set up auth, flickr service
    when(user.getId()).thenReturn("userId");
    when(authInterface.checkToken(any(Token.class))).thenReturn(auth);
    when(flickr.getPhotosetsInterface()).thenReturn(photosetsInterface);
    when(flickr.getPhotosInterface()).thenReturn(photosInterface);
    when(flickr.getAuthInterface()).thenReturn(authInterface);
    // getting photos from a set with id photosetsId and page 1
    int page = 1;
    String photosetsId = "photosetsId";
    ExportInformation exportInformation = new ExportInformation(null, new IdOnlyContainerResource(photosetsId));
    // make lots of photos and add them to PhotoList (also adding pagination information)
    int numPhotos = 4;
    PhotoList<Photo> photosList = new PhotoList<>();
    for (int i = 0; i < numPhotos; i++) {
        photosList.add(FlickrTestUtils.initializePhoto("title" + 1, "url" + i, "description" + i, MEDIA_TYPE));
    }
    photosList.setPage(page);
    photosList.setPages(page + 1);
    when(photosetsInterface.getPhotos(anyString(), anySet(), anyInt(), anyInt(), anyInt())).thenReturn(photosList);
    // run test
    FlickrPhotosExporter exporter = new FlickrPhotosExporter(flickr);
    ExportResult<PhotosContainerResource> result = exporter.export(UUID.randomUUID(), new TokenSecretAuthData("token", "secret"), exportInformation);
    assertThat(result.getExportedData().getPhotos().size()).isEqualTo(numPhotos);
    assertThat(result.getExportedData().getAlbums()).isEmpty();
    ContinuationData continuationData = (ContinuationData) result.getContinuationData();
    assertThat(continuationData.getContainerResources()).isEmpty();
    assertThat(((IntPaginationToken) continuationData.getPaginationData()).getStart()).isEqualTo(page + 1);
}
Also used : IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) Token(org.scribe.model.Token) Photo(com.flickr4java.flickr.photos.Photo) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) Matchers.anyString(org.mockito.Matchers.anyString) ExportInformation(org.dataportabilityproject.spi.transfer.types.ExportInformation) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) PhotoList(com.flickr4java.flickr.photos.PhotoList) TokenSecretAuthData(org.dataportabilityproject.types.transfer.auth.TokenSecretAuthData) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) Test(org.junit.Test)

Example 8 with PhotosContainerResource

use of org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource in project data-transfer-project by google.

the class FlickrPhotosImporterTest method importStoresAlbumInJobStore.

@Test
public void importStoresAlbumInJobStore() throws FlickrException, IOException {
    UUID jobId = UUID.randomUUID();
    PhotosContainerResource photosContainerResource = new PhotosContainerResource(Collections.singletonList(PHOTO_ALBUM), Collections.singletonList(PHOTO_MODEL));
    // Setup Mock
    when(user.getId()).thenReturn("userId");
    when(authInterface.checkToken(any(Token.class))).thenReturn(auth);
    when(flickr.getPhotosetsInterface()).thenReturn(photosetsInterface);
    when(flickr.getUploader()).thenReturn(uploader);
    when(flickr.getAuthInterface()).thenReturn(authInterface);
    when(imageStreamProvider.get(FETCHABLE_URL)).thenReturn(bufferedInputStream);
    when(uploader.upload(any(BufferedInputStream.class), any(UploadMetaData.class))).thenReturn(FLICKR_PHOTO_ID);
    String flickrAlbumTitle = FlickrPhotosImporter.COPY_PREFIX + ALBUM_NAME;
    Photoset photoset = FlickrTestUtils.initializePhotoset(FLICKR_ALBUM_ID, ALBUM_DESCRIPTION, FLICKR_PHOTO_ID);
    when(photosetsInterface.create(flickrAlbumTitle, ALBUM_DESCRIPTION, FLICKR_PHOTO_ID)).thenReturn(photoset);
    // Run test
    FlickrPhotosImporter importer = new FlickrPhotosImporter(flickr, jobStore, imageStreamProvider);
    ImportResult result = importer.importItem(jobId, new TokenSecretAuthData("token", "secret"), photosContainerResource);
    // Verify that the image stream provider got the correct URL and that the correct info was uploaded
    verify(imageStreamProvider).get(FETCHABLE_URL);
    ArgumentCaptor<UploadMetaData> uploadMetaDataArgumentCaptor = ArgumentCaptor.forClass(UploadMetaData.class);
    verify(uploader).upload(eq(bufferedInputStream), uploadMetaDataArgumentCaptor.capture());
    UploadMetaData actualUploadMetaData = uploadMetaDataArgumentCaptor.getValue();
    assertThat(actualUploadMetaData.getTitle()).isEqualTo(FlickrPhotosImporter.COPY_PREFIX + PHOTO_TITLE);
    assertThat(actualUploadMetaData.getDescription()).isEqualTo(PHOTO_DESCRIPTION);
    // Verify the photosets interface got the command to create the correct album
    verify(photosetsInterface).create(flickrAlbumTitle, ALBUM_DESCRIPTION, FLICKR_PHOTO_ID);
    // Check contents of JobStore
    TempPhotosData tempPhotosData = jobStore.findData(TempPhotosData.class, jobId);
    assertThat(tempPhotosData).isNotNull();
    String expectedAlbumKey = FlickrPhotosImporter.CACHE_ALBUM_METADATA_PREFIX + ALBUM_ID;
    assertThat(tempPhotosData.lookupAlbum(expectedAlbumKey)).isNotNull();
    assertThat(tempPhotosData.lookupAlbum(expectedAlbumKey)).isEqualTo(PHOTO_ALBUM);
    assertThat(tempPhotosData.lookupNewAlbumId(ALBUM_ID)).isEqualTo(FLICKR_ALBUM_ID);
}
Also used : PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) TempPhotosData(org.dataportabilityproject.spi.transfer.types.TempPhotosData) ImportResult(org.dataportabilityproject.spi.transfer.provider.ImportResult) TokenSecretAuthData(org.dataportabilityproject.types.transfer.auth.TokenSecretAuthData) BufferedInputStream(java.io.BufferedInputStream) Photoset(com.flickr4java.flickr.photosets.Photoset) Token(org.scribe.model.Token) UploadMetaData(com.flickr4java.flickr.uploader.UploadMetaData) UUID(java.util.UUID) Test(org.junit.Test)

Example 9 with PhotosContainerResource

use of org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource in project data-transfer-project by google.

the class SmugMugPhotosExporter method exportAlbums.

private ExportResult<PhotosContainerResource> exportAlbums(Optional<PaginationData> paginationData) throws IOException {
    // Make request to SmugMug
    String albumInfoUri;
    if (paginationData.isPresent()) {
        String token = ((StringPaginationToken) paginationData.get()).getToken();
        Preconditions.checkState(token.startsWith(ALBUM_TOKEN_PREFIX), "Invalid pagination token " + token);
        albumInfoUri = token.substring(ALBUM_TOKEN_PREFIX.length());
    } else {
        SmugMugResponse<SmugMugUserResponse> userResponse = smugMugInterface.makeUserRequest(USER_URL);
        albumInfoUri = userResponse.getResponse().getUser().getUris().get(ALBUMS_KEY).getUri();
    }
    SmugMugResponse<SmugMugAlbumsResponse> albumsResponse = smugMugInterface.makeAlbumRequest(albumInfoUri);
    // Set up continuation data
    StringPaginationToken paginationToken = null;
    if (albumsResponse.getResponse().getPageInfo() != null && albumsResponse.getResponse().getPageInfo().getNextPage() != null) {
        paginationToken = new StringPaginationToken(ALBUM_TOKEN_PREFIX + albumsResponse.getResponse().getPageInfo().getNextPage());
    }
    ContinuationData continuationData = new ContinuationData(paginationToken);
    // Build album list
    List<PhotoAlbum> albumsList = new ArrayList<>();
    for (SmugMugAlbum album : albumsResponse.getResponse().getAlbums()) {
        albumsList.add(new PhotoAlbum(album.getAlbumKey(), album.getTitle(), album.getDescription()));
        continuationData.addContainerResource(new IdOnlyContainerResource(album.getAlbumKey()));
    }
    PhotosContainerResource resource = new PhotosContainerResource(albumsList, null);
    // Get result type
    ResultType resultType = ResultType.CONTINUE;
    if (paginationToken == null) {
        resultType = ResultType.END;
    }
    return new ExportResult<>(resultType, resource, continuationData);
}
Also used : SmugMugUserResponse(org.dataportabilityproject.transfer.smugmug.photos.model.SmugMugUserResponse) ArrayList(java.util.ArrayList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) SmugMugAlbum(org.dataportabilityproject.transfer.smugmug.photos.model.SmugMugAlbum) SmugMugAlbumsResponse(org.dataportabilityproject.transfer.smugmug.photos.model.SmugMugAlbumsResponse) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) PhotoAlbum(org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Aggregations

PhotosContainerResource (org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource)9 ContinuationData (org.dataportabilityproject.spi.transfer.types.ContinuationData)8 ArrayList (java.util.ArrayList)6 ExportResult (org.dataportabilityproject.spi.transfer.provider.ExportResult)6 ResultType (org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType)6 IdOnlyContainerResource (org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource)6 PhotoAlbum (org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum)5 Photoset (com.flickr4java.flickr.photosets.Photoset)4 IntPaginationToken (org.dataportabilityproject.spi.transfer.types.IntPaginationToken)4 PaginationData (org.dataportabilityproject.spi.transfer.types.PaginationData)4 StringPaginationToken (org.dataportabilityproject.spi.transfer.types.StringPaginationToken)4 Photosets (com.flickr4java.flickr.photosets.Photosets)3 TokenSecretAuthData (org.dataportabilityproject.types.transfer.auth.TokenSecretAuthData)3 Test (org.junit.Test)3 Token (org.scribe.model.Token)3 FlickrException (com.flickr4java.flickr.FlickrException)2 Photo (com.flickr4java.flickr.photos.Photo)2 PhotoList (com.flickr4java.flickr.photos.PhotoList)2 ImmutableList (com.google.common.collect.ImmutableList)2 GphotoEntry (com.google.gdata.data.photos.GphotoEntry)2