Search in sources :

Example 1 with IntPaginationToken

use of org.dataportabilityproject.spi.transfer.types.IntPaginationToken in project data-transfer-project by google.

the class FlickrPhotosExporterTest method exportAlbumInitial.

@Test
public void exportAlbumInitial() throws FlickrException {
    // set up auth, flickr service
    when(user.getId()).thenReturn("userId");
    when(authInterface.checkToken(any(Token.class))).thenReturn(auth);
    when(flickr.getPhotosetsInterface()).thenReturn(photosetsInterface);
    when(flickr.getPhotosInterface()).thenReturn(photosInterface);
    when(flickr.getAuthInterface()).thenReturn(authInterface);
    // setup photoset
    Photoset photoset = FlickrTestUtils.initializePhotoset("photosetId", "title", "description");
    // setup photoset list (aka album view)
    int page = 1;
    Photosets photosetsList = new Photosets();
    photosetsList.setPage(page);
    photosetsList.setPages(page + 1);
    photosetsList.setPhotosets(Collections.singletonList(photoset));
    when(photosetsInterface.getList(anyString(), anyInt(), anyInt(), anyString())).thenReturn(photosetsList);
    // run test
    FlickrPhotosExporter exporter = new FlickrPhotosExporter(flickr);
    AuthData authData = new TokenSecretAuthData("token", "secret");
    ExportResult<PhotosContainerResource> result = exporter.export(UUID.randomUUID(), authData);
    // make sure album and photo information is correct
    assertThat(result.getExportedData().getPhotos()).isEmpty();
    Collection<PhotoAlbum> albums = result.getExportedData().getAlbums();
    assertThat(albums.size()).isEqualTo(1);
    assertThat(albums).containsExactly(new PhotoAlbum("photosetId", "title", "description"));
    // check continuation information
    ContinuationData continuationData = (ContinuationData) result.getContinuationData();
    assertThat(continuationData.getPaginationData()).isInstanceOf(IntPaginationToken.class);
    assertThat(((IntPaginationToken) continuationData.getPaginationData()).getStart()).isEqualTo(page + 1);
    Collection<? extends ContainerResource> subResources = continuationData.getContainerResources();
    assertThat(subResources.size()).isEqualTo(1);
    assertThat(subResources).containsExactly(new IdOnlyContainerResource("photosetId"));
}
Also used : IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) AuthData(org.dataportabilityproject.types.transfer.auth.AuthData) TokenSecretAuthData(org.dataportabilityproject.types.transfer.auth.TokenSecretAuthData) IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) Token(org.scribe.model.Token) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) TokenSecretAuthData(org.dataportabilityproject.types.transfer.auth.TokenSecretAuthData) Photoset(com.flickr4java.flickr.photosets.Photoset) Photosets(com.flickr4java.flickr.photosets.Photosets) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) PhotoAlbum(org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum) Test(org.junit.Test)

Example 2 with IntPaginationToken

use of org.dataportabilityproject.spi.transfer.types.IntPaginationToken in project data-transfer-project by google.

the class FlickrPhotosExporter method getAlbums.

private ExportResult<PhotosContainerResource> getAlbums(PaginationData paginationData, Auth auth) {
    ImmutableList.Builder<PhotoAlbum> albumBuilder = ImmutableList.builder();
    List<IdOnlyContainerResource> subResources = new ArrayList<>();
    int page = paginationData == null ? 1 : ((IntPaginationToken) paginationData).getStart();
    Photosets photoSetList;
    try {
        photoSetList = photosetsInterface.getList(auth.getUser().getId(), PHOTO_SETS_PER_PAGE, page, PHOTOSET_EXTRAS);
    } catch (FlickrException e) {
        return new ExportResult<>(ResultType.ERROR, "Error exporting Flickr album: " + e.getErrorMessage());
    }
    for (Photoset photoSet : photoSetList.getPhotosets()) {
        // Saving data to the album allows the target service to recreate the album structure
        albumBuilder.add(new PhotoAlbum(photoSet.getId(), photoSet.getTitle(), photoSet.getDescription()));
        // Adding subresources tells the framework to recall export to get all the photos
        subResources.add(new IdOnlyContainerResource(photoSet.getId()));
    }
    PaginationData newPage = null;
    boolean hasMore = photoSetList.getPage() != photoSetList.getPages() && !photoSetList.getPhotosets().isEmpty();
    if (hasMore)
        newPage = new IntPaginationToken(page + 1);
    PhotosContainerResource photosContainerResource = new PhotosContainerResource(albumBuilder.build(), null);
    ContinuationData continuationData = new ContinuationData(newPage);
    subResources.forEach(resource -> continuationData.addContainerResource(resource));
    // Get result type
    ResultType resultType = ResultType.CONTINUE;
    if (newPage == null) {
        resultType = ResultType.END;
    }
    return new ExportResult<>(resultType, photosContainerResource, continuationData);
}
Also used : IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) FlickrException(com.flickr4java.flickr.FlickrException) ImmutableList(com.google.common.collect.ImmutableList) ArrayList(java.util.ArrayList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) Photoset(com.flickr4java.flickr.photosets.Photoset) Photosets(com.flickr4java.flickr.photosets.Photosets) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) PhotoAlbum(org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 3 with IntPaginationToken

use of org.dataportabilityproject.spi.transfer.types.IntPaginationToken in project data-transfer-project by google.

the class FlickrPhotosExporter method getPhotos.

private ExportResult<PhotosContainerResource> getPhotos(IdOnlyContainerResource resource, PaginationData paginationData) {
    String photoSetId = resource.getId();
    int page = paginationData == null ? 1 : ((IntPaginationToken) paginationData).getStart();
    PhotoList<Photo> photoSetList;
    try {
        if (photoSetId == null) {
            RequestContext.getRequestContext().setExtras(EXTRAS);
            photoSetList = photosInterface.getNotInSet(PHOTO_PER_PAGE, page);
            RequestContext.getRequestContext().setExtras(ImmutableList.of());
        } else {
            photoSetList = photosetsInterface.getPhotos(photoSetId, ImmutableSet.copyOf(EXTRAS), 0, PHOTO_PER_PAGE, page);
        }
    } catch (FlickrException e) {
        return new ExportResult<>(ResultType.ERROR, "Error exporting Flickr photo: " + e.getErrorMessage());
    }
    boolean hasMore = photoSetList.getPage() != photoSetList.getPages() && !photoSetList.isEmpty();
    Collection<PhotoModel> photos = photoSetList.stream().map(p -> toCommonPhoto(p, photoSetId)).collect(Collectors.toList());
    PaginationData newPage = null;
    if (hasMore) {
        newPage = new IntPaginationToken(page + 1);
    }
    // Get result type
    ResultType resultType = ResultType.CONTINUE;
    if (newPage == null) {
        resultType = ResultType.END;
    }
    PhotosContainerResource photosContainerResource = new PhotosContainerResource(null, photos);
    return new ExportResult<>(resultType, photosContainerResource, new ContinuationData(newPage));
}
Also used : IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) PhotoAlbum(org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum) FlickrException(com.flickr4java.flickr.FlickrException) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult) LoggerFactory(org.slf4j.LoggerFactory) IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) Photoset(com.flickr4java.flickr.photosets.Photoset) REST(com.flickr4java.flickr.REST) ArrayList(java.util.ArrayList) Photo(com.flickr4java.flickr.photos.Photo) Strings(com.google.common.base.Strings) ImmutableList(com.google.common.collect.ImmutableList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) AuthData(org.dataportabilityproject.types.transfer.auth.AuthData) Flickr(com.flickr4java.flickr.Flickr) PhotoList(com.flickr4java.flickr.photos.PhotoList) Auth(com.flickr4java.flickr.auth.Auth) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) ExportInformation(org.dataportabilityproject.spi.transfer.types.ExportInformation) ImmutableSet(com.google.common.collect.ImmutableSet) Exporter(org.dataportabilityproject.spi.transfer.provider.Exporter) Logger(org.slf4j.Logger) PhotosInterface(com.flickr4java.flickr.photos.PhotosInterface) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) Collection(java.util.Collection) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) Photosets(com.flickr4java.flickr.photosets.Photosets) List(java.util.List) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) Preconditions(com.google.common.base.Preconditions) AppCredentials(org.dataportabilityproject.types.transfer.auth.AppCredentials) VisibleForTesting(com.google.common.annotations.VisibleForTesting) RequestContext(com.flickr4java.flickr.RequestContext) PhotosetsInterface(com.flickr4java.flickr.photosets.PhotosetsInterface) IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) FlickrException(com.flickr4java.flickr.FlickrException) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) Photo(com.flickr4java.flickr.photos.Photo) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 4 with IntPaginationToken

use of org.dataportabilityproject.spi.transfer.types.IntPaginationToken in project data-transfer-project by google.

the class FlickrPhotosExporterTest method exportPhotosFromPhotoset.

@Test
public void exportPhotosFromPhotoset() throws FlickrException {
    // set up auth, flickr service
    when(user.getId()).thenReturn("userId");
    when(authInterface.checkToken(any(Token.class))).thenReturn(auth);
    when(flickr.getPhotosetsInterface()).thenReturn(photosetsInterface);
    when(flickr.getPhotosInterface()).thenReturn(photosInterface);
    when(flickr.getAuthInterface()).thenReturn(authInterface);
    // getting photos from a set with id photosetsId and page 1
    int page = 1;
    String photosetsId = "photosetsId";
    ExportInformation exportInformation = new ExportInformation(null, new IdOnlyContainerResource(photosetsId));
    // make lots of photos and add them to PhotoList (also adding pagination information)
    int numPhotos = 4;
    PhotoList<Photo> photosList = new PhotoList<>();
    for (int i = 0; i < numPhotos; i++) {
        photosList.add(FlickrTestUtils.initializePhoto("title" + 1, "url" + i, "description" + i, MEDIA_TYPE));
    }
    photosList.setPage(page);
    photosList.setPages(page + 1);
    when(photosetsInterface.getPhotos(anyString(), anySet(), anyInt(), anyInt(), anyInt())).thenReturn(photosList);
    // run test
    FlickrPhotosExporter exporter = new FlickrPhotosExporter(flickr);
    ExportResult<PhotosContainerResource> result = exporter.export(UUID.randomUUID(), new TokenSecretAuthData("token", "secret"), exportInformation);
    assertThat(result.getExportedData().getPhotos().size()).isEqualTo(numPhotos);
    assertThat(result.getExportedData().getAlbums()).isEmpty();
    ContinuationData continuationData = (ContinuationData) result.getContinuationData();
    assertThat(continuationData.getContainerResources()).isEmpty();
    assertThat(((IntPaginationToken) continuationData.getPaginationData()).getStart()).isEqualTo(page + 1);
}
Also used : IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) Token(org.scribe.model.Token) Photo(com.flickr4java.flickr.photos.Photo) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) Matchers.anyString(org.mockito.Matchers.anyString) ExportInformation(org.dataportabilityproject.spi.transfer.types.ExportInformation) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) PhotoList(com.flickr4java.flickr.photos.PhotoList) TokenSecretAuthData(org.dataportabilityproject.types.transfer.auth.TokenSecretAuthData) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) Test(org.junit.Test)

Aggregations

ContinuationData (org.dataportabilityproject.spi.transfer.types.ContinuationData)4 IdOnlyContainerResource (org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource)4 IntPaginationToken (org.dataportabilityproject.spi.transfer.types.IntPaginationToken)4 PhotosContainerResource (org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource)4 Photoset (com.flickr4java.flickr.photosets.Photoset)3 Photosets (com.flickr4java.flickr.photosets.Photosets)3 PhotoAlbum (org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum)3 FlickrException (com.flickr4java.flickr.FlickrException)2 Photo (com.flickr4java.flickr.photos.Photo)2 PhotoList (com.flickr4java.flickr.photos.PhotoList)2 ImmutableList (com.google.common.collect.ImmutableList)2 ArrayList (java.util.ArrayList)2 ExportResult (org.dataportabilityproject.spi.transfer.provider.ExportResult)2 ResultType (org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType)2 ExportInformation (org.dataportabilityproject.spi.transfer.types.ExportInformation)2 PaginationData (org.dataportabilityproject.spi.transfer.types.PaginationData)2 AuthData (org.dataportabilityproject.types.transfer.auth.AuthData)2 TokenSecretAuthData (org.dataportabilityproject.types.transfer.auth.TokenSecretAuthData)2 Test (org.junit.Test)2 Token (org.scribe.model.Token)2