Search in sources :

Example 11 with ExportResult

use of org.dataportabilityproject.spi.transfer.provider.ExportResult in project data-transfer-project by google.

the class FlickrPhotosExporter method getAlbums.

private ExportResult<PhotosContainerResource> getAlbums(PaginationData paginationData, Auth auth) {
    ImmutableList.Builder<PhotoAlbum> albumBuilder = ImmutableList.builder();
    List<IdOnlyContainerResource> subResources = new ArrayList<>();
    int page = paginationData == null ? 1 : ((IntPaginationToken) paginationData).getStart();
    Photosets photoSetList;
    try {
        photoSetList = photosetsInterface.getList(auth.getUser().getId(), PHOTO_SETS_PER_PAGE, page, PHOTOSET_EXTRAS);
    } catch (FlickrException e) {
        return new ExportResult<>(ResultType.ERROR, "Error exporting Flickr album: " + e.getErrorMessage());
    }
    for (Photoset photoSet : photoSetList.getPhotosets()) {
        // Saving data to the album allows the target service to recreate the album structure
        albumBuilder.add(new PhotoAlbum(photoSet.getId(), photoSet.getTitle(), photoSet.getDescription()));
        // Adding subresources tells the framework to recall export to get all the photos
        subResources.add(new IdOnlyContainerResource(photoSet.getId()));
    }
    PaginationData newPage = null;
    boolean hasMore = photoSetList.getPage() != photoSetList.getPages() && !photoSetList.getPhotosets().isEmpty();
    if (hasMore)
        newPage = new IntPaginationToken(page + 1);
    PhotosContainerResource photosContainerResource = new PhotosContainerResource(albumBuilder.build(), null);
    ContinuationData continuationData = new ContinuationData(newPage);
    subResources.forEach(resource -> continuationData.addContainerResource(resource));
    // Get result type
    ResultType resultType = ResultType.CONTINUE;
    if (newPage == null) {
        resultType = ResultType.END;
    }
    return new ExportResult<>(resultType, photosContainerResource, continuationData);
}
Also used : IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) FlickrException(com.flickr4java.flickr.FlickrException) ImmutableList(com.google.common.collect.ImmutableList) ArrayList(java.util.ArrayList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) Photoset(com.flickr4java.flickr.photosets.Photoset) Photosets(com.flickr4java.flickr.photosets.Photosets) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) PhotoAlbum(org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 12 with ExportResult

use of org.dataportabilityproject.spi.transfer.provider.ExportResult in project data-transfer-project by google.

the class FlickrPhotosExporter method getPhotos.

private ExportResult<PhotosContainerResource> getPhotos(IdOnlyContainerResource resource, PaginationData paginationData) {
    String photoSetId = resource.getId();
    int page = paginationData == null ? 1 : ((IntPaginationToken) paginationData).getStart();
    PhotoList<Photo> photoSetList;
    try {
        if (photoSetId == null) {
            RequestContext.getRequestContext().setExtras(EXTRAS);
            photoSetList = photosInterface.getNotInSet(PHOTO_PER_PAGE, page);
            RequestContext.getRequestContext().setExtras(ImmutableList.of());
        } else {
            photoSetList = photosetsInterface.getPhotos(photoSetId, ImmutableSet.copyOf(EXTRAS), 0, PHOTO_PER_PAGE, page);
        }
    } catch (FlickrException e) {
        return new ExportResult<>(ResultType.ERROR, "Error exporting Flickr photo: " + e.getErrorMessage());
    }
    boolean hasMore = photoSetList.getPage() != photoSetList.getPages() && !photoSetList.isEmpty();
    Collection<PhotoModel> photos = photoSetList.stream().map(p -> toCommonPhoto(p, photoSetId)).collect(Collectors.toList());
    PaginationData newPage = null;
    if (hasMore) {
        newPage = new IntPaginationToken(page + 1);
    }
    // Get result type
    ResultType resultType = ResultType.CONTINUE;
    if (newPage == null) {
        resultType = ResultType.END;
    }
    PhotosContainerResource photosContainerResource = new PhotosContainerResource(null, photos);
    return new ExportResult<>(resultType, photosContainerResource, new ContinuationData(newPage));
}
Also used : IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) PhotoAlbum(org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum) FlickrException(com.flickr4java.flickr.FlickrException) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult) LoggerFactory(org.slf4j.LoggerFactory) IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) Photoset(com.flickr4java.flickr.photosets.Photoset) REST(com.flickr4java.flickr.REST) ArrayList(java.util.ArrayList) Photo(com.flickr4java.flickr.photos.Photo) Strings(com.google.common.base.Strings) ImmutableList(com.google.common.collect.ImmutableList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) AuthData(org.dataportabilityproject.types.transfer.auth.AuthData) Flickr(com.flickr4java.flickr.Flickr) PhotoList(com.flickr4java.flickr.photos.PhotoList) Auth(com.flickr4java.flickr.auth.Auth) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) ExportInformation(org.dataportabilityproject.spi.transfer.types.ExportInformation) ImmutableSet(com.google.common.collect.ImmutableSet) Exporter(org.dataportabilityproject.spi.transfer.provider.Exporter) Logger(org.slf4j.Logger) PhotosInterface(com.flickr4java.flickr.photos.PhotosInterface) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) Collection(java.util.Collection) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) Photosets(com.flickr4java.flickr.photosets.Photosets) List(java.util.List) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) Preconditions(com.google.common.base.Preconditions) AppCredentials(org.dataportabilityproject.types.transfer.auth.AppCredentials) VisibleForTesting(com.google.common.annotations.VisibleForTesting) RequestContext(com.flickr4java.flickr.RequestContext) PhotosetsInterface(com.flickr4java.flickr.photosets.PhotosetsInterface) IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) FlickrException(com.flickr4java.flickr.FlickrException) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) Photo(com.flickr4java.flickr.photos.Photo) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 13 with ExportResult

use of org.dataportabilityproject.spi.transfer.provider.ExportResult in project data-transfer-project by google.

the class GoogleCalendarExporter method getCalendarEvents.

private ExportResult<CalendarContainerResource> getCalendarEvents(AuthData authData, String id, Optional<PaginationData> pageData) {
    Calendar.Events.List listRequest;
    Events listResult;
    // Get event information
    try {
        listRequest = getOrCreateCalendarInterface(authData).events().list(id).setMaxAttendees(GoogleStaticObjects.MAX_ATTENDEES);
        if (pageData.isPresent()) {
            StringPaginationToken paginationToken = (StringPaginationToken) pageData.get();
            Preconditions.checkState(paginationToken.getToken().startsWith(EVENT_TOKEN_PREFIX), "Token is not applicable");
            listRequest.setPageToken(((StringPaginationToken) pageData.get()).getToken().substring(EVENT_TOKEN_PREFIX.length()));
        }
        listResult = listRequest.execute();
    } catch (IOException e) {
        return new ExportResult<CalendarContainerResource>(ResultType.ERROR, e.getMessage());
    }
    // Set up continuation data
    PaginationData nextPageData = null;
    if (listResult.getNextPageToken() != null) {
        nextPageData = new StringPaginationToken(EVENT_TOKEN_PREFIX + listResult.getNextPageToken());
    }
    ContinuationData continuationData = new ContinuationData(nextPageData);
    // Process event list
    List<CalendarEventModel> eventModels = new ArrayList<>(listResult.getItems().size());
    for (Event eventData : listResult.getItems()) {
        CalendarEventModel model = convertToCalendarEventModel(id, eventData);
        eventModels.add(model);
    }
    CalendarContainerResource calendarContainerResource = new CalendarContainerResource(null, eventModels);
    // Get result type
    ExportResult.ResultType resultType = ResultType.CONTINUE;
    if (nextPageData == null) {
        resultType = ResultType.END;
    }
    return new ExportResult<CalendarContainerResource>(resultType, calendarContainerResource, continuationData);
}
Also used : PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) ArrayList(java.util.ArrayList) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) IOException(java.io.IOException) CalendarEventModel(org.dataportabilityproject.types.transfer.models.calendar.CalendarEventModel) CalendarContainerResource(org.dataportabilityproject.types.transfer.models.calendar.CalendarContainerResource) Events(com.google.api.services.calendar.model.Events) Event(com.google.api.services.calendar.model.Event) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 14 with ExportResult

use of org.dataportabilityproject.spi.transfer.provider.ExportResult in project data-transfer-project by google.

the class MicrosoftCalendarExportTest method testExport.

@Test
public void testExport() throws Exception {
    server.enqueue(new MockResponse().setBody(CALENDARS_RESPONSE));
    server.enqueue(new MockResponse().setBody(CALENDAR1_EVENTS_RESPONSE));
    server.enqueue(new MockResponse().setBody(CALENDAR2_EVENTS_RESPONSE));
    server.start();
    HttpUrl baseUrl = server.url("");
    MicrosoftCalendarExporter exporter = new MicrosoftCalendarExporter(baseUrl.toString(), client, mapper, transformerService);
    ExportResult<CalendarContainerResource> resource = exporter.export(UUID.randomUUID(), token);
    CalendarContainerResource calendarResource = resource.getExportedData();
    Assert.assertEquals(2, calendarResource.getCalendars().size());
    Assert.assertFalse(calendarResource.getCalendars().stream().anyMatch(c -> "Calendar1".equals(c.getId()) && "Calendar2".equals(c.getId())));
    Assert.assertEquals(2, calendarResource.getEvents().size());
    Assert.assertFalse(calendarResource.getEvents().stream().anyMatch(e -> "Test Appointment 1".equals(e.getTitle()) && "Test Appointment 2".equals(e.getTitle())));
}
Also used : MicrosoftCalendarExporter(org.dataportabilityproject.transfer.microsoft.calendar.MicrosoftCalendarExporter) MockWebServer(com.squareup.okhttp.mockwebserver.MockWebServer) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) MockResponse(com.squareup.okhttp.mockwebserver.MockResponse) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult) TokenAuthData(org.dataportabilityproject.types.transfer.auth.TokenAuthData) Test(org.junit.Test) UUID(java.util.UUID) MicrosoftCalendarExporter(org.dataportabilityproject.transfer.microsoft.calendar.MicrosoftCalendarExporter) HttpUrl(com.squareup.okhttp.HttpUrl) CalendarContainerResource(org.dataportabilityproject.types.transfer.models.calendar.CalendarContainerResource) TransformerServiceImpl(org.dataportabilityproject.transfer.microsoft.transformer.TransformerServiceImpl) OkHttpClient(okhttp3.OkHttpClient) After(org.junit.After) Assert(org.junit.Assert) Before(org.junit.Before) MockResponse(com.squareup.okhttp.mockwebserver.MockResponse) CalendarContainerResource(org.dataportabilityproject.types.transfer.models.calendar.CalendarContainerResource) HttpUrl(com.squareup.okhttp.HttpUrl) Test(org.junit.Test)

Example 15 with ExportResult

use of org.dataportabilityproject.spi.transfer.provider.ExportResult in project data-transfer-project by google.

the class SmugMugPhotosExporter method exportAlbums.

private ExportResult<PhotosContainerResource> exportAlbums(Optional<PaginationData> paginationData) throws IOException {
    // Make request to SmugMug
    String albumInfoUri;
    if (paginationData.isPresent()) {
        String token = ((StringPaginationToken) paginationData.get()).getToken();
        Preconditions.checkState(token.startsWith(ALBUM_TOKEN_PREFIX), "Invalid pagination token " + token);
        albumInfoUri = token.substring(ALBUM_TOKEN_PREFIX.length());
    } else {
        SmugMugResponse<SmugMugUserResponse> userResponse = smugMugInterface.makeUserRequest(USER_URL);
        albumInfoUri = userResponse.getResponse().getUser().getUris().get(ALBUMS_KEY).getUri();
    }
    SmugMugResponse<SmugMugAlbumsResponse> albumsResponse = smugMugInterface.makeAlbumRequest(albumInfoUri);
    // Set up continuation data
    StringPaginationToken paginationToken = null;
    if (albumsResponse.getResponse().getPageInfo() != null && albumsResponse.getResponse().getPageInfo().getNextPage() != null) {
        paginationToken = new StringPaginationToken(ALBUM_TOKEN_PREFIX + albumsResponse.getResponse().getPageInfo().getNextPage());
    }
    ContinuationData continuationData = new ContinuationData(paginationToken);
    // Build album list
    List<PhotoAlbum> albumsList = new ArrayList<>();
    for (SmugMugAlbum album : albumsResponse.getResponse().getAlbums()) {
        albumsList.add(new PhotoAlbum(album.getAlbumKey(), album.getTitle(), album.getDescription()));
        continuationData.addContainerResource(new IdOnlyContainerResource(album.getAlbumKey()));
    }
    PhotosContainerResource resource = new PhotosContainerResource(albumsList, null);
    // Get result type
    ResultType resultType = ResultType.CONTINUE;
    if (paginationToken == null) {
        resultType = ResultType.END;
    }
    return new ExportResult<>(resultType, resource, continuationData);
}
Also used : SmugMugUserResponse(org.dataportabilityproject.transfer.smugmug.photos.model.SmugMugUserResponse) ArrayList(java.util.ArrayList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) SmugMugAlbum(org.dataportabilityproject.transfer.smugmug.photos.model.SmugMugAlbum) SmugMugAlbumsResponse(org.dataportabilityproject.transfer.smugmug.photos.model.SmugMugAlbumsResponse) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) PhotoAlbum(org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Aggregations

ExportResult (org.dataportabilityproject.spi.transfer.provider.ExportResult)15 ContinuationData (org.dataportabilityproject.spi.transfer.types.ContinuationData)13 ResultType (org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType)11 PaginationData (org.dataportabilityproject.spi.transfer.types.PaginationData)10 StringPaginationToken (org.dataportabilityproject.spi.transfer.types.StringPaginationToken)10 IOException (java.io.IOException)9 ArrayList (java.util.ArrayList)9 IdOnlyContainerResource (org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource)8 PhotosContainerResource (org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource)6 List (java.util.List)5 UUID (java.util.UUID)5 CalendarContainerResource (org.dataportabilityproject.types.transfer.models.calendar.CalendarContainerResource)5 PhotoAlbum (org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum)4 VisibleForTesting (com.google.common.annotations.VisibleForTesting)3 ImmutableList (com.google.common.collect.ImmutableList)3 Collectors (java.util.stream.Collectors)3 ExportInformation (org.dataportabilityproject.spi.transfer.types.ExportInformation)3 CalendarModel (org.dataportabilityproject.types.transfer.models.calendar.CalendarModel)3 FlickrException (com.flickr4java.flickr.FlickrException)2 Photoset (com.flickr4java.flickr.photosets.Photoset)2