Search in sources :

Example 6 with ResultType

use of org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType in project data-transfer-project by google.

the class GooglePhotosExporter method exportAlbums.

private ExportResult<PhotosContainerResource> exportAlbums(TokensAndUrlAuthData authData, Optional<PaginationData> paginationData) {
    try {
        int startItem = 1;
        if (paginationData.isPresent()) {
            String token = ((StringPaginationToken) paginationData.get()).getToken();
            Preconditions.checkArgument(token.startsWith(ALBUM_TOKEN_PREFIX), "Invalid pagination token " + token);
            startItem = Integer.parseInt(token.substring(ALBUM_TOKEN_PREFIX.length()));
        }
        URL albumUrl = new URL(String.format(URL_ALBUM_FEED_FORMAT, startItem, MAX_RESULTS));
        UserFeed albumFeed = getOrCreatePhotosService(authData).getFeed(albumUrl, UserFeed.class);
        PaginationData nextPageData = null;
        if (albumFeed.getAlbumEntries().size() == MAX_RESULTS) {
            int nextPageStart = startItem + MAX_RESULTS;
            nextPageData = new StringPaginationToken(ALBUM_TOKEN_PREFIX + nextPageStart);
        }
        ContinuationData continuationData = new ContinuationData(nextPageData);
        List<PhotoAlbum> albums = new ArrayList<>(albumFeed.getAlbumEntries().size());
        for (GphotoEntry googleAlbum : albumFeed.getAlbumEntries()) {
            // Add album info to list so album can be recreated later
            albums.add(new PhotoAlbum(googleAlbum.getGphotoId(), googleAlbum.getTitle().getPlainText(), googleAlbum.getDescription().getPlainText()));
            // Add album id to continuation data
            continuationData.addContainerResource(new IdOnlyContainerResource(googleAlbum.getGphotoId()));
        }
        ResultType resultType = ResultType.CONTINUE;
        if (nextPageData == null || continuationData.getContainerResources().isEmpty()) {
            resultType = ResultType.END;
        }
        PhotosContainerResource containerResource = new PhotosContainerResource(albums, null);
        return new ExportResult<>(resultType, containerResource, continuationData);
    } catch (ServiceException | IOException e) {
        return new ExportResult<>(ResultType.ERROR, e.getMessage());
    }
}
Also used : PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) ArrayList(java.util.ArrayList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) IOException(java.io.IOException) URL(java.net.URL) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) ServiceException(com.google.gdata.util.ServiceException) UserFeed(com.google.gdata.data.photos.UserFeed) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) PhotoAlbum(org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum) GphotoEntry(com.google.gdata.data.photos.GphotoEntry) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 7 with ResultType

use of org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType in project data-transfer-project by google.

the class GoogleTasksExporter method getTasksList.

private ExportResult getTasksList(Tasks tasksSerivce, PaginationData paginationData) throws IOException {
    Tasks.Tasklists.List query = tasksSerivce.tasklists().list().setMaxResults(PAGE_SIZE);
    if (paginationData != null) {
        query.setPageToken(((StringPaginationToken) paginationData).getToken());
    }
    TaskLists result = query.execute();
    ImmutableList.Builder<TaskListModel> newTaskListsBuilder = ImmutableList.builder();
    ImmutableList.Builder<IdOnlyContainerResource> newResourcesBuilder = ImmutableList.builder();
    for (TaskList taskList : result.getItems()) {
        newTaskListsBuilder.add(new TaskListModel(taskList.getId(), taskList.getTitle()));
        newResourcesBuilder.add(new IdOnlyContainerResource(taskList.getId()));
    }
    PaginationData newPage = null;
    ResultType resultType = ResultType.END;
    if (result.getNextPageToken() != null) {
        newPage = new StringPaginationToken(result.getNextPageToken());
        resultType = ResultType.CONTINUE;
    }
    List<IdOnlyContainerResource> newResources = newResourcesBuilder.build();
    if (!newResources.isEmpty()) {
        resultType = ResultType.CONTINUE;
    }
    TaskContainerResource taskContainerResource = new TaskContainerResource(newTaskListsBuilder.build(), null);
    ContinuationData continuationData = new ContinuationData(newPage);
    newResourcesBuilder.build().forEach(resource -> continuationData.addContainerResource(resource));
    return new ExportResult<>(resultType, taskContainerResource, continuationData);
}
Also used : PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) ImmutableList(com.google.common.collect.ImmutableList) TaskList(com.google.api.services.tasks.model.TaskList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) TaskContainerResource(org.dataportabilityproject.types.transfer.models.tasks.TaskContainerResource) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) TaskLists(com.google.api.services.tasks.model.TaskLists) TaskListModel(org.dataportabilityproject.types.transfer.models.tasks.TaskListModel) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 8 with ResultType

use of org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType in project data-transfer-project by google.

the class FlickrPhotosExporter method getAlbums.

private ExportResult<PhotosContainerResource> getAlbums(PaginationData paginationData, Auth auth) {
    ImmutableList.Builder<PhotoAlbum> albumBuilder = ImmutableList.builder();
    List<IdOnlyContainerResource> subResources = new ArrayList<>();
    int page = paginationData == null ? 1 : ((IntPaginationToken) paginationData).getStart();
    Photosets photoSetList;
    try {
        photoSetList = photosetsInterface.getList(auth.getUser().getId(), PHOTO_SETS_PER_PAGE, page, PHOTOSET_EXTRAS);
    } catch (FlickrException e) {
        return new ExportResult<>(ResultType.ERROR, "Error exporting Flickr album: " + e.getErrorMessage());
    }
    for (Photoset photoSet : photoSetList.getPhotosets()) {
        // Saving data to the album allows the target service to recreate the album structure
        albumBuilder.add(new PhotoAlbum(photoSet.getId(), photoSet.getTitle(), photoSet.getDescription()));
        // Adding subresources tells the framework to recall export to get all the photos
        subResources.add(new IdOnlyContainerResource(photoSet.getId()));
    }
    PaginationData newPage = null;
    boolean hasMore = photoSetList.getPage() != photoSetList.getPages() && !photoSetList.getPhotosets().isEmpty();
    if (hasMore)
        newPage = new IntPaginationToken(page + 1);
    PhotosContainerResource photosContainerResource = new PhotosContainerResource(albumBuilder.build(), null);
    ContinuationData continuationData = new ContinuationData(newPage);
    subResources.forEach(resource -> continuationData.addContainerResource(resource));
    // Get result type
    ResultType resultType = ResultType.CONTINUE;
    if (newPage == null) {
        resultType = ResultType.END;
    }
    return new ExportResult<>(resultType, photosContainerResource, continuationData);
}
Also used : IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) FlickrException(com.flickr4java.flickr.FlickrException) ImmutableList(com.google.common.collect.ImmutableList) ArrayList(java.util.ArrayList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) Photoset(com.flickr4java.flickr.photosets.Photoset) Photosets(com.flickr4java.flickr.photosets.Photosets) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) PhotoAlbum(org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 9 with ResultType

use of org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType in project data-transfer-project by google.

the class FlickrPhotosExporter method getPhotos.

private ExportResult<PhotosContainerResource> getPhotos(IdOnlyContainerResource resource, PaginationData paginationData) {
    String photoSetId = resource.getId();
    int page = paginationData == null ? 1 : ((IntPaginationToken) paginationData).getStart();
    PhotoList<Photo> photoSetList;
    try {
        if (photoSetId == null) {
            RequestContext.getRequestContext().setExtras(EXTRAS);
            photoSetList = photosInterface.getNotInSet(PHOTO_PER_PAGE, page);
            RequestContext.getRequestContext().setExtras(ImmutableList.of());
        } else {
            photoSetList = photosetsInterface.getPhotos(photoSetId, ImmutableSet.copyOf(EXTRAS), 0, PHOTO_PER_PAGE, page);
        }
    } catch (FlickrException e) {
        return new ExportResult<>(ResultType.ERROR, "Error exporting Flickr photo: " + e.getErrorMessage());
    }
    boolean hasMore = photoSetList.getPage() != photoSetList.getPages() && !photoSetList.isEmpty();
    Collection<PhotoModel> photos = photoSetList.stream().map(p -> toCommonPhoto(p, photoSetId)).collect(Collectors.toList());
    PaginationData newPage = null;
    if (hasMore) {
        newPage = new IntPaginationToken(page + 1);
    }
    // Get result type
    ResultType resultType = ResultType.CONTINUE;
    if (newPage == null) {
        resultType = ResultType.END;
    }
    PhotosContainerResource photosContainerResource = new PhotosContainerResource(null, photos);
    return new ExportResult<>(resultType, photosContainerResource, new ContinuationData(newPage));
}
Also used : IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) PhotoAlbum(org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum) FlickrException(com.flickr4java.flickr.FlickrException) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult) LoggerFactory(org.slf4j.LoggerFactory) IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) Photoset(com.flickr4java.flickr.photosets.Photoset) REST(com.flickr4java.flickr.REST) ArrayList(java.util.ArrayList) Photo(com.flickr4java.flickr.photos.Photo) Strings(com.google.common.base.Strings) ImmutableList(com.google.common.collect.ImmutableList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) AuthData(org.dataportabilityproject.types.transfer.auth.AuthData) Flickr(com.flickr4java.flickr.Flickr) PhotoList(com.flickr4java.flickr.photos.PhotoList) Auth(com.flickr4java.flickr.auth.Auth) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) ExportInformation(org.dataportabilityproject.spi.transfer.types.ExportInformation) ImmutableSet(com.google.common.collect.ImmutableSet) Exporter(org.dataportabilityproject.spi.transfer.provider.Exporter) Logger(org.slf4j.Logger) PhotosInterface(com.flickr4java.flickr.photos.PhotosInterface) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) Collection(java.util.Collection) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) Photosets(com.flickr4java.flickr.photosets.Photosets) List(java.util.List) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) Preconditions(com.google.common.base.Preconditions) AppCredentials(org.dataportabilityproject.types.transfer.auth.AppCredentials) VisibleForTesting(com.google.common.annotations.VisibleForTesting) RequestContext(com.flickr4java.flickr.RequestContext) PhotosetsInterface(com.flickr4java.flickr.photosets.PhotosetsInterface) IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) FlickrException(com.flickr4java.flickr.FlickrException) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) Photo(com.flickr4java.flickr.photos.Photo) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 10 with ResultType

use of org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType in project data-transfer-project by google.

the class GoogleCalendarExporter method getCalendarEvents.

private ExportResult<CalendarContainerResource> getCalendarEvents(AuthData authData, String id, Optional<PaginationData> pageData) {
    Calendar.Events.List listRequest;
    Events listResult;
    // Get event information
    try {
        listRequest = getOrCreateCalendarInterface(authData).events().list(id).setMaxAttendees(GoogleStaticObjects.MAX_ATTENDEES);
        if (pageData.isPresent()) {
            StringPaginationToken paginationToken = (StringPaginationToken) pageData.get();
            Preconditions.checkState(paginationToken.getToken().startsWith(EVENT_TOKEN_PREFIX), "Token is not applicable");
            listRequest.setPageToken(((StringPaginationToken) pageData.get()).getToken().substring(EVENT_TOKEN_PREFIX.length()));
        }
        listResult = listRequest.execute();
    } catch (IOException e) {
        return new ExportResult<CalendarContainerResource>(ResultType.ERROR, e.getMessage());
    }
    // Set up continuation data
    PaginationData nextPageData = null;
    if (listResult.getNextPageToken() != null) {
        nextPageData = new StringPaginationToken(EVENT_TOKEN_PREFIX + listResult.getNextPageToken());
    }
    ContinuationData continuationData = new ContinuationData(nextPageData);
    // Process event list
    List<CalendarEventModel> eventModels = new ArrayList<>(listResult.getItems().size());
    for (Event eventData : listResult.getItems()) {
        CalendarEventModel model = convertToCalendarEventModel(id, eventData);
        eventModels.add(model);
    }
    CalendarContainerResource calendarContainerResource = new CalendarContainerResource(null, eventModels);
    // Get result type
    ExportResult.ResultType resultType = ResultType.CONTINUE;
    if (nextPageData == null) {
        resultType = ResultType.END;
    }
    return new ExportResult<CalendarContainerResource>(resultType, calendarContainerResource, continuationData);
}
Also used : PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) ArrayList(java.util.ArrayList) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) IOException(java.io.IOException) CalendarEventModel(org.dataportabilityproject.types.transfer.models.calendar.CalendarEventModel) CalendarContainerResource(org.dataportabilityproject.types.transfer.models.calendar.CalendarContainerResource) Events(com.google.api.services.calendar.model.Events) Event(com.google.api.services.calendar.model.Event) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Aggregations

ExportResult (org.dataportabilityproject.spi.transfer.provider.ExportResult)11 ResultType (org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType)11 ContinuationData (org.dataportabilityproject.spi.transfer.types.ContinuationData)11 PaginationData (org.dataportabilityproject.spi.transfer.types.PaginationData)9 StringPaginationToken (org.dataportabilityproject.spi.transfer.types.StringPaginationToken)9 ArrayList (java.util.ArrayList)8 IdOnlyContainerResource (org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource)7 IOException (java.io.IOException)6 PhotosContainerResource (org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource)6 ImmutableList (com.google.common.collect.ImmutableList)4 PhotoAlbum (org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum)4 VisibleForTesting (com.google.common.annotations.VisibleForTesting)3 List (java.util.List)3 UUID (java.util.UUID)3 Collectors (java.util.stream.Collectors)3 Exporter (org.dataportabilityproject.spi.transfer.provider.Exporter)3 FlickrException (com.flickr4java.flickr.FlickrException)2 Photoset (com.flickr4java.flickr.photosets.Photoset)2 Photosets (com.flickr4java.flickr.photosets.Photosets)2 BearerToken (com.google.api.client.auth.oauth2.BearerToken)2