Search in sources :

Example 1 with ResultType

use of org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType in project data-transfer-project by google.

the class GoogleCalendarExporter method exportCalendars.

private ExportResult<CalendarContainerResource> exportCalendars(AuthData authData, Optional<PaginationData> pageData) {
    Calendar.CalendarList.List listRequest;
    CalendarList listResult;
    // Get calendar information
    try {
        listRequest = getOrCreateCalendarInterface(authData).calendarList().list();
        if (pageData.isPresent()) {
            StringPaginationToken paginationToken = (StringPaginationToken) pageData.get();
            Preconditions.checkState(paginationToken.getToken().startsWith(CALENDAR_TOKEN_PREFIX), "Token is not applicable");
            listRequest.setPageToken(((StringPaginationToken) pageData.get()).getToken().substring(CALENDAR_TOKEN_PREFIX.length()));
        }
        listResult = listRequest.execute();
    } catch (IOException e) {
        return new ExportResult<CalendarContainerResource>(ResultType.ERROR, e.getMessage());
    }
    // Set up continuation data
    PaginationData nextPageData = null;
    if (listResult.getNextPageToken() != null) {
        nextPageData = new StringPaginationToken(CALENDAR_TOKEN_PREFIX + listResult.getNextPageToken());
    }
    ContinuationData continuationData = new ContinuationData(nextPageData);
    // Process calendar list
    List<CalendarModel> calendarModels = new ArrayList<>(listResult.getItems().size());
    for (CalendarListEntry calendarData : listResult.getItems()) {
        CalendarModel model = convertToCalendarModel(calendarData);
        continuationData.addContainerResource(new IdOnlyContainerResource(calendarData.getId()));
        calendarModels.add(model);
    }
    CalendarContainerResource calendarContainerResource = new CalendarContainerResource(calendarModels, null);
    // Get result type
    ExportResult.ResultType resultType = ResultType.CONTINUE;
    if (calendarModels.isEmpty()) {
        resultType = ResultType.END;
    }
    return new ExportResult<CalendarContainerResource>(resultType, calendarContainerResource, continuationData);
}
Also used : PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) ArrayList(java.util.ArrayList) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) CalendarModel(org.dataportabilityproject.types.transfer.models.calendar.CalendarModel) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) IOException(java.io.IOException) CalendarContainerResource(org.dataportabilityproject.types.transfer.models.calendar.CalendarContainerResource) CalendarListEntry(com.google.api.services.calendar.model.CalendarListEntry) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) CalendarList(com.google.api.services.calendar.model.CalendarList) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 2 with ResultType

use of org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType in project data-transfer-project by google.

the class GoogleContactsExporter method exportContacts.

private ExportResult<ContactsModelWrapper> exportContacts(TokensAndUrlAuthData authData, Optional<PaginationData> pageData) {
    try {
        // Set up connection
        Connections.List connectionsListRequest = getOrCreatePeopleService(authData).people().connections().list(SELF_RESOURCE);
        // Get next page, if we have a page token
        if (pageData.isPresent()) {
            StringPaginationToken paginationToken = (StringPaginationToken) pageData.get();
            connectionsListRequest.setPageToken(paginationToken.getToken());
        }
        // Get list of connections (nb: not a list containing full info of each Person)
        ListConnectionsResponse response = connectionsListRequest.setPersonFields(PERSON_FIELDS).execute();
        List<Person> peopleList = response.getConnections();
        // Get list of resource names, then get list of Persons
        List<String> resourceNames = peopleList.stream().map(Person::getResourceName).collect(Collectors.toList());
        GetPeopleResponse batchResponse = getOrCreatePeopleService(authData).people().getBatchGet().setResourceNames(resourceNames).setPersonFields(PERSON_FIELDS).execute();
        List<PersonResponse> personResponseList = batchResponse.getResponses();
        // Convert Persons to VCards
        List<VCard> vCards = personResponseList.stream().map(a -> convert(a.getPerson())).collect(Collectors.toList());
        // Determine if there's a next page
        StringPaginationToken nextPageData = null;
        if (response.getNextPageToken() != null) {
            nextPageData = new StringPaginationToken(response.getNextPageToken());
        }
        ContinuationData continuationData = new ContinuationData(nextPageData);
        ContactsModelWrapper wrapper = new ContactsModelWrapper(makeVCardString(vCards));
        // Get result type
        ResultType resultType = ResultType.CONTINUE;
        if (nextPageData == null) {
            resultType = ResultType.END;
        }
        return new ExportResult<ContactsModelWrapper>(resultType, wrapper, continuationData);
    } catch (IOException e) {
        return new ExportResult<ContactsModelWrapper>(ResultType.ERROR, e.getMessage());
    }
}
Also used : Connections(com.google.api.services.people.v1.PeopleService.People.Connections) EmailAddress(com.google.api.services.people.v1.model.EmailAddress) Connections(com.google.api.services.people.v1.PeopleService.People.Connections) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult) LoggerFactory(org.slf4j.LoggerFactory) PhoneNumber(com.google.api.services.people.v1.model.PhoneNumber) StructuredName(ezvcard.property.StructuredName) TokensAndUrlAuthData(org.dataportabilityproject.types.transfer.auth.TokensAndUrlAuthData) Person(com.google.api.services.people.v1.model.Person) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) Credential(com.google.api.client.auth.oauth2.Credential) VCARD_PRIMARY_PREF(org.dataportabilityproject.datatransfer.google.common.GoogleStaticObjects.VCARD_PRIMARY_PREF) LinkedList(java.util.LinkedList) GetPeopleResponse(com.google.api.services.people.v1.model.GetPeopleResponse) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) SELF_RESOURCE(org.dataportabilityproject.datatransfer.google.common.GoogleStaticObjects.SELF_RESOURCE) ListConnectionsResponse(com.google.api.services.people.v1.model.ListConnectionsResponse) VCard(ezvcard.VCard) PeopleService(com.google.api.services.people.v1.PeopleService) ExportInformation(org.dataportabilityproject.spi.transfer.types.ExportInformation) Exporter(org.dataportabilityproject.spi.transfer.provider.Exporter) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) Logger(org.slf4j.Logger) BearerToken(com.google.api.client.auth.oauth2.BearerToken) StringWriter(java.io.StringWriter) Telephone(ezvcard.property.Telephone) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) IOException(java.io.IOException) UUID(java.util.UUID) JCardWriter(ezvcard.io.json.JCardWriter) Collectors(java.util.stream.Collectors) SOURCE_PARAM_NAME_TYPE(org.dataportabilityproject.datatransfer.google.common.GoogleStaticObjects.SOURCE_PARAM_NAME_TYPE) List(java.util.List) PERSON_FIELDS(org.dataportabilityproject.datatransfer.google.common.GoogleStaticObjects.PERSON_FIELDS) FieldMetadata(com.google.api.services.people.v1.model.FieldMetadata) PersonResponse(com.google.api.services.people.v1.model.PersonResponse) Email(ezvcard.property.Email) ContactsModelWrapper(org.dataportabilityproject.types.transfer.models.contacts.ContactsModelWrapper) Optional(java.util.Optional) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) Name(com.google.api.services.people.v1.model.Name) GoogleStaticObjects(org.dataportabilityproject.datatransfer.google.common.GoogleStaticObjects) ListConnectionsResponse(com.google.api.services.people.v1.model.ListConnectionsResponse) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ContactsModelWrapper(org.dataportabilityproject.types.transfer.models.contacts.ContactsModelWrapper) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) IOException(java.io.IOException) GetPeopleResponse(com.google.api.services.people.v1.model.GetPeopleResponse) Person(com.google.api.services.people.v1.model.Person) VCard(ezvcard.VCard) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) PersonResponse(com.google.api.services.people.v1.model.PersonResponse) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 3 with ResultType

use of org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType in project data-transfer-project by google.

the class GooglePhotosExporter method exportPhotos.

private ExportResult<PhotosContainerResource> exportPhotos(TokensAndUrlAuthData authData, String albumId, Optional<PaginationData> paginationData) {
    try {
        int startItem = 1;
        if (paginationData.isPresent()) {
            String token = ((StringPaginationToken) paginationData.get()).getToken();
            Preconditions.checkArgument(token.startsWith(PHOTO_TOKEN_PREFIX), "Invalid pagination token " + token);
            startItem = Integer.parseInt(token.substring(PHOTO_TOKEN_PREFIX.length()));
        }
        URL photosUrl = new URL(String.format(URL_PHOTO_FEED_FORMAT, albumId, startItem, MAX_RESULTS));
        AlbumFeed photoFeed = getOrCreatePhotosService(authData).getFeed(photosUrl, AlbumFeed.class);
        PaginationData nextPageData = null;
        if (photoFeed.getEntries().size() == MAX_RESULTS) {
            int nextPageStart = startItem + MAX_RESULTS;
            nextPageData = new StringPaginationToken(PHOTO_TOKEN_PREFIX + nextPageStart);
        }
        ContinuationData continuationData = new ContinuationData(nextPageData);
        List<PhotoModel> photos = new ArrayList<>(photoFeed.getEntries().size());
        for (GphotoEntry photo : photoFeed.getEntries()) {
            MediaContent mediaContent = (MediaContent) photo.getContent();
            photos.add(new PhotoModel(photo.getTitle().getPlainText(), mediaContent.getUri(), photo.getDescription().getPlainText(), mediaContent.getMimeType().getMediaType(), albumId));
        }
        PhotosContainerResource containerResource = new PhotosContainerResource(null, photos);
        ResultType resultType = ResultType.CONTINUE;
        if (nextPageData == null) {
            resultType = ResultType.END;
        }
        return new ExportResult<>(resultType, containerResource, continuationData);
    } catch (ServiceException | IOException e) {
        return new ExportResult<>(ResultType.ERROR, e.getMessage());
    }
}
Also used : PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) AlbumFeed(com.google.gdata.data.photos.AlbumFeed) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) ArrayList(java.util.ArrayList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) IOException(java.io.IOException) URL(java.net.URL) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) ServiceException(com.google.gdata.util.ServiceException) MediaContent(com.google.gdata.data.MediaContent) GphotoEntry(com.google.gdata.data.photos.GphotoEntry) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 4 with ResultType

use of org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType in project data-transfer-project by google.

the class GoogleTasksExporter method getTasks.

private ExportResult getTasks(Tasks tasksService, IdOnlyContainerResource resource, PaginationData paginationData) throws IOException {
    Tasks.TasksOperations.List query = tasksService.tasks().list(resource.getId()).setMaxResults(PAGE_SIZE);
    if (paginationData != null) {
        query.setPageToken(((StringPaginationToken) paginationData).getToken());
    }
    com.google.api.services.tasks.model.Tasks result = query.execute();
    List<TaskModel> newTasks = result.getItems().stream().map(t -> new TaskModel(resource.getId(), t.getTitle(), t.getNotes())).collect(Collectors.toList());
    PaginationData newPage = null;
    ResultType resultType = ResultType.END;
    if (result.getNextPageToken() != null) {
        newPage = new StringPaginationToken(result.getNextPageToken());
        resultType = ResultType.CONTINUE;
    }
    TaskContainerResource taskContainerResource = new TaskContainerResource(null, newTasks);
    return new ExportResult<>(resultType, taskContainerResource, new ContinuationData(newPage));
}
Also used : ExportInformation(org.dataportabilityproject.spi.transfer.types.ExportInformation) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) Exporter(org.dataportabilityproject.spi.transfer.provider.Exporter) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) TaskModel(org.dataportabilityproject.types.transfer.models.tasks.TaskModel) BearerToken(com.google.api.client.auth.oauth2.BearerToken) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) TaskContainerResource(org.dataportabilityproject.types.transfer.models.tasks.TaskContainerResource) IOException(java.io.IOException) Tasks(com.google.api.services.tasks.Tasks) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) TokensAndUrlAuthData(org.dataportabilityproject.types.transfer.auth.TokensAndUrlAuthData) TaskListModel(org.dataportabilityproject.types.transfer.models.tasks.TaskListModel) List(java.util.List) TaskList(com.google.api.services.tasks.model.TaskList) ImmutableList(com.google.common.collect.ImmutableList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) Credential(com.google.api.client.auth.oauth2.Credential) VisibleForTesting(com.google.common.annotations.VisibleForTesting) TaskLists(com.google.api.services.tasks.model.TaskLists) GoogleStaticObjects(org.dataportabilityproject.datatransfer.google.common.GoogleStaticObjects) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) TaskContainerResource(org.dataportabilityproject.types.transfer.models.tasks.TaskContainerResource) TaskModel(org.dataportabilityproject.types.transfer.models.tasks.TaskModel) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 5 with ResultType

use of org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType in project data-transfer-project by google.

the class SmugMugPhotosExporter method exportPhotos.

private ExportResult<PhotosContainerResource> exportPhotos(IdOnlyContainerResource containerResource, Optional<PaginationData> paginationData) throws IOException {
    List<PhotoModel> photoList = new ArrayList<>();
    // Make request to SmugMug
    String photoInfoUri;
    if (paginationData.isPresent()) {
        String token = ((StringPaginationToken) paginationData.get()).getToken();
        Preconditions.checkState(token.startsWith(PHOTO_TOKEN_PREFIX), "Invalid pagination token " + token);
        photoInfoUri = token.substring(PHOTO_TOKEN_PREFIX.length());
    } else {
        String id = containerResource.getId();
        photoInfoUri = String.format(ALBUM_URL_FORMATTER, id);
    }
    SmugMugResponse<SmugMugAlbumInfoResponse> albumInfoResponse = smugMugInterface.makeAlbumInfoRequest(photoInfoUri);
    // Set up continuation data
    StringPaginationToken pageToken = null;
    if (albumInfoResponse.getResponse().getPageInfo().getNextPage() != null) {
        pageToken = new StringPaginationToken(PHOTO_TOKEN_PREFIX + albumInfoResponse.getResponse().getPageInfo().getNextPage());
    }
    ContinuationData continuationData = new ContinuationData(pageToken);
    // Make list of photos
    for (SmugMugAlbumImage image : albumInfoResponse.getResponse().getImages()) {
        String title = image.getTitle();
        if (Strings.isNullOrEmpty(title)) {
            title = image.getFileName();
        }
        // TODO(olsona): this.authConsumer.sign(image.getArchivedUri()) ?
        photoList.add(new PhotoModel(title, image.getArchivedUri(), image.getCaption(), image.getFormat(), containerResource.getId()));
    }
    PhotosContainerResource resource = new PhotosContainerResource(null, photoList);
    // Get result type
    ResultType resultType = ResultType.CONTINUE;
    if (pageToken == null) {
        resultType = ResultType.END;
    }
    return new ExportResult<>(resultType, resource, continuationData);
}
Also used : PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) SmugMugAlbumImage(org.dataportabilityproject.transfer.smugmug.photos.model.SmugMugAlbumImage) ArrayList(java.util.ArrayList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) SmugMugAlbumInfoResponse(org.dataportabilityproject.transfer.smugmug.photos.model.SmugMugAlbumInfoResponse) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Aggregations

ExportResult (org.dataportabilityproject.spi.transfer.provider.ExportResult)11 ResultType (org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType)11 ContinuationData (org.dataportabilityproject.spi.transfer.types.ContinuationData)11 PaginationData (org.dataportabilityproject.spi.transfer.types.PaginationData)9 StringPaginationToken (org.dataportabilityproject.spi.transfer.types.StringPaginationToken)9 ArrayList (java.util.ArrayList)8 IdOnlyContainerResource (org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource)7 IOException (java.io.IOException)6 PhotosContainerResource (org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource)6 ImmutableList (com.google.common.collect.ImmutableList)4 PhotoAlbum (org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum)4 VisibleForTesting (com.google.common.annotations.VisibleForTesting)3 List (java.util.List)3 UUID (java.util.UUID)3 Collectors (java.util.stream.Collectors)3 Exporter (org.dataportabilityproject.spi.transfer.provider.Exporter)3 FlickrException (com.flickr4java.flickr.FlickrException)2 Photoset (com.flickr4java.flickr.photosets.Photoset)2 Photosets (com.flickr4java.flickr.photosets.Photosets)2 BearerToken (com.google.api.client.auth.oauth2.BearerToken)2