Search in sources :

Example 6 with ContinuationInformation

use of org.dataportabilityproject.dataModels.ContinuationInformation in project data-transfer-project by google.

the class GoogleTaskService method getTasks.

private TaskModelWrapper getTasks(String taskListId, Optional<PaginationInformation> pageInfo) throws IOException {
    com.google.api.services.tasks.model.Tasks result;
    Tasks.TasksOperations.List query = taskClient.tasks().list(taskListId).setMaxResults(PAGE_SIZE);
    if (pageInfo.isPresent()) {
        query.setPageToken(((StringPaginationToken) pageInfo.get()).getId());
    }
    result = query.execute();
    List<TaskModel> newTasks = result.getItems().stream().map(t -> new TaskModel(taskListId, t.getTitle(), t.getNotes())).collect(Collectors.toList());
    PaginationInformation newPageInfo = null;
    if (result.getNextPageToken() != null) {
        newPageInfo = new StringPaginationToken(result.getNextPageToken());
    }
    return new TaskModelWrapper(null, newTasks, new ContinuationInformation(null, newPageInfo));
}
Also used : Importer(org.dataportabilityproject.dataModels.Importer) Resource(org.dataportabilityproject.dataModels.Resource) ExportInformation(org.dataportabilityproject.dataModels.ExportInformation) IOException(java.io.IOException) IdOnlyResource(org.dataportabilityproject.shared.IdOnlyResource) Tasks(com.google.api.services.tasks.Tasks) TaskModel(org.dataportabilityproject.dataModels.tasks.TaskModel) Collectors(java.util.stream.Collectors) TaskModelWrapper(org.dataportabilityproject.dataModels.tasks.TaskModelWrapper) Task(com.google.api.services.tasks.model.Task) ArrayList(java.util.ArrayList) Exporter(org.dataportabilityproject.dataModels.Exporter) List(java.util.List) TaskList(com.google.api.services.tasks.model.TaskList) PaginationInformation(org.dataportabilityproject.dataModels.PaginationInformation) JobDataCache(org.dataportabilityproject.cloud.interfaces.JobDataCache) ContinuationInformation(org.dataportabilityproject.dataModels.ContinuationInformation) Optional(java.util.Optional) Credential(com.google.api.client.auth.oauth2.Credential) TaskLists(com.google.api.services.tasks.model.TaskLists) GoogleStaticObjects(org.dataportabilityproject.serviceProviders.google.GoogleStaticObjects) TaskListModel(org.dataportabilityproject.dataModels.tasks.TaskListModel) StringPaginationToken(org.dataportabilityproject.shared.StringPaginationToken) TaskModelWrapper(org.dataportabilityproject.dataModels.tasks.TaskModelWrapper) ContinuationInformation(org.dataportabilityproject.dataModels.ContinuationInformation) TaskModel(org.dataportabilityproject.dataModels.tasks.TaskModel) PaginationInformation(org.dataportabilityproject.dataModels.PaginationInformation) StringPaginationToken(org.dataportabilityproject.shared.StringPaginationToken)

Example 7 with ContinuationInformation

use of org.dataportabilityproject.dataModels.ContinuationInformation in project data-transfer-project by google.

the class SmugMugPhotoService method getAlbums.

private PhotosModelWrapper getAlbums(Optional<PaginationInformation> paginationInformation) throws IOException {
    String albumUri;
    if (paginationInformation.isPresent()) {
        albumUri = ((StringPaginationToken) paginationInformation.get()).getId();
    } else {
        SmugMugResponse<SmugMugUserResponse> userResponse = makeUserRequest(USER_URL);
        albumUri = userResponse.getResponse().getUser().getUris().get("UserAlbums").getUri();
    }
    List<PhotoAlbum> albums = new ArrayList<>();
    List<Resource> resources = new ArrayList<>();
    SmugMugResponse<SmugmugAlbumsResponse> albumResponse = makeAlbumRequest(albumUri);
    for (SmugMugAlbum album : albumResponse.getResponse().getAlbums()) {
        albums.add(new PhotoAlbum(album.getAlbumKey(), album.getTitle(), album.getDescription()));
        resources.add(new IdOnlyResource(album.getAlbumKey()));
    }
    StringPaginationToken pageToken = null;
    if (albumResponse.getResponse().getPageInfo() != null && albumResponse.getResponse().getPageInfo().getNextPage() != null) {
        pageToken = new StringPaginationToken(albumResponse.getResponse().getPageInfo().getNextPage());
    }
    return new PhotosModelWrapper(albums, null, new ContinuationInformation(resources, pageToken));
}
Also used : SmugMugUserResponse(org.dataportabilityproject.serviceProviders.smugmug.model.SmugMugUserResponse) ArrayList(java.util.ArrayList) Resource(org.dataportabilityproject.dataModels.Resource) IdOnlyResource(org.dataportabilityproject.shared.IdOnlyResource) PhotosModelWrapper(org.dataportabilityproject.dataModels.photos.PhotosModelWrapper) IdOnlyResource(org.dataportabilityproject.shared.IdOnlyResource) SmugmugAlbumsResponse(org.dataportabilityproject.serviceProviders.smugmug.model.SmugmugAlbumsResponse) ContinuationInformation(org.dataportabilityproject.dataModels.ContinuationInformation) SmugMugAlbum(org.dataportabilityproject.serviceProviders.smugmug.model.SmugMugAlbum) PhotoAlbum(org.dataportabilityproject.dataModels.photos.PhotoAlbum) StringPaginationToken(org.dataportabilityproject.shared.StringPaginationToken)

Example 8 with ContinuationInformation

use of org.dataportabilityproject.dataModels.ContinuationInformation in project data-transfer-project by google.

the class MicrosoftCalendarService method getCalendars.

private CalendarModelWrapper getCalendars(Optional<PaginationInformation> pageInfo) throws IOException {
    URL url = new URL("https://outlook.office.com/api/v2.0/me/calendars");
    HttpRequest getRequest = requestFactory.buildGetRequest(new GenericUrl(url));
    getRequest.setParser(new JsonObjectParser(new JacksonFactory()));
    HttpResponse response;
    try {
        response = getRequest.execute();
    } catch (HttpResponseException e) {
        logger.debug("Error fetching content");
        logger.debug("response status code: {}", e.getStatusCode());
        logger.debug("response status message: {}", e.getStatusMessage());
        logger.debug("response headers: {}", e.getHeaders());
        logger.debug("response content: {}", e.getContent());
        e.printStackTrace();
        throw e;
    }
    int statusCode = response.getStatusCode();
    if (statusCode != 200) {
        throw new IOException("Bad status code: " + statusCode + " error: " + response.getStatusMessage());
    }
    // Parse response into model
    OutlookCalendarList data = response.parseAs(OutlookCalendarList.class);
    List<CalendarModel> calendars = new ArrayList<>(data.list.size());
    List<Resource> resources = new ArrayList<>(data.list.size());
    for (OutlookCalendar calendar : data.list) {
        calendars.add(new CalendarModel(calendar.id, calendar.name, null));
        resources.add(new IdOnlyResource(calendar.id));
    }
    return new CalendarModelWrapper(calendars, null, new ContinuationInformation(resources, null));
}
Also used : HttpRequest(com.google.api.client.http.HttpRequest) ArrayList(java.util.ArrayList) Resource(org.dataportabilityproject.dataModels.Resource) IdOnlyResource(org.dataportabilityproject.shared.IdOnlyResource) HttpResponse(com.google.api.client.http.HttpResponse) CalendarModel(org.dataportabilityproject.dataModels.calendar.CalendarModel) HttpResponseException(com.google.api.client.http.HttpResponseException) GenericUrl(com.google.api.client.http.GenericUrl) IOException(java.io.IOException) IdOnlyResource(org.dataportabilityproject.shared.IdOnlyResource) JacksonFactory(com.google.api.client.json.jackson2.JacksonFactory) URL(java.net.URL) CalendarModelWrapper(org.dataportabilityproject.dataModels.calendar.CalendarModelWrapper) ContinuationInformation(org.dataportabilityproject.dataModels.ContinuationInformation) JsonObjectParser(com.google.api.client.json.JsonObjectParser)

Example 9 with ContinuationInformation

use of org.dataportabilityproject.dataModels.ContinuationInformation in project data-transfer-project by google.

the class GoogleContactsService method export.

public ContactsModelWrapper export(ExportInformation continuationInformation) throws IOException {
    // Set up connection
    Connections.List connectionsList = peopleService.people().connections().list(GoogleContactsConstants.SELF_RESOURCE);
    // Get next page, if we have a page token
    if (continuationInformation.getPaginationInformation().isPresent()) {
        String pageToken = ((StringPaginationToken) continuationInformation.getPaginationInformation().get()).getId();
        connectionsList.setPageToken(pageToken);
    }
    // Get list of connections (nb: not a list containing full info of each Person)
    ListConnectionsResponse response = connectionsList.setPersonFields(GoogleContactsConstants.PERSON_FIELDS).execute();
    List<Person> peopleList = response.getConnections();
    // Get list of resource names, then get list of Persons
    List<String> resourceNames = peopleList.stream().map(Person::getResourceName).collect(Collectors.toList());
    GetPeopleResponse batchResponse = peopleService.people().getBatchGet().setResourceNames(resourceNames).setPersonFields(GoogleContactsConstants.PERSON_FIELDS).execute();
    List<PersonResponse> personResponseList = batchResponse.getResponses();
    // Convert Persons to VCards
    List<VCard> vCards = personResponseList.stream().map(a -> GoogleContactToVCardConverter.convert(a.getPerson())).collect(Collectors.toList());
    // Determine if there's a next page
    StringPaginationToken newPage = null;
    if (response.getNextPageToken() != null) {
        newPage = new StringPaginationToken(response.getNextPageToken());
    }
    ContinuationInformation newContinuationInformation = null;
    if (newPage != null) {
        newContinuationInformation = new ContinuationInformation(null, newPage);
    }
    return new ContactsModelWrapper(vCards, newContinuationInformation);
}
Also used : Connections(com.google.api.services.people.v1.PeopleService.People.Connections) VCard(ezvcard.VCard) PeopleService(com.google.api.services.people.v1.PeopleService) Importer(org.dataportabilityproject.dataModels.Importer) Logger(org.slf4j.Logger) Connections(com.google.api.services.people.v1.PeopleService.People.Connections) ExportInformation(org.dataportabilityproject.dataModels.ExportInformation) Collection(java.util.Collection) LoggerFactory(org.slf4j.LoggerFactory) IOException(java.io.IOException) Collectors(java.util.stream.Collectors) Exporter(org.dataportabilityproject.dataModels.Exporter) ContactsModelWrapper(org.dataportabilityproject.dataModels.contacts.ContactsModelWrapper) Person(com.google.api.services.people.v1.model.Person) List(java.util.List) PersonResponse(com.google.api.services.people.v1.model.PersonResponse) JobDataCache(org.dataportabilityproject.cloud.interfaces.JobDataCache) ContinuationInformation(org.dataportabilityproject.dataModels.ContinuationInformation) Credential(com.google.api.client.auth.oauth2.Credential) VisibleForTesting(com.google.common.annotations.VisibleForTesting) GoogleStaticObjects(org.dataportabilityproject.serviceProviders.google.GoogleStaticObjects) GetPeopleResponse(com.google.api.services.people.v1.model.GetPeopleResponse) ListConnectionsResponse(com.google.api.services.people.v1.model.ListConnectionsResponse) StringPaginationToken(org.dataportabilityproject.shared.StringPaginationToken) ListConnectionsResponse(com.google.api.services.people.v1.model.ListConnectionsResponse) ContactsModelWrapper(org.dataportabilityproject.dataModels.contacts.ContactsModelWrapper) GetPeopleResponse(com.google.api.services.people.v1.model.GetPeopleResponse) ContinuationInformation(org.dataportabilityproject.dataModels.ContinuationInformation) Person(com.google.api.services.people.v1.model.Person) VCard(ezvcard.VCard) StringPaginationToken(org.dataportabilityproject.shared.StringPaginationToken) PersonResponse(com.google.api.services.people.v1.model.PersonResponse)

Example 10 with ContinuationInformation

use of org.dataportabilityproject.dataModels.ContinuationInformation in project data-transfer-project by google.

the class GooglePhotosService method exportAlbums.

private PhotosModelWrapper exportAlbums(Optional<PaginationInformation> pageInfo) throws IOException {
    URL albumUrl = new URL("https://picasaweb.google.com/data/feed/api/user/default?kind=album");
    UserFeed albumFeed;
    try {
        albumFeed = service.getFeed(albumUrl, UserFeed.class);
    } catch (ServiceException e) {
        throw new IOException("Problem making request to: " + albumUrl, e);
    }
    List<PhotoAlbum> albums = new ArrayList<>(albumFeed.getEntries().size());
    List<Resource> resources = new ArrayList<>(albumFeed.getEntries().size());
    for (GphotoEntry myAlbum : albumFeed.getEntries()) {
        // Adding sub-resources tells the framework to re-call
        // export to get all the photos.
        resources.add(new IdOnlyResource(myAlbum.getGphotoId()));
        // Saving data to the album allows the target service
        // to recreate the album structure.
        albums.add(new PhotoAlbum(myAlbum.getGphotoId(), myAlbum.getTitle().getPlainText(), myAlbum.getDescription().getPlainText()));
    }
    return new PhotosModelWrapper(albums, null, new ContinuationInformation(resources, null));
}
Also used : ArrayList(java.util.ArrayList) Resource(org.dataportabilityproject.dataModels.Resource) IdOnlyResource(org.dataportabilityproject.shared.IdOnlyResource) PhotosModelWrapper(org.dataportabilityproject.dataModels.photos.PhotosModelWrapper) IOException(java.io.IOException) IdOnlyResource(org.dataportabilityproject.shared.IdOnlyResource) URL(java.net.URL) ContinuationInformation(org.dataportabilityproject.dataModels.ContinuationInformation) ServiceException(com.google.gdata.util.ServiceException) UserFeed(com.google.gdata.data.photos.UserFeed) PhotoAlbum(org.dataportabilityproject.dataModels.photos.PhotoAlbum) GphotoEntry(com.google.gdata.data.photos.GphotoEntry)

Aggregations

ContinuationInformation (org.dataportabilityproject.dataModels.ContinuationInformation)18 ArrayList (java.util.ArrayList)13 IdOnlyResource (org.dataportabilityproject.shared.IdOnlyResource)11 Resource (org.dataportabilityproject.dataModels.Resource)10 IOException (java.io.IOException)8 PhotosModelWrapper (org.dataportabilityproject.dataModels.photos.PhotosModelWrapper)8 StringPaginationToken (org.dataportabilityproject.shared.StringPaginationToken)8 PaginationInformation (org.dataportabilityproject.dataModels.PaginationInformation)7 ExportInformation (org.dataportabilityproject.dataModels.ExportInformation)5 Photoset (com.flickr4java.flickr.photosets.Photoset)4 PhotoAlbum (org.dataportabilityproject.dataModels.photos.PhotoAlbum)4 Photosets (com.flickr4java.flickr.photosets.Photosets)3 List (java.util.List)3 Collectors (java.util.stream.Collectors)3 FlickrException (com.flickr4java.flickr.FlickrException)2 UploadMetaData (com.flickr4java.flickr.uploader.UploadMetaData)2 Credential (com.google.api.client.auth.oauth2.Credential)2 VisibleForTesting (com.google.common.annotations.VisibleForTesting)2 ImmutableList (com.google.common.collect.ImmutableList)2 BufferedInputStream (java.io.BufferedInputStream)2