Search in sources :

Example 11 with TokensAndUrlAuthData

use of org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData in project data-transfer-project by google.

the class GoogleContactsExporter method exportContacts.

private ExportResult<ContactsModelWrapper> exportContacts(TokensAndUrlAuthData authData, Optional<PaginationData> pageData) {
    try {
        // Set up connection
        Connections.List connectionsListRequest = getOrCreatePeopleService(authData).people().connections().list(SELF_RESOURCE);
        // Get next page, if we have a page token
        if (pageData.isPresent()) {
            StringPaginationToken paginationToken = (StringPaginationToken) pageData.get();
            connectionsListRequest.setPageToken(paginationToken.getToken());
        }
        // Get list of connections (nb: not a list containing full info of each Person)
        ListConnectionsResponse response = connectionsListRequest.setPersonFields(PERSON_FIELDS).execute();
        List<Person> peopleList = response.getConnections();
        // Get list of resource names, then get list of Persons
        List<String> resourceNames = peopleList.stream().map(Person::getResourceName).collect(Collectors.toList());
        GetPeopleResponse batchResponse = getOrCreatePeopleService(authData).people().getBatchGet().setResourceNames(resourceNames).setPersonFields(PERSON_FIELDS).execute();
        List<PersonResponse> personResponseList = batchResponse.getResponses();
        // Convert Persons to VCards
        List<VCard> vCards = personResponseList.stream().map(a -> convert(a.getPerson())).collect(Collectors.toList());
        // Determine if there's a next page
        StringPaginationToken nextPageData = null;
        if (response.getNextPageToken() != null) {
            nextPageData = new StringPaginationToken(response.getNextPageToken());
        }
        ContinuationData continuationData = new ContinuationData(nextPageData);
        ContactsModelWrapper wrapper = new ContactsModelWrapper(makeVCardString(vCards));
        // Get result type
        ResultType resultType = ResultType.CONTINUE;
        if (nextPageData == null) {
            resultType = ResultType.END;
        }
        return new ExportResult<ContactsModelWrapper>(resultType, wrapper, continuationData);
    } catch (IOException e) {
        return new ExportResult<ContactsModelWrapper>(e);
    }
}
Also used : Connections(com.google.api.services.people.v1.PeopleService.People.Connections) VCARD_PRIMARY_PREF(org.datatransferproject.datatransfer.google.common.GoogleStaticObjects.VCARD_PRIMARY_PREF) EmailAddress(com.google.api.services.people.v1.model.EmailAddress) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult) Connections(com.google.api.services.people.v1.PeopleService.People.Connections) PhoneNumber(com.google.api.services.people.v1.model.PhoneNumber) StructuredName(ezvcard.property.StructuredName) GoogleStaticObjects(org.datatransferproject.datatransfer.google.common.GoogleStaticObjects) ExportInformation(org.datatransferproject.types.common.ExportInformation) Person(com.google.api.services.people.v1.model.Person) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) PERSON_FIELDS(org.datatransferproject.datatransfer.google.common.GoogleStaticObjects.PERSON_FIELDS) Credential(com.google.api.client.auth.oauth2.Credential) LinkedList(java.util.LinkedList) GetPeopleResponse(com.google.api.services.people.v1.model.GetPeopleResponse) GoogleCredentialFactory(org.datatransferproject.datatransfer.google.common.GoogleCredentialFactory) ListConnectionsResponse(com.google.api.services.people.v1.model.ListConnectionsResponse) VCard(ezvcard.VCard) PeopleService(com.google.api.services.people.v1.PeopleService) SOURCE_PARAM_NAME_TYPE(org.datatransferproject.datatransfer.google.common.GoogleStaticObjects.SOURCE_PARAM_NAME_TYPE) ResultType(org.datatransferproject.spi.transfer.provider.ExportResult.ResultType) StringWriter(java.io.StringWriter) Telephone(ezvcard.property.Telephone) Exporter(org.datatransferproject.spi.transfer.provider.Exporter) IOException(java.io.IOException) PaginationData(org.datatransferproject.types.common.PaginationData) UUID(java.util.UUID) JCardWriter(ezvcard.io.json.JCardWriter) Collectors(java.util.stream.Collectors) SELF_RESOURCE(org.datatransferproject.datatransfer.google.common.GoogleStaticObjects.SELF_RESOURCE) List(java.util.List) FieldMetadata(com.google.api.services.people.v1.model.FieldMetadata) TokensAndUrlAuthData(org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData) PersonResponse(com.google.api.services.people.v1.model.PersonResponse) Email(ezvcard.property.Email) ContactsModelWrapper(org.datatransferproject.types.common.models.contacts.ContactsModelWrapper) Optional(java.util.Optional) Preconditions(com.google.common.base.Preconditions) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) VisibleForTesting(com.google.common.annotations.VisibleForTesting) Name(com.google.api.services.people.v1.model.Name) ListConnectionsResponse(com.google.api.services.people.v1.model.ListConnectionsResponse) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) ContactsModelWrapper(org.datatransferproject.types.common.models.contacts.ContactsModelWrapper) ResultType(org.datatransferproject.spi.transfer.provider.ExportResult.ResultType) IOException(java.io.IOException) GetPeopleResponse(com.google.api.services.people.v1.model.GetPeopleResponse) Person(com.google.api.services.people.v1.model.Person) VCard(ezvcard.VCard) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) PersonResponse(com.google.api.services.people.v1.model.PersonResponse) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult)

Example 12 with TokensAndUrlAuthData

use of org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData in project data-transfer-project by google.

the class FacebookPhotosExporterTest method testExportPhoto.

@Test
public void testExportPhoto() throws CopyExceptionWithFailureReason {
    ExportResult<PhotosContainerResource> result = facebookPhotosExporter.export(uuid, new TokensAndUrlAuthData("accessToken", null, null), Optional.of(new ExportInformation(null, new IdOnlyContainerResource(ALBUM_ID))));
    assertEquals(ExportResult.ResultType.END, result.getType());
    PhotosContainerResource exportedData = result.getExportedData();
    assertEquals(1, exportedData.getPhotos().size());
    assertEquals(new PhotoModel(PHOTO_ID + ".jpg", PHOTO_ID, PHOTO_NAME, "image/jpg", PHOTO_ID, ALBUM_ID, false, PHOTO_TIME), exportedData.getPhotos().toArray()[0]);
}
Also used : PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) ExportInformation(org.datatransferproject.types.common.ExportInformation) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) TokensAndUrlAuthData(org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) Test(org.junit.Test)

Example 13 with TokensAndUrlAuthData

use of org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData in project data-transfer-project by google.

the class FacebookVideosExporterTest method testExportVideo.

@Test
public void testExportVideo() throws CopyExceptionWithFailureReason {
    ExportResult<VideosContainerResource> result = facebookVideosExporter.export(uuid, new TokensAndUrlAuthData("accessToken", null, null), Optional.of(new ExportInformation(null, null)));
    assertEquals(ExportResult.ResultType.END, result.getType());
    VideosContainerResource exportedData = result.getExportedData();
    assertEquals(1, exportedData.getVideos().size());
    assertEquals(new VideoModel(VIDEO_ID + ".mp4", VIDEO_SOURCE, VIDEO_NAME, "video/mp4", VIDEO_ID, null, false), exportedData.getVideos().toArray()[0]);
}
Also used : ExportInformation(org.datatransferproject.types.common.ExportInformation) VideosContainerResource(org.datatransferproject.types.common.models.videos.VideosContainerResource) TokensAndUrlAuthData(org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData) VideoModel(org.datatransferproject.types.common.models.videos.VideoModel) Test(org.junit.Test)

Example 14 with TokensAndUrlAuthData

use of org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData in project data-transfer-project by google.

the class DaybookPostsImporter method insertActivity.

private String insertActivity(IdempotentImportExecutor executor, SocialActivityModel activity, TokensAndUrlAuthData authData) throws IOException {
    Map<String, String> imageMap = new HashMap<>();
    Map<String, String> linkMap = new HashMap<>();
    String content = activity.getContent() == null ? "" : activity.getContent();
    String title = activity.getTitle() == null ? "" : activity.getTitle();
    String location = activity.getLocation() == null || activity.getLocation().getName() == null ? "" : activity.getLocation().getName();
    String published = activity.getPublished().toString() == null ? "" : activity.getPublished().toString();
    Request.Builder requestBuilder = new Request.Builder().url(baseUrl);
    requestBuilder.header("token", authData.getAccessToken());
    FormBody.Builder builder = new FormBody.Builder().add("type", "POSTS");
    builder.add("exporter", JobMetadata.getExportService());
    builder.add("content", content);
    builder.add("title", title);
    builder.add("location", location);
    builder.add("published", published);
    Collection<SocialActivityAttachment> linkAttachments = activity.getAttachments().stream().filter(attachment -> attachment.getType() == SocialActivityAttachmentType.LINK).collect(Collectors.toList());
    Collection<SocialActivityAttachment> imageAttachments = activity.getAttachments().stream().filter(attachment -> attachment.getType() == SocialActivityAttachmentType.IMAGE).collect(Collectors.toList());
    // don't know how they were laid out in the originating service.
    if (!linkAttachments.isEmpty()) {
        for (SocialActivityAttachment attachment : linkAttachments) {
            linkMap.put(attachment.getName(), attachment.getUrl());
        }
        try {
            String json = objectMapper.writeValueAsString(linkMap);
            builder.add("link", json);
        } catch (JsonProcessingException e) {
            monitor.info(() -> String.format("Error processing JSON: %s", e.getMessage()));
        }
    }
    if (!imageAttachments.isEmpty()) {
        for (SocialActivityAttachment image : imageAttachments) {
            imageMap.put(image.getName() != null ? image.getName() : image.getUrl(), image.getUrl());
        }
        try {
            String json = objectMapper.writeValueAsString(imageMap);
            builder.add("image", json);
        } catch (JsonProcessingException e) {
            monitor.info(() -> String.format("Error processing JSON: %s", e.getMessage()));
        }
    }
    FormBody formBody = builder.build();
    requestBuilder.post(formBody);
    try (Response response = client.newCall(requestBuilder.build()).execute()) {
        int code = response.code();
        // Though sometimes it returns error code for success requests
        if (code < 200 || code > 299) {
            throw new IOException(String.format("Error occurred in request for adding entry, message: %s", response.message()));
        }
        return response.message();
    }
}
Also used : IdempotentImportExecutor(org.datatransferproject.spi.transfer.idempotentexecutor.IdempotentImportExecutor) ImportResult(org.datatransferproject.spi.transfer.provider.ImportResult) SocialActivityAttachment(org.datatransferproject.types.common.models.social.SocialActivityAttachment) Collection(java.util.Collection) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) SocialActivityContainerResource(org.datatransferproject.types.common.models.social.SocialActivityContainerResource) SocialActivityType(org.datatransferproject.types.common.models.social.SocialActivityType) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException) IOException(java.io.IOException) HashMap(java.util.HashMap) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) Monitor(org.datatransferproject.api.launcher.Monitor) TokensAndUrlAuthData(org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData) okhttp3(okhttp3) JobMetadata(org.datatransferproject.transfer.JobMetadata) Map(java.util.Map) SocialActivityAttachmentType(org.datatransferproject.types.common.models.social.SocialActivityAttachmentType) Importer(org.datatransferproject.spi.transfer.provider.Importer) SocialActivityModel(org.datatransferproject.types.common.models.social.SocialActivityModel) HashMap(java.util.HashMap) SocialActivityAttachment(org.datatransferproject.types.common.models.social.SocialActivityAttachment) IOException(java.io.IOException) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException)

Aggregations

TokensAndUrlAuthData (org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData)14 Monitor (org.datatransferproject.api.launcher.Monitor)7 Before (org.junit.Before)6 IdempotentImportExecutor (org.datatransferproject.spi.transfer.idempotentexecutor.IdempotentImportExecutor)5 IOException (java.io.IOException)4 UUID (java.util.UUID)4 ExportInformation (org.datatransferproject.types.common.ExportInformation)4 Test (org.junit.Test)4 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)3 Collection (java.util.Collection)3 Collectors (java.util.stream.Collectors)3 ImportResult (org.datatransferproject.spi.transfer.provider.ImportResult)3 Importer (org.datatransferproject.spi.transfer.provider.Importer)3 KoofrClient (org.datatransferproject.transfer.koofr.common.KoofrClient)3 KoofrClientFactory (org.datatransferproject.transfer.koofr.common.KoofrClientFactory)3 Credential (com.google.api.client.auth.oauth2.Credential)2 VisibleForTesting (com.google.common.annotations.VisibleForTesting)2 Preconditions (com.google.common.base.Preconditions)2 Strings (com.google.common.base.Strings)2 HashMap (java.util.HashMap)2