Search in sources :

Example 1 with ContainerResource

use of org.dataportabilityproject.types.transfer.models.ContainerResource in project data-transfer-project by google.

the class GoogleCalendarExporterTest method exportEventFirstSet.

@Test
public void exportEventFirstSet() throws IOException {
    setUpSingleEventResponse();
    // Looking at first page, with at least one page after it
    ContainerResource containerResource = new IdOnlyContainerResource(CALENDAR_ID);
    ExportInformation exportInformation = new ExportInformation(null, containerResource);
    eventListResponse.setNextPageToken(NEXT_TOKEN);
    // Run test
    ExportResult<CalendarContainerResource> result = googleCalendarExporter.export(UUID.randomUUID(), null, exportInformation);
    // Check results
    // Verify correct methods were called
    verify(calendarEvents).list(CALENDAR_ID);
    verify(eventListRequest).setMaxAttendees(MAX_ATTENDEES);
    verify(eventListRequest).execute();
    // Check events
    Collection<CalendarEventModel> actualEvents = result.getExportedData().getEvents();
    assertThat(actualEvents.stream().map(CalendarEventModel::getCalendarId).collect(Collectors.toList())).containsExactly(CALENDAR_ID);
    assertThat(actualEvents.stream().map(CalendarEventModel::getTitle).collect(Collectors.toList())).containsExactly(EVENT_DESCRIPTION);
    // Check pagination token
    ContinuationData continuationData = (ContinuationData) result.getContinuationData();
    StringPaginationToken paginationToken = (StringPaginationToken) continuationData.getPaginationData();
    assertThat(paginationToken.getToken()).isEqualTo(EVENT_TOKEN_PREFIX + NEXT_TOKEN);
}
Also used : ExportInformation(org.dataportabilityproject.spi.transfer.types.ExportInformation) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) CalendarContainerResource(org.dataportabilityproject.types.transfer.models.calendar.CalendarContainerResource) ContainerResource(org.dataportabilityproject.types.transfer.models.ContainerResource) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) CalendarEventModel(org.dataportabilityproject.types.transfer.models.calendar.CalendarEventModel) CalendarContainerResource(org.dataportabilityproject.types.transfer.models.calendar.CalendarContainerResource) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) Test(org.junit.Test)

Example 2 with ContainerResource

use of org.dataportabilityproject.types.transfer.models.ContainerResource in project data-transfer-project by google.

the class GoogleCalendarExporterTest method exportCalendarFirstSet.

@Test
public void exportCalendarFirstSet() throws IOException {
    setUpSingleCalendarResponse();
    // Looking at first page, with at least one page after it
    calendarListResponse.setNextPageToken(NEXT_TOKEN);
    // Run test
    ExportResult<CalendarContainerResource> result = googleCalendarExporter.export(JOB_ID, null);
    // Check results
    // Verify correct methods were called
    verify(calendarClient).calendarList();
    verify(calendarCalendarList).list();
    verify(calendarListRequest).execute();
    // Check pagination token
    ContinuationData continuationData = (ContinuationData) result.getContinuationData();
    StringPaginationToken paginationToken = (StringPaginationToken) continuationData.getPaginationData();
    assertThat(paginationToken.getToken()).isEqualTo(CALENDAR_TOKEN_PREFIX + NEXT_TOKEN);
    // Check calendars
    Collection<CalendarModel> actualCalendars = result.getExportedData().getCalendars();
    assertThat(actualCalendars.stream().map(CalendarModel::getId).collect(Collectors.toList())).containsExactly(CALENDAR_ID);
    // Check events (should be empty, even though there is an event in the calendar
    Collection<CalendarEventModel> actualEvents = result.getExportedData().getEvents();
    assertThat(actualEvents).isEmpty();
    // Should be one container in the resource list
    List<ContainerResource> actualResources = continuationData.getContainerResources();
    assertThat(actualResources.stream().map(a -> ((IdOnlyContainerResource) a).getId()).collect(Collectors.toList())).containsExactly(CALENDAR_ID);
}
Also used : IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult) CalendarContainerResource(org.dataportabilityproject.types.transfer.models.calendar.CalendarContainerResource) Events(com.google.api.services.calendar.model.Events) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) Calendar(com.google.api.services.calendar.Calendar) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) Before(org.junit.Before) Event(com.google.api.services.calendar.model.Event) ExportInformation(org.dataportabilityproject.spi.transfer.types.ExportInformation) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) InOrder(org.mockito.InOrder) EVENT_TOKEN_PREFIX(org.dataportabilityproject.datatransfer.google.common.GoogleStaticObjects.EVENT_TOKEN_PREFIX) CalendarList(com.google.api.services.calendar.model.CalendarList) Collection(java.util.Collection) Test(org.junit.Test) IOException(java.io.IOException) MAX_ATTENDEES(org.dataportabilityproject.datatransfer.google.common.GoogleStaticObjects.MAX_ATTENDEES) UUID(java.util.UUID) Mockito.when(org.mockito.Mockito.when) Truth.assertThat(com.google.common.truth.Truth.assertThat) Collectors(java.util.stream.Collectors) Mockito.verify(org.mockito.Mockito.verify) Mockito(org.mockito.Mockito) List(java.util.List) CalendarListEntry(com.google.api.services.calendar.model.CalendarListEntry) CALENDAR_TOKEN_PREFIX(org.dataportabilityproject.datatransfer.google.common.GoogleStaticObjects.CALENDAR_TOKEN_PREFIX) CalendarEventModel(org.dataportabilityproject.types.transfer.models.calendar.CalendarEventModel) ContainerResource(org.dataportabilityproject.types.transfer.models.ContainerResource) Collections(java.util.Collections) CalendarModel(org.dataportabilityproject.types.transfer.models.calendar.CalendarModel) Mockito.mock(org.mockito.Mockito.mock) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) CalendarContainerResource(org.dataportabilityproject.types.transfer.models.calendar.CalendarContainerResource) ContainerResource(org.dataportabilityproject.types.transfer.models.ContainerResource) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) CalendarModel(org.dataportabilityproject.types.transfer.models.calendar.CalendarModel) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) CalendarEventModel(org.dataportabilityproject.types.transfer.models.calendar.CalendarEventModel) CalendarContainerResource(org.dataportabilityproject.types.transfer.models.calendar.CalendarContainerResource) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) Test(org.junit.Test)

Example 3 with ContainerResource

use of org.dataportabilityproject.types.transfer.models.ContainerResource in project data-transfer-project by google.

the class TasksModelWrapperTest method verifySerializeDeserialize.

@Test
public void verifySerializeDeserialize() throws Exception {
    ObjectMapper objectMapper = new ObjectMapper();
    objectMapper.registerSubtypes(TaskContainerResource.class);
    List<TaskListModel> taskLists = ImmutableList.of(new TaskListModel("id1", "List 1"));
    List<TaskModel> tasks = ImmutableList.of(new TaskModel("id1", "Write Better tests", "Do this soon"), new TaskModel("id1", "Liberate all the data", "do this in stages"));
    ContainerResource data = new TaskContainerResource(taskLists, tasks);
    String serialized = objectMapper.writeValueAsString(data);
    ContainerResource deserializedModel = objectMapper.readValue(serialized, ContainerResource.class);
    Truth.assertThat(deserializedModel).isNotNull();
    Truth.assertThat(deserializedModel).isInstanceOf(TaskContainerResource.class);
    TaskContainerResource deserialized = (TaskContainerResource) deserializedModel;
    Truth.assertThat(deserialized.getLists()).hasSize(1);
    Truth.assertThat(deserialized.getTasks()).hasSize(2);
}
Also used : ContainerResource(org.dataportabilityproject.types.transfer.models.ContainerResource) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Test(org.junit.Test)

Example 4 with ContainerResource

use of org.dataportabilityproject.types.transfer.models.ContainerResource in project data-transfer-project by google.

the class PortabilityInMemoryDataCopier method copyHelper.

/**
 * Transfers data from the given {@code exporter} optionally starting at the point specified in
 * the provided {@code exportInformation}. Imports the data using the provided {@code importer}.
 * If there is more data to required to be exported, recursively copies using the specific {@link
 * ExportInformation} to continue the process.
 *
 * @param exportAuthData The auth data for the export
 * @param importAuthData The auth data for the import
 * @param exportInformation Any pagination or resource information to use for subsequent calls.
 */
private void copyHelper(UUID jobId, AuthData exportAuthData, AuthData importAuthData, ExportInformation exportInformation) {
    logger.debug("copy iteration: {}", COPY_ITERATION_COUNTER.incrementAndGet());
    // NOTE: order is important below, do the import of all the items, then do continuation
    // then do sub resources, this ensures all parents are populated before children get
    // processed.
    logger.debug("Starting export, ExportInformation: {}", exportInformation);
    ExportResult<?> exportResult = exporter.get().export(jobId, exportAuthData, exportInformation);
    logger.debug("Finished export, results: {}", exportResult);
    logger.debug("Starting import");
    importer.get().importItem(jobId, importAuthData, exportResult.getExportedData());
    logger.debug("Finished import");
    ContinuationData continuationData = (ContinuationData) exportResult.getContinuationData();
    if (null != continuationData) {
        // Process the next page of items for the resource
        if (null != continuationData.getPaginationData()) {
            logger.debug("start off a new copy iteration with pagination info");
            copyHelper(jobId, exportAuthData, importAuthData, new ExportInformation(continuationData.getPaginationData(), exportInformation.getContainerResource()));
        }
        // Start processing sub-resources
        if (continuationData.getContainerResources() != null && !continuationData.getContainerResources().isEmpty()) {
            for (ContainerResource resource : continuationData.getContainerResources()) {
                copyHelper(jobId, exportAuthData, importAuthData, new ExportInformation(null, resource));
            }
        }
    }
}
Also used : ExportInformation(org.dataportabilityproject.spi.transfer.types.ExportInformation) ContainerResource(org.dataportabilityproject.types.transfer.models.ContainerResource) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData)

Example 5 with ContainerResource

use of org.dataportabilityproject.types.transfer.models.ContainerResource in project data-transfer-project by google.

the class GoogleCalendarExporterTest method exportEventSubsequentSet.

@Test
public void exportEventSubsequentSet() throws IOException {
    setUpSingleEventResponse();
    // Looking at subsequent page, with no pages after it
    ContainerResource containerResource = new IdOnlyContainerResource(CALENDAR_ID);
    PaginationData paginationData = new StringPaginationToken(EVENT_TOKEN_PREFIX + NEXT_TOKEN);
    ExportInformation exportInformation = new ExportInformation(paginationData, containerResource);
    eventListResponse.setNextPageToken(null);
    // Run test
    ExportResult<CalendarContainerResource> result = googleCalendarExporter.export(UUID.randomUUID(), null, exportInformation);
    // Check results
    // Verify correct methods were called in order
    InOrder inOrder = Mockito.inOrder(eventListRequest);
    inOrder.verify(eventListRequest).setPageToken(NEXT_TOKEN);
    inOrder.verify(eventListRequest).execute();
    // Check pagination token
    ContinuationData continuationData = (ContinuationData) result.getContinuationData();
    StringPaginationToken paginationToken = (StringPaginationToken) continuationData.getPaginationData();
    assertThat(paginationToken).isNull();
}
Also used : ExportInformation(org.dataportabilityproject.spi.transfer.types.ExportInformation) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) InOrder(org.mockito.InOrder) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) CalendarContainerResource(org.dataportabilityproject.types.transfer.models.calendar.CalendarContainerResource) ContainerResource(org.dataportabilityproject.types.transfer.models.ContainerResource) IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) CalendarContainerResource(org.dataportabilityproject.types.transfer.models.calendar.CalendarContainerResource) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) Test(org.junit.Test)

Aggregations

ContainerResource (org.dataportabilityproject.types.transfer.models.ContainerResource)8 Test (org.junit.Test)7 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)4 ContinuationData (org.dataportabilityproject.spi.transfer.types.ContinuationData)4 ExportInformation (org.dataportabilityproject.spi.transfer.types.ExportInformation)4 IdOnlyContainerResource (org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource)3 StringPaginationToken (org.dataportabilityproject.spi.transfer.types.StringPaginationToken)3 CalendarContainerResource (org.dataportabilityproject.types.transfer.models.calendar.CalendarContainerResource)3 PaginationData (org.dataportabilityproject.spi.transfer.types.PaginationData)2 CalendarEventModel (org.dataportabilityproject.types.transfer.models.calendar.CalendarEventModel)2 InOrder (org.mockito.InOrder)2 JavaTimeModule (com.fasterxml.jackson.datatype.jsr310.JavaTimeModule)1 Calendar (com.google.api.services.calendar.Calendar)1 CalendarList (com.google.api.services.calendar.model.CalendarList)1 CalendarListEntry (com.google.api.services.calendar.model.CalendarListEntry)1 Event (com.google.api.services.calendar.model.Event)1 Events (com.google.api.services.calendar.model.Events)1 Truth.assertThat (com.google.common.truth.Truth.assertThat)1 IOException (java.io.IOException)1 Collection (java.util.Collection)1