Search in sources :

Example 1 with PhotoAlbum

use of org.datatransferproject.types.common.models.photos.PhotoAlbum in project data-transfer-project by google.

the class FacebookPhotosExporter method export.

@Override
public ExportResult<PhotosContainerResource> export(UUID jobId, TokensAndUrlAuthData authData, Optional<ExportInformation> exportInformation) throws CopyExceptionWithFailureReason {
    Preconditions.checkNotNull(authData);
    if (!exportInformation.isPresent()) {
        // Start by getting the list of albums to export
        return exportAlbums(authData, Optional.empty());
    }
    StringPaginationToken paginationToken = (StringPaginationToken) exportInformation.get().getPaginationData();
    ContainerResource containerResource = exportInformation.get().getContainerResource();
    boolean containerResourcePresent = containerResource != null;
    boolean paginationDataPresent = paginationToken != null;
    if (!containerResourcePresent && paginationDataPresent && paginationToken.getToken().startsWith(ALBUM_TOKEN_PREFIX)) {
        // Continue exporting albums
        return exportAlbums(authData, Optional.of(paginationToken));
    } else if (containerResourcePresent && containerResource instanceof PhotosContainerResource) {
        // We have had albums specified from the front end so process them for import
        PhotosContainerResource photosContainerResource = (PhotosContainerResource) containerResource;
        Preconditions.checkNotNull(photosContainerResource.getAlbums());
        ContinuationData continuationData = new ContinuationData(null);
        for (PhotoAlbum album : photosContainerResource.getAlbums()) {
            continuationData.addContainerResource(new IdOnlyContainerResource(album.getId()));
        }
        return new ExportResult<>(ExportResult.ResultType.CONTINUE, photosContainerResource, continuationData);
    } else if (containerResourcePresent && containerResource instanceof IdOnlyContainerResource) {
        // Export photos
        return exportPhotos(jobId, authData, (IdOnlyContainerResource) containerResource, Optional.ofNullable(paginationToken));
    } else {
        throw new IllegalStateException(String.format("Invalid state passed into FacebookPhotosExporter. ExportInformation: %s", exportInformation));
    }
}
Also used : PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) ContainerResource(org.datatransferproject.types.common.models.ContainerResource) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken)

Example 2 with PhotoAlbum

use of org.datatransferproject.types.common.models.photos.PhotoAlbum in project data-transfer-project by google.

the class FacebookPhotosExporter method exportAlbums.

private ExportResult<PhotosContainerResource> exportAlbums(TokensAndUrlAuthData authData, Optional<StringPaginationToken> paginationData) throws CopyExceptionWithFailureReason {
    Optional<String> paginationToken = stripTokenPrefix(paginationData, ALBUM_TOKEN_PREFIX);
    // Get albums
    Connection<Album> connection = getOrCreatePhotosInterface(authData).getAlbums(paginationToken);
    PaginationData nextPageData = null;
    String token = connection.getAfterCursor();
    if (!Strings.isNullOrEmpty(token)) {
        nextPageData = new StringPaginationToken(ALBUM_TOKEN_PREFIX + token);
    }
    ContinuationData continuationData = new ContinuationData(nextPageData);
    List<Album> albums = connection.getData();
    if (albums.isEmpty()) {
        return new ExportResult<>(ExportResult.ResultType.END, null, null);
    }
    ArrayList<PhotoAlbum> exportAlbums = new ArrayList<>();
    for (Album album : albums) {
        exportAlbums.add(new PhotoAlbum(album.getId(), album.getName(), album.getDescription()));
        continuationData.addContainerResource(new IdOnlyContainerResource(album.getId()));
    }
    return new ExportResult<>(ExportResult.ResultType.CONTINUE, new PhotosContainerResource(exportAlbums, null), continuationData);
}
Also used : PaginationData(org.datatransferproject.types.common.PaginationData) ArrayList(java.util.ArrayList) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) Album(com.restfb.types.Album) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult)

Example 3 with PhotoAlbum

use of org.datatransferproject.types.common.models.photos.PhotoAlbum in project data-transfer-project by google.

the class BackblazePhotosImporterTest method testImportAlbum.

@Test
public void testImportAlbum() throws Exception {
    String albumId = "albumId";
    PhotoAlbum album = new PhotoAlbum(albumId, "", "");
    ArrayList<PhotoAlbum> albums = new ArrayList<>();
    albums.add(album);
    PhotosContainerResource data = mock(PhotosContainerResource.class);
    when(data.getAlbums()).thenReturn(albums);
    BackblazePhotosImporter sut = new BackblazePhotosImporter(monitor, dataStore, streamProvider, clientFactory);
    sut.importItem(UUID.randomUUID(), executor, authData, data);
    verify(executor, times(1)).executeAndSwallowIOExceptions(eq(albumId), eq("Caching album name for album 'albumId'"), any());
}
Also used : PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) ArrayList(java.util.ArrayList) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) Test(org.junit.Test)

Example 4 with PhotoAlbum

use of org.datatransferproject.types.common.models.photos.PhotoAlbum in project data-transfer-project by google.

the class SmugMugPhotosImporter method getDestinationAlbumTempData.

/**
 * Get the proper album upload information for the photo. Takes into account size limits of the
 * albums and completed uploads.
 */
@VisibleForTesting
SmugMugPhotoTempData getDestinationAlbumTempData(UUID jobId, IdempotentImportExecutor idempotentExecutor, String baseAlbumId, SmugMugInterface smugMugInterface) throws Exception {
    SmugMugPhotoTempData baseAlbumTempData = jobStore.findData(jobId, getTempDataId(baseAlbumId), SmugMugPhotoTempData.class);
    SmugMugPhotoTempData albumTempData = baseAlbumTempData;
    int depth = 0;
    while (albumTempData.getPhotoCount() >= transmogrificationConfig.getAlbumMaxSize()) {
        if (albumTempData.getOverflowAlbumExportId() == null) {
            PhotoAlbum newAlbum = createOverflowAlbum(baseAlbumTempData.getAlbumExportId(), baseAlbumTempData.getAlbumName(), baseAlbumTempData.getAlbumDescription(), depth + 1);
            // since the album is full and has no overflow, we need to create a new one
            String newUri = idempotentExecutor.executeOrThrowException(newAlbum.getId(), newAlbum.getName(), () -> importSingleAlbum(jobId, newAlbum, smugMugInterface));
            albumTempData.setOverflowAlbumExportId(newAlbum.getId());
            jobStore.update(jobId, getTempDataId(albumTempData.getAlbumExportId()), albumTempData);
            albumTempData = jobStore.findData(jobId, getTempDataId(albumTempData.getOverflowAlbumExportId()), SmugMugPhotoTempData.class);
        } else {
            albumTempData = jobStore.findData(jobId, getTempDataId(albumTempData.getOverflowAlbumExportId()), SmugMugPhotoTempData.class);
        }
        depth += 1;
    }
    return albumTempData;
}
Also used : PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 5 with PhotoAlbum

use of org.datatransferproject.types.common.models.photos.PhotoAlbum in project data-transfer-project by google.

the class MicrosoftPhotosExporterTest method exportOneAlbumWithNextPage.

@Test
public void exportOneAlbumWithNextPage() throws IOException {
    // Setup
    MicrosoftDriveItem folderItem = setUpSingleAlbum();
    when(driveItemsResponse.getDriveItems()).thenReturn(new MicrosoftDriveItem[] { folderItem });
    when(driveItemsResponse.getNextPageLink()).thenReturn(DRIVE_PAGE_URL);
    // Run
    ExportResult<PhotosContainerResource> result = microsoftPhotosExporter.exportOneDrivePhotos(null, Optional.empty(), Optional.empty(), uuid);
    // Verify method calls
    verify(photosInterface).getDriveItemsFromSpecialFolder(MicrosoftSpecialFolder.FolderType.photos);
    verify(driveItemsResponse).getDriveItems();
    // Verify pagination token is set
    ContinuationData continuationData = result.getContinuationData();
    StringPaginationToken paginationToken = (StringPaginationToken) continuationData.getPaginationData();
    assertThat(paginationToken.getToken()).isEqualTo(DRIVE_TOKEN_PREFIX + DRIVE_PAGE_URL);
    // Verify one album is ready for import
    Collection<PhotoAlbum> actualAlbums = result.getExportedData().getAlbums();
    assertThat(actualAlbums.stream().map(PhotoAlbum::getId).collect(Collectors.toList())).containsExactly(FOLDER_ID);
    // Verify photos should be empty (in the root)
    Collection<PhotoModel> actualPhotos = result.getExportedData().getPhotos();
    assertThat(actualPhotos).isEmpty();
    // Verify there is one container ready for sub-processing
    List<ContainerResource> actualResources = continuationData.getContainerResources();
    assertThat(actualResources.stream().map(a -> ((IdOnlyContainerResource) a).getId()).collect(Collectors.toList())).containsExactly(FOLDER_ID);
}
Also used : ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) Matchers(org.mockito.Matchers) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) Before(org.junit.Before) JacksonFactory(com.google.api.client.json.jackson2.JacksonFactory) Collection(java.util.Collection) IOException(java.io.IOException) ContainerResource(org.datatransferproject.types.common.models.ContainerResource) Test(org.junit.Test) UUID(java.util.UUID) Truth.assertThat(com.google.common.truth.Truth.assertThat) Collectors(java.util.stream.Collectors) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) Mockito(org.mockito.Mockito) List(java.util.List) Monitor(org.datatransferproject.api.launcher.Monitor) TokensAndUrlAuthData(org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData) DRIVE_TOKEN_PREFIX(org.datatransferproject.transfer.microsoft.photos.MicrosoftPhotosExporter.DRIVE_TOKEN_PREFIX) Optional(java.util.Optional) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) MicrosoftCredentialFactory(org.datatransferproject.transfer.microsoft.common.MicrosoftCredentialFactory) org.datatransferproject.transfer.microsoft.driveModels(org.datatransferproject.transfer.microsoft.driveModels) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) ContainerResource(org.datatransferproject.types.common.models.ContainerResource) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) Test(org.junit.Test)

Aggregations

PhotoAlbum (org.datatransferproject.types.common.models.photos.PhotoAlbum)45 PhotosContainerResource (org.datatransferproject.types.common.models.photos.PhotosContainerResource)36 Test (org.junit.Test)27 PhotoModel (org.datatransferproject.types.common.models.photos.PhotoModel)23 IdOnlyContainerResource (org.datatransferproject.types.common.models.IdOnlyContainerResource)19 ContinuationData (org.datatransferproject.spi.transfer.types.ContinuationData)18 ExportResult (org.datatransferproject.spi.transfer.provider.ExportResult)15 ArrayList (java.util.ArrayList)12 StringPaginationToken (org.datatransferproject.types.common.StringPaginationToken)12 UUID (java.util.UUID)11 IOException (java.io.IOException)8 PaginationData (org.datatransferproject.types.common.PaginationData)7 GoogleAlbum (org.datatransferproject.datatransfer.google.mediaModels.GoogleAlbum)6 ImportResult (org.datatransferproject.spi.transfer.provider.ImportResult)6 ContainerResource (org.datatransferproject.types.common.models.ContainerResource)6 ArgumentMatchers.anyString (org.mockito.ArgumentMatchers.anyString)6 InputStreamWrapper (org.datatransferproject.spi.cloud.storage.TemporaryPerJobDataStore.InputStreamWrapper)5 PortabilityJob (org.datatransferproject.spi.cloud.types.PortabilityJob)5 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)4 JacksonFactory (com.google.api.client.json.jackson2.JacksonFactory)4