Search in sources :

Example 31 with PhotoAlbum

use of org.datatransferproject.types.common.models.photos.PhotoAlbum in project data-transfer-project by google.

the class KoofrPhotosImporterTest method testImportItemFromJobStoreUserTimeZone.

@Test
public void testImportItemFromJobStoreUserTimeZone() throws Exception {
    ByteArrayInputStream inputStream = new ByteArrayInputStream(new byte[] { 0, 1, 2, 3, 4 });
    when(jobStore.getStream(any(), any())).thenReturn(new InputStreamWrapper(inputStream, 5L));
    UUID jobId = UUID.randomUUID();
    PortabilityJob job = mock(PortabilityJob.class);
    when(job.userTimeZone()).thenReturn(TimeZone.getTimeZone("Europe/Rome"));
    when(jobStore.findJob(jobId)).thenReturn(job);
    Collection<PhotoAlbum> albums = ImmutableList.of(new PhotoAlbum("id1", "Album 1", "This is a fake album"));
    DateFormat format = new SimpleDateFormat("yyyy:MM:dd HH:mm:ss");
    format.setTimeZone(TimeZone.getTimeZone("Europe/Kiev"));
    Collection<PhotoModel> photos = ImmutableList.of(new PhotoModel("pic1.jpg", "http://fake.com/1.jpg", "A pic", "image/jpeg", "p1", "id1", true, format.parse("2021:02:16 11:55:00")));
    PhotosContainerResource resource = spy(new PhotosContainerResource(albums, photos));
    importer.importItem(jobId, executor, authData, resource);
    InOrder clientInOrder = Mockito.inOrder(client);
    clientInOrder.verify(client).uploadFile(any(), eq("2021-02-16 10.55.00 pic1.jpg"), any(), any(), any(), any());
}
Also used : PortabilityJob(org.datatransferproject.spi.cloud.types.PortabilityJob) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) InputStreamWrapper(org.datatransferproject.spi.cloud.storage.TemporaryPerJobDataStore.InputStreamWrapper) InOrder(org.mockito.InOrder) ByteArrayInputStream(java.io.ByteArrayInputStream) SimpleDateFormat(java.text.SimpleDateFormat) DateFormat(java.text.DateFormat) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) UUID(java.util.UUID) SimpleDateFormat(java.text.SimpleDateFormat) Test(org.junit.Test)

Example 32 with PhotoAlbum

use of org.datatransferproject.types.common.models.photos.PhotoAlbum in project data-transfer-project by google.

the class MicrosoftPhotosImporter method importItem.

@Override
public ImportResult importItem(UUID jobId, IdempotentImportExecutor idempotentImportExecutor, TokensAndUrlAuthData authData, PhotosContainerResource resource) throws Exception {
    // Ensure credential is populated
    getOrCreateCredential(authData);
    monitor.debug(() -> String.format("%s: Importing %s albums and %s photos before transmogrification", jobId, resource.getAlbums().size(), resource.getPhotos().size()));
    // Make the data onedrive compatible
    resource.transmogrify(transmogrificationConfig);
    monitor.debug(() -> String.format("%s: Importing %s albums and %s photos after transmogrification", jobId, resource.getAlbums().size(), resource.getPhotos().size()));
    for (PhotoAlbum album : resource.getAlbums()) {
        // Create a OneDrive folder and then save the id with the mapping data
        idempotentImportExecutor.executeAndSwallowIOExceptions(album.getId(), album.getName(), () -> createOneDriveFolder(album));
    }
    for (PhotoModel photoModel : resource.getPhotos()) {
        idempotentImportExecutor.executeAndSwallowIOExceptions(IdempotentImportExecutorHelper.getPhotoIdempotentId(photoModel), photoModel.getTitle(), () -> importSinglePhoto(photoModel, jobId, idempotentImportExecutor));
    }
    return ImportResult.OK;
}
Also used : PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum)

Example 33 with PhotoAlbum

use of org.datatransferproject.types.common.models.photos.PhotoAlbum in project data-transfer-project by google.

the class GooglePhotosExporterTest method exportPhotoFirstSet.

@Test
public void exportPhotoFirstSet() throws IOException, InvalidTokenException, PermissionDeniedException {
    setUpSingleAlbum();
    when(albumListResponse.getNextPageToken()).thenReturn(null);
    GoogleMediaItem mediaItem = setUpSinglePhoto(IMG_URI, PHOTO_ID);
    when(mediaItemSearchResponse.getMediaItems()).thenReturn(new GoogleMediaItem[] { mediaItem });
    when(mediaItemSearchResponse.getNextPageToken()).thenReturn(PHOTO_TOKEN);
    IdOnlyContainerResource idOnlyContainerResource = new IdOnlyContainerResource(ALBUM_ID);
    ExportResult<PhotosContainerResource> result = googlePhotosExporter.exportPhotos(null, Optional.of(idOnlyContainerResource), Optional.empty(), uuid);
    // Check results
    // Verify correct methods were called
    verify(photosInterface).listMediaItems(Optional.of(ALBUM_ID), Optional.empty());
    verify(mediaItemSearchResponse).getMediaItems();
    // Check pagination
    ContinuationData continuationData = result.getContinuationData();
    StringPaginationToken paginationToken = (StringPaginationToken) continuationData.getPaginationData();
    assertThat(paginationToken.getToken()).isEqualTo(PHOTO_TOKEN_PREFIX + PHOTO_TOKEN);
    // Check albums field of container (should be empty)
    Collection<PhotoAlbum> actualAlbums = result.getExportedData().getAlbums();
    assertThat(actualAlbums).isEmpty();
    // Check photos field of container
    Collection<PhotoModel> actualPhotos = result.getExportedData().getPhotos();
    assertThat(actualPhotos.stream().map(PhotoModel::getFetchableUrl).collect(Collectors.toList())).containsExactly(// for download
    IMG_URI + "=d");
    assertThat(actualPhotos.stream().map(PhotoModel::getAlbumId).collect(Collectors.toList())).containsExactly(ALBUM_ID);
    assertThat(actualPhotos.stream().map(PhotoModel::getTitle).collect(Collectors.toList())).containsExactly(FILENAME);
}
Also used : PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) GoogleMediaItem(org.datatransferproject.datatransfer.google.mediaModels.GoogleMediaItem) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) Test(org.junit.Test)

Example 34 with PhotoAlbum

use of org.datatransferproject.types.common.models.photos.PhotoAlbum in project data-transfer-project by google.

the class GooglePhotosImporterTest method importAlbum.

@Test
public void importAlbum() throws Exception {
    // Set up
    String albumName = "Album Name";
    String albumDescription = "Album description";
    PhotoAlbum albumModel = new PhotoAlbum(OLD_ALBUM_ID, albumName, albumDescription);
    GoogleAlbum responseAlbum = new GoogleAlbum();
    responseAlbum.setId(NEW_ALBUM_ID);
    Mockito.when(googlePhotosInterface.createAlbum(any(GoogleAlbum.class))).thenReturn(responseAlbum);
    // Run test
    googlePhotosImporter.importSingleAlbum(uuid, null, albumModel);
    // Check results
    ArgumentCaptor<GoogleAlbum> albumArgumentCaptor = ArgumentCaptor.forClass(GoogleAlbum.class);
    Mockito.verify(googlePhotosInterface).createAlbum(albumArgumentCaptor.capture());
    assertEquals(albumArgumentCaptor.getValue().getTitle(), albumName);
    assertNull(albumArgumentCaptor.getValue().getId());
}
Also used : ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) GoogleAlbum(org.datatransferproject.datatransfer.google.mediaModels.GoogleAlbum) Test(org.junit.Test)

Example 35 with PhotoAlbum

use of org.datatransferproject.types.common.models.photos.PhotoAlbum in project data-transfer-project by google.

the class ImgurPhotosExporter method requestNonAlbumPhotos.

/**
 * Queries all photos for the account. Chooses photos which are not included to the collection of
 * photos from albums.
 *
 * @param authData authentication information
 * @param paginationData pagination information to use for subsequent calls.
 */
private ExportResult<PhotosContainerResource> requestNonAlbumPhotos(TokensAndUrlAuthData authData, PaginationData paginationData, UUID jobId) throws IOException {
    int page = paginationData == null ? 0 : ((IntPaginationToken) paginationData).getStart();
    String url = format(ALL_PHOTOS_URL_TEMPLATE, page);
    Set<PhotoAlbum> albums = new HashSet<>();
    List<PhotoModel> photos = new ArrayList<>();
    List<Map<String, Object>> items = requestData(authData, url);
    boolean hasMore = (items != null && items.size() != 0);
    for (Map<String, Object> item : items) {
        String photoId = (String) item.get("id");
        // Select photos which are not included to the collection of retrieved album photos
        if (!albumPhotos.contains(photoId)) {
            PhotoModel photoModel = new PhotoModel((String) item.get("name"), (String) item.get("link"), (String) item.get("description"), (String) item.get("type"), (String) item.get("id"), DEFAULT_ALBUM_ID, true);
            photos.add(photoModel);
            InputStream inputStream = getImageAsStream(photoModel.getFetchableUrl());
            jobStore.create(jobId, photoModel.getFetchableUrl(), inputStream);
        }
    }
    if (!containsNonAlbumPhotos && photos.size() > 0) {
        // Add album for non-album photos
        albums.add(new PhotoAlbum(DEFAULT_ALBUM_ID, "Non-album photos", "Contains non-album photos"));
        // Make sure album will not be added multiply times on subsequent calls
        containsNonAlbumPhotos = true;
    }
    PaginationData newPage = null;
    if (hasMore) {
        newPage = new IntPaginationToken(page + 1);
        monitor.info(() -> format("added non-album photos, size: %s", photos.size()));
    }
    PhotosContainerResource photosContainerResource = new PhotosContainerResource(albums, photos);
    ContinuationData continuationData = new ContinuationData(newPage);
    ExportResult.ResultType resultType = ExportResult.ResultType.CONTINUE;
    if (newPage == null) {
        resultType = ExportResult.ResultType.END;
    }
    return new ExportResult<>(resultType, photosContainerResource, continuationData);
}
Also used : IntPaginationToken(org.datatransferproject.types.common.IntPaginationToken) PaginationData(org.datatransferproject.types.common.PaginationData) InputStream(java.io.InputStream) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) ArrayList(java.util.ArrayList) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) Map(java.util.Map) HashSet(java.util.HashSet) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult)

Aggregations

PhotoAlbum (org.datatransferproject.types.common.models.photos.PhotoAlbum)45 PhotosContainerResource (org.datatransferproject.types.common.models.photos.PhotosContainerResource)36 Test (org.junit.Test)27 PhotoModel (org.datatransferproject.types.common.models.photos.PhotoModel)23 IdOnlyContainerResource (org.datatransferproject.types.common.models.IdOnlyContainerResource)19 ContinuationData (org.datatransferproject.spi.transfer.types.ContinuationData)18 ExportResult (org.datatransferproject.spi.transfer.provider.ExportResult)15 ArrayList (java.util.ArrayList)12 StringPaginationToken (org.datatransferproject.types.common.StringPaginationToken)12 UUID (java.util.UUID)11 IOException (java.io.IOException)8 PaginationData (org.datatransferproject.types.common.PaginationData)7 GoogleAlbum (org.datatransferproject.datatransfer.google.mediaModels.GoogleAlbum)6 ImportResult (org.datatransferproject.spi.transfer.provider.ImportResult)6 ContainerResource (org.datatransferproject.types.common.models.ContainerResource)6 ArgumentMatchers.anyString (org.mockito.ArgumentMatchers.anyString)6 InputStreamWrapper (org.datatransferproject.spi.cloud.storage.TemporaryPerJobDataStore.InputStreamWrapper)5 PortabilityJob (org.datatransferproject.spi.cloud.types.PortabilityJob)5 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)4 JacksonFactory (com.google.api.client.json.jackson2.JacksonFactory)4