Search in sources :

Example 41 with PhotosContainerResource

use of org.datatransferproject.types.common.models.photos.PhotosContainerResource in project data-transfer-project by google.

the class ImgurPhotoExporterTest method testAlbumsExport.

@Test
public void testAlbumsExport() throws Exception {
    server.enqueue(new MockResponse().setBody(albumsResponse));
    // export albums
    ExportResult<PhotosContainerResource> result = exporter.export(UUID.randomUUID(), token, Optional.empty());
    PhotosContainerResource resource = result.getExportedData();
    assertThat(resource.getPhotos()).isEmpty();
    PhotoAlbum album1 = resource.getAlbums().stream().filter(album -> "albumId1".equals(album.getId())).findFirst().get();
    assertThat(album1.getName()).isEqualTo("Album 1");
    assertThat(album1.getDescription()).isEqualTo(null);
    PhotoAlbum album2 = resource.getAlbums().stream().filter(album -> "albumId2".equals(album.getId())).findFirst().get();
    assertThat(album2.getName()).isEqualTo("Album 2");
    assertThat(album2.getDescription()).isEqualTo("Description for Album 2");
    assertThat(resource.getAlbums()).containsExactly(album1, album2).inOrder();
}
Also used : MockResponse(com.squareup.okhttp.mockwebserver.MockResponse) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) Test(org.junit.Test)

Example 42 with PhotosContainerResource

use of org.datatransferproject.types.common.models.photos.PhotosContainerResource in project data-transfer-project by google.

the class ImgurPhotoExporterTest method testAlbumPhotosExport.

@Test
public void testAlbumPhotosExport() throws Exception {
    server.enqueue(new MockResponse().setBody(albumsResponse));
    server.enqueue(new MockResponse().setBody(album1ImagesResponse));
    // export albums
    exporter.export(UUID.randomUUID(), token, Optional.empty());
    // export album photos
    ExportResult<PhotosContainerResource> result = exporter.export(UUID.randomUUID(), token, Optional.of(new ExportInformation(null, new IdOnlyContainerResource("albumId1"))));
    assertThat(result.getExportedData().getPhotos()).containsExactly(ALBUM_PHOTO_1, ALBUM_PHOTO_2).inOrder();
}
Also used : MockResponse(com.squareup.okhttp.mockwebserver.MockResponse) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) ExportInformation(org.datatransferproject.types.common.ExportInformation) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) Test(org.junit.Test)

Example 43 with PhotosContainerResource

use of org.datatransferproject.types.common.models.photos.PhotosContainerResource in project data-transfer-project by google.

the class KoofrPhotosExporter method export.

@Override
public ExportResult<PhotosContainerResource> export(UUID jobId, TokensAndUrlAuthData authData, Optional<ExportInformation> exportInformation) throws CopyExceptionWithFailureReason {
    Preconditions.checkNotNull(authData);
    KoofrClient koofrClient = koofrClientFactory.create(authData);
    KoofrMediaExport export = new KoofrMediaExport(koofrClient, monitor);
    try {
        export.export();
        List<PhotoAlbum> exportAlbums = export.getPhotoAlbums();
        List<PhotoModel> exportPhotos = export.getPhotos();
        PhotosContainerResource containerResource = new PhotosContainerResource(exportAlbums, exportPhotos);
        return new ExportResult<>(ExportResult.ResultType.END, containerResource, null);
    } catch (IOException e) {
        return new ExportResult<>(e);
    }
}
Also used : PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) KoofrClient(org.datatransferproject.transfer.koofr.common.KoofrClient) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) IOException(java.io.IOException) KoofrMediaExport(org.datatransferproject.transfer.koofr.common.KoofrMediaExport) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult)

Example 44 with PhotosContainerResource

use of org.datatransferproject.types.common.models.photos.PhotosContainerResource in project data-transfer-project by google.

the class KoofrPhotosImporterTest method testImportItemFromURL.

@Test
public void testImportItemFromURL() throws Exception {
    // blank.jpg generated using
    // convert -size 1x1 xc:white blank.jpg
    // exiftool "-CreateDate=2020:08:03 11:55:24" blank.jpg
    final byte[] blankBytes = IOUtils.toByteArray(getClass().getClassLoader().getResourceAsStream("blank.jpg"));
    server.enqueue(new MockResponse().setResponseCode(200).setBody("123"));
    server.enqueue(new MockResponse().setResponseCode(200).setBody("4567"));
    final Buffer blankBuffer = new Buffer();
    blankBuffer.write(blankBytes);
    server.enqueue(new MockResponse().setResponseCode(200).setBody(blankBuffer));
    blankBuffer.close();
    server.enqueue(new MockResponse().setResponseCode(200).setBody("89"));
    server.enqueue(new MockResponse().setResponseCode(200).setBody("0"));
    when(client.ensureRootFolder()).thenReturn("/root");
    when(executor.getCachedValue(eq("id1"))).thenReturn("/root/Album 1");
    when(executor.getCachedValue(eq("id2"))).thenReturn("/root/Album");
    when(client.fileExists("/root/Album 1/pic1.jpg")).thenReturn(false);
    when(client.fileExists("/root/Album 1/pic2.png")).thenReturn(true);
    when(client.fileExists("/root/Album 1/2020-08-03 11.55.24 pic3.jpg")).thenReturn(false);
    when(client.fileExists("/root/Album 1/2020-08-17 11.55.24 pic4.jpg")).thenReturn(false);
    when(client.fileExists("/root/Album/pic5.jpg")).thenReturn(false);
    String description1000 = new String(new char[1000]).replace("\0", "a");
    String description1001 = new String(new char[1001]).replace("\0", "a");
    UUID jobId = UUID.randomUUID();
    PortabilityJob job = mock(PortabilityJob.class);
    when(jobStore.findJob(jobId)).thenReturn(job);
    Collection<PhotoAlbum> albums = ImmutableList.of(new PhotoAlbum("id1", "Album 1", "This is a fake album"), new PhotoAlbum("id2", "", description1001));
    Collection<PhotoModel> photos = ImmutableList.of(new PhotoModel("pic1.jpg", server.url("/1.jpg").toString(), null, "image/jpeg", "p1", "id1", false, null), new PhotoModel("pic2.png", server.url("/2.png").toString(), "fine art", "image/png", "p2", "id1", false, null), new PhotoModel("pic3.jpg", server.url("/3.jpg").toString(), "A pic with EXIF", "image/jpeg", "p3", "id1", false, null), new PhotoModel("pic4.jpg", server.url("/4.jpg").toString(), "A pic with uploaded time", "image/jpeg", "p4", "id1", false, new SimpleDateFormat("yyyy:MM:dd HH:mm:ss").parse("2020:08:17 11:55:24")), new PhotoModel("pic5.jpg", server.url("/5.jpg").toString(), description1001, "image/jpeg", "p5", "id2", false, null));
    PhotosContainerResource resource = spy(new PhotosContainerResource(albums, photos));
    importer.importItem(jobId, executor, authData, resource);
    InOrder clientInOrder = Mockito.inOrder(client);
    verify(resource).transmogrify(any(KoofrTransmogrificationConfig.class));
    clientInOrder.verify(client).ensureRootFolder();
    clientInOrder.verify(client).ensureFolder("/root", "Album 1");
    clientInOrder.verify(client).addDescription("/root/Album 1", "This is a fake album");
    clientInOrder.verify(client).ensureFolder("/root", "Album");
    clientInOrder.verify(client).addDescription("/root/Album", description1000);
    clientInOrder.verify(client).fileExists(eq("/root/Album 1/pic1.jpg"));
    clientInOrder.verify(client).uploadFile(eq("/root/Album 1"), eq("pic1.jpg"), any(), eq("image/jpeg"), isNull(), isNull());
    clientInOrder.verify(client).fileExists(eq("/root/Album 1/pic2.png"));
    clientInOrder.verify(client).fileExists(eq("/root/Album 1/2020-08-03 11.55.24 pic3.jpg"));
    clientInOrder.verify(client).uploadFile(eq("/root/Album 1"), eq("2020-08-03 11.55.24 pic3.jpg"), any(), eq("image/jpeg"), eq(new SimpleDateFormat("yyyy:MM:dd HH:mm:ss").parse("2020:08:03 11:55:24")), eq("A pic with EXIF"));
    clientInOrder.verify(client).uploadFile(eq("/root/Album 1"), eq("2020-08-17 11.55.24 pic4.jpg"), any(), eq("image/jpeg"), eq(new SimpleDateFormat("yyyy:MM:dd HH:mm:ss").parse("2020:08:17 11:55:24")), eq("A pic with uploaded time"));
    clientInOrder.verify(client).fileExists(eq("/root/Album/pic5.jpg"));
    clientInOrder.verify(client).uploadFile(eq("/root/Album"), eq("pic5.jpg"), any(), eq("image/jpeg"), isNull(), eq(description1000));
    clientInOrder.verifyNoMoreInteractions();
}
Also used : Buffer(okio.Buffer) MockResponse(okhttp3.mockwebserver.MockResponse) InOrder(org.mockito.InOrder) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) PortabilityJob(org.datatransferproject.spi.cloud.types.PortabilityJob) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) KoofrTransmogrificationConfig(org.datatransferproject.transfer.koofr.KoofrTransmogrificationConfig) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) UUID(java.util.UUID) SimpleDateFormat(java.text.SimpleDateFormat) Test(org.junit.Test)

Example 45 with PhotosContainerResource

use of org.datatransferproject.types.common.models.photos.PhotosContainerResource in project data-transfer-project by google.

the class TwitterPhotosExporter method export.

@Override
public ExportResult<PhotosContainerResource> export(UUID jobId, TokenSecretAuthData authData, Optional<ExportInformation> exportInformation) {
    Twitter twitterApi = TwitterApiWrapper.getInstance(appCredentials, authData);
    int pageNumber = 1;
    if (exportInformation.isPresent()) {
        IntPaginationToken pageToken = (IntPaginationToken) exportInformation.get().getPaginationData();
        if (pageToken != null && pageToken.getStart() > 1) {
            pageNumber = pageToken.getStart();
        }
    }
    Paging paging = new Paging(pageNumber, PAGE_SIZE);
    try {
        String page = "" + pageNumber;
        long id = twitterApi.getId();
        monitor.debug(() -> format("Getting tweets for %s (page %s)", id, page));
        ResponseList<Status> statuses = twitterApi.getUserTimeline(id, paging);
        List<PhotoModel> photos = new ArrayList<>();
        for (Status status : statuses) {
            boolean hasMedia = status.getMediaEntities().length > 0;
            if (hasMedia && !status.isRetweet()) {
                for (MediaEntity mediaEntity : status.getMediaEntities()) {
                    photos.add(new PhotoModel("Twitter Photo " + mediaEntity.getId(), mediaEntity.getMediaURL(), status.getText(), null, Long.toString(status.getId()), null, false));
                }
            }
        }
        boolean moreData = statuses.size() == PAGE_SIZE;
        return new ExportResult<>(moreData ? ResultType.CONTINUE : ResultType.END, new PhotosContainerResource(null, photos), moreData ? new ContinuationData(new IntPaginationToken(pageNumber + 1)) : null);
    } catch (TwitterException e) {
        return new ExportResult<>(e);
    }
}
Also used : Status(twitter4j.Status) IntPaginationToken(org.datatransferproject.types.common.IntPaginationToken) Paging(twitter4j.Paging) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) ArrayList(java.util.ArrayList) Twitter(twitter4j.Twitter) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) TwitterException(twitter4j.TwitterException) MediaEntity(twitter4j.MediaEntity) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult)

Aggregations

PhotosContainerResource (org.datatransferproject.types.common.models.photos.PhotosContainerResource)57 PhotoAlbum (org.datatransferproject.types.common.models.photos.PhotoAlbum)37 Test (org.junit.Test)37 IdOnlyContainerResource (org.datatransferproject.types.common.models.IdOnlyContainerResource)29 PhotoModel (org.datatransferproject.types.common.models.photos.PhotoModel)29 ContinuationData (org.datatransferproject.spi.transfer.types.ContinuationData)27 ExportResult (org.datatransferproject.spi.transfer.provider.ExportResult)21 ArrayList (java.util.ArrayList)18 StringPaginationToken (org.datatransferproject.types.common.StringPaginationToken)18 UUID (java.util.UUID)14 PaginationData (org.datatransferproject.types.common.PaginationData)12 IOException (java.io.IOException)10 ExportInformation (org.datatransferproject.types.common.ExportInformation)9 ImportResult (org.datatransferproject.spi.transfer.provider.ImportResult)8 IntPaginationToken (org.datatransferproject.types.common.IntPaginationToken)8 InputStreamWrapper (org.datatransferproject.spi.cloud.storage.TemporaryPerJobDataStore.InputStreamWrapper)6 ContainerResource (org.datatransferproject.types.common.models.ContainerResource)6 TokensAndUrlAuthData (org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData)6 Photoset (com.flickr4java.flickr.photosets.Photoset)5 ImmutableList (com.google.common.collect.ImmutableList)5