Search in sources :

Example 1 with PhotoModel

use of org.dataportabilityproject.types.transfer.models.photos.PhotoModel in project data-transfer-project by google.

the class GooglePhotosExporter method exportPhotos.

private ExportResult<PhotosContainerResource> exportPhotos(TokensAndUrlAuthData authData, String albumId, Optional<PaginationData> paginationData) {
    try {
        int startItem = 1;
        if (paginationData.isPresent()) {
            String token = ((StringPaginationToken) paginationData.get()).getToken();
            Preconditions.checkArgument(token.startsWith(PHOTO_TOKEN_PREFIX), "Invalid pagination token " + token);
            startItem = Integer.parseInt(token.substring(PHOTO_TOKEN_PREFIX.length()));
        }
        URL photosUrl = new URL(String.format(URL_PHOTO_FEED_FORMAT, albumId, startItem, MAX_RESULTS));
        AlbumFeed photoFeed = getOrCreatePhotosService(authData).getFeed(photosUrl, AlbumFeed.class);
        PaginationData nextPageData = null;
        if (photoFeed.getEntries().size() == MAX_RESULTS) {
            int nextPageStart = startItem + MAX_RESULTS;
            nextPageData = new StringPaginationToken(PHOTO_TOKEN_PREFIX + nextPageStart);
        }
        ContinuationData continuationData = new ContinuationData(nextPageData);
        List<PhotoModel> photos = new ArrayList<>(photoFeed.getEntries().size());
        for (GphotoEntry photo : photoFeed.getEntries()) {
            MediaContent mediaContent = (MediaContent) photo.getContent();
            photos.add(new PhotoModel(photo.getTitle().getPlainText(), mediaContent.getUri(), photo.getDescription().getPlainText(), mediaContent.getMimeType().getMediaType(), albumId));
        }
        PhotosContainerResource containerResource = new PhotosContainerResource(null, photos);
        ResultType resultType = ResultType.CONTINUE;
        if (nextPageData == null) {
            resultType = ResultType.END;
        }
        return new ExportResult<>(resultType, containerResource, continuationData);
    } catch (ServiceException | IOException e) {
        return new ExportResult<>(ResultType.ERROR, e.getMessage());
    }
}
Also used : PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) AlbumFeed(com.google.gdata.data.photos.AlbumFeed) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) ArrayList(java.util.ArrayList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) IOException(java.io.IOException) URL(java.net.URL) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) ServiceException(com.google.gdata.util.ServiceException) MediaContent(com.google.gdata.data.MediaContent) GphotoEntry(com.google.gdata.data.photos.GphotoEntry) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 2 with PhotoModel

use of org.dataportabilityproject.types.transfer.models.photos.PhotoModel in project data-transfer-project by google.

the class SmugMugPhotosExporter method exportPhotos.

private ExportResult<PhotosContainerResource> exportPhotos(IdOnlyContainerResource containerResource, Optional<PaginationData> paginationData) throws IOException {
    List<PhotoModel> photoList = new ArrayList<>();
    // Make request to SmugMug
    String photoInfoUri;
    if (paginationData.isPresent()) {
        String token = ((StringPaginationToken) paginationData.get()).getToken();
        Preconditions.checkState(token.startsWith(PHOTO_TOKEN_PREFIX), "Invalid pagination token " + token);
        photoInfoUri = token.substring(PHOTO_TOKEN_PREFIX.length());
    } else {
        String id = containerResource.getId();
        photoInfoUri = String.format(ALBUM_URL_FORMATTER, id);
    }
    SmugMugResponse<SmugMugAlbumInfoResponse> albumInfoResponse = smugMugInterface.makeAlbumInfoRequest(photoInfoUri);
    // Set up continuation data
    StringPaginationToken pageToken = null;
    if (albumInfoResponse.getResponse().getPageInfo().getNextPage() != null) {
        pageToken = new StringPaginationToken(PHOTO_TOKEN_PREFIX + albumInfoResponse.getResponse().getPageInfo().getNextPage());
    }
    ContinuationData continuationData = new ContinuationData(pageToken);
    // Make list of photos
    for (SmugMugAlbumImage image : albumInfoResponse.getResponse().getImages()) {
        String title = image.getTitle();
        if (Strings.isNullOrEmpty(title)) {
            title = image.getFileName();
        }
        // TODO(olsona): this.authConsumer.sign(image.getArchivedUri()) ?
        photoList.add(new PhotoModel(title, image.getArchivedUri(), image.getCaption(), image.getFormat(), containerResource.getId()));
    }
    PhotosContainerResource resource = new PhotosContainerResource(null, photoList);
    // Get result type
    ResultType resultType = ResultType.CONTINUE;
    if (pageToken == null) {
        resultType = ResultType.END;
    }
    return new ExportResult<>(resultType, resource, continuationData);
}
Also used : PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) SmugMugAlbumImage(org.dataportabilityproject.transfer.smugmug.photos.model.SmugMugAlbumImage) ArrayList(java.util.ArrayList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) StringPaginationToken(org.dataportabilityproject.spi.transfer.types.StringPaginationToken) SmugMugAlbumInfoResponse(org.dataportabilityproject.transfer.smugmug.photos.model.SmugMugAlbumInfoResponse) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 3 with PhotoModel

use of org.dataportabilityproject.types.transfer.models.photos.PhotoModel in project data-transfer-project by google.

the class SmugMugPhotosImporter method importItem.

@Override
public ImportResult importItem(UUID jobId, AuthData authData, PhotosContainerResource data) {
    try {
        String folder = null;
        if (!data.getAlbums().isEmpty()) {
            SmugMugResponse<SmugMugUserResponse> userResponse = smugMugInterface.makeUserRequest(smugMugInterface.USER_URL);
            folder = userResponse.getResponse().getUser().getUris().get("Folder").getUri();
        }
        for (PhotoAlbum album : data.getAlbums()) {
            importSingleAlbum(jobId, folder, album);
        }
        for (PhotoModel photo : data.getPhotos()) {
            importSinglePhoto(jobId, photo);
        }
    } catch (IOException e) {
        // TODO(olsona): we should retry on individual errors
        return new ImportResult(ResultType.ERROR, e.getMessage());
    }
    return ImportResult.OK;
}
Also used : SmugMugUserResponse(org.dataportabilityproject.transfer.smugmug.photos.model.SmugMugUserResponse) ImportResult(org.dataportabilityproject.spi.transfer.provider.ImportResult) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) PhotoAlbum(org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum) IOException(java.io.IOException)

Example 4 with PhotoModel

use of org.dataportabilityproject.types.transfer.models.photos.PhotoModel in project data-transfer-project by google.

the class FlickrPhotosExporter method getPhotos.

private ExportResult<PhotosContainerResource> getPhotos(IdOnlyContainerResource resource, PaginationData paginationData) {
    String photoSetId = resource.getId();
    int page = paginationData == null ? 1 : ((IntPaginationToken) paginationData).getStart();
    PhotoList<Photo> photoSetList;
    try {
        if (photoSetId == null) {
            RequestContext.getRequestContext().setExtras(EXTRAS);
            photoSetList = photosInterface.getNotInSet(PHOTO_PER_PAGE, page);
            RequestContext.getRequestContext().setExtras(ImmutableList.of());
        } else {
            photoSetList = photosetsInterface.getPhotos(photoSetId, ImmutableSet.copyOf(EXTRAS), 0, PHOTO_PER_PAGE, page);
        }
    } catch (FlickrException e) {
        return new ExportResult<>(ResultType.ERROR, "Error exporting Flickr photo: " + e.getErrorMessage());
    }
    boolean hasMore = photoSetList.getPage() != photoSetList.getPages() && !photoSetList.isEmpty();
    Collection<PhotoModel> photos = photoSetList.stream().map(p -> toCommonPhoto(p, photoSetId)).collect(Collectors.toList());
    PaginationData newPage = null;
    if (hasMore) {
        newPage = new IntPaginationToken(page + 1);
    }
    // Get result type
    ResultType resultType = ResultType.CONTINUE;
    if (newPage == null) {
        resultType = ResultType.END;
    }
    PhotosContainerResource photosContainerResource = new PhotosContainerResource(null, photos);
    return new ExportResult<>(resultType, photosContainerResource, new ContinuationData(newPage));
}
Also used : IdOnlyContainerResource(org.dataportabilityproject.spi.transfer.types.IdOnlyContainerResource) PhotoAlbum(org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum) FlickrException(com.flickr4java.flickr.FlickrException) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult) LoggerFactory(org.slf4j.LoggerFactory) IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) Photoset(com.flickr4java.flickr.photosets.Photoset) REST(com.flickr4java.flickr.REST) ArrayList(java.util.ArrayList) Photo(com.flickr4java.flickr.photos.Photo) Strings(com.google.common.base.Strings) ImmutableList(com.google.common.collect.ImmutableList) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) AuthData(org.dataportabilityproject.types.transfer.auth.AuthData) Flickr(com.flickr4java.flickr.Flickr) PhotoList(com.flickr4java.flickr.photos.PhotoList) Auth(com.flickr4java.flickr.auth.Auth) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) ExportInformation(org.dataportabilityproject.spi.transfer.types.ExportInformation) ImmutableSet(com.google.common.collect.ImmutableSet) Exporter(org.dataportabilityproject.spi.transfer.provider.Exporter) Logger(org.slf4j.Logger) PhotosInterface(com.flickr4java.flickr.photos.PhotosInterface) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) Collection(java.util.Collection) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) Photosets(com.flickr4java.flickr.photosets.Photosets) List(java.util.List) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) Preconditions(com.google.common.base.Preconditions) AppCredentials(org.dataportabilityproject.types.transfer.auth.AppCredentials) VisibleForTesting(com.google.common.annotations.VisibleForTesting) RequestContext(com.flickr4java.flickr.RequestContext) PhotosetsInterface(com.flickr4java.flickr.photosets.PhotosetsInterface) IntPaginationToken(org.dataportabilityproject.spi.transfer.types.IntPaginationToken) PaginationData(org.dataportabilityproject.spi.transfer.types.PaginationData) FlickrException(com.flickr4java.flickr.FlickrException) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) Photo(com.flickr4java.flickr.photos.Photo) ContinuationData(org.dataportabilityproject.spi.transfer.types.ContinuationData) ResultType(org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType) PhotosContainerResource(org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource) ExportResult(org.dataportabilityproject.spi.transfer.provider.ExportResult)

Example 5 with PhotoModel

use of org.dataportabilityproject.types.transfer.models.photos.PhotoModel in project data-transfer-project by google.

the class FlickrPhotosImporter method importItem.

@Override
public ImportResult importItem(UUID jobId, AuthData authData, PhotosContainerResource data) {
    Auth auth;
    try {
        auth = FlickrUtils.getAuth(authData, flickr);
    } catch (FlickrException e) {
        return new ImportResult(ImportResult.ResultType.ERROR, "Error authorizing Flickr Auth: " + e.getErrorMessage());
    }
    RequestContext.getRequestContext().setAuth(auth);
    // Store any album data in the cache because Flickr only allows you to create an album with a
    // photo in it, so we have to wait for the first photo to create the album
    TempPhotosData tempPhotosData = jobStore.findData(TempPhotosData.class, jobId);
    if (tempPhotosData == null) {
        tempPhotosData = new TempPhotosData(jobId);
        jobStore.create(jobId, tempPhotosData);
    }
    for (PhotoAlbum album : data.getAlbums()) {
        tempPhotosData.addAlbum(CACHE_ALBUM_METADATA_PREFIX + album.getId(), album);
    }
    jobStore.update(jobId, tempPhotosData);
    for (PhotoModel photo : data.getPhotos()) {
        try {
            String photoId = uploadPhoto(photo);
            String oldAlbumId = photo.getAlbumId();
            TempPhotosData tempData = jobStore.findData(TempPhotosData.class, jobId);
            String newAlbumId = tempData.lookupNewAlbumId(oldAlbumId);
            if (Strings.isNullOrEmpty(newAlbumId)) {
                // This means that we havent created the new album yet, create the photoset
                PhotoAlbum album = tempData.lookupAlbum(CACHE_ALBUM_METADATA_PREFIX + oldAlbumId);
                Photoset photoset = photosetsInterface.create(COPY_PREFIX + album.getName(), album.getDescription(), photoId);
                tempData.addAlbumId(oldAlbumId, photoset.getId());
            } else {
                // We've already created a new album, add the photo to the new album
                photosetsInterface.addPhoto(newAlbumId, photoId);
            }
            jobStore.update(jobId, tempData);
        } catch (FlickrException | IOException e) {
            // TODO: figure out retries
            return new ImportResult(ImportResult.ResultType.ERROR, "Error importing item: " + e.getMessage());
        }
    }
    return new ImportResult(ImportResult.ResultType.OK);
}
Also used : TempPhotosData(org.dataportabilityproject.spi.transfer.types.TempPhotosData) ImportResult(org.dataportabilityproject.spi.transfer.provider.ImportResult) FlickrException(com.flickr4java.flickr.FlickrException) Photoset(com.flickr4java.flickr.photosets.Photoset) Auth(com.flickr4java.flickr.auth.Auth) PhotoModel(org.dataportabilityproject.types.transfer.models.photos.PhotoModel) PhotoAlbum(org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum) IOException(java.io.IOException)

Aggregations

PhotoModel (org.dataportabilityproject.types.transfer.models.photos.PhotoModel)6 IOException (java.io.IOException)3 ArrayList (java.util.ArrayList)3 ExportResult (org.dataportabilityproject.spi.transfer.provider.ExportResult)3 ResultType (org.dataportabilityproject.spi.transfer.provider.ExportResult.ResultType)3 ContinuationData (org.dataportabilityproject.spi.transfer.types.ContinuationData)3 PhotoAlbum (org.dataportabilityproject.types.transfer.models.photos.PhotoAlbum)3 PhotosContainerResource (org.dataportabilityproject.types.transfer.models.photos.PhotosContainerResource)3 FlickrException (com.flickr4java.flickr.FlickrException)2 Auth (com.flickr4java.flickr.auth.Auth)2 Photo (com.flickr4java.flickr.photos.Photo)2 Photoset (com.flickr4java.flickr.photosets.Photoset)2 ImportResult (org.dataportabilityproject.spi.transfer.provider.ImportResult)2 PaginationData (org.dataportabilityproject.spi.transfer.types.PaginationData)2 StringPaginationToken (org.dataportabilityproject.spi.transfer.types.StringPaginationToken)2 Flickr (com.flickr4java.flickr.Flickr)1 REST (com.flickr4java.flickr.REST)1 RequestContext (com.flickr4java.flickr.RequestContext)1 PhotoList (com.flickr4java.flickr.photos.PhotoList)1 PhotosInterface (com.flickr4java.flickr.photos.PhotosInterface)1