Search in sources :

Example 1 with PermissionDeniedException

use of org.datatransferproject.spi.transfer.types.PermissionDeniedException in project data-transfer-project by google.

the class GooglePhotosExporterTest method exportAlbumFirstSet.

@Test
public void exportAlbumFirstSet() throws IOException, InvalidTokenException, PermissionDeniedException {
    setUpSingleAlbum();
    when(albumListResponse.getNextPageToken()).thenReturn(ALBUM_TOKEN);
    // Run test
    ExportResult<PhotosContainerResource> result = googlePhotosExporter.exportAlbums(null, Optional.empty(), uuid);
    // Check results
    // Verify correct methods were called
    verify(photosInterface).listAlbums(Optional.empty());
    verify(albumListResponse).getAlbums();
    // Check pagination token
    ContinuationData continuationData = result.getContinuationData();
    StringPaginationToken paginationToken = (StringPaginationToken) continuationData.getPaginationData();
    assertThat(paginationToken.getToken()).isEqualTo(ALBUM_TOKEN_PREFIX + ALBUM_TOKEN);
    // Check albums field of container
    Collection<PhotoAlbum> actualAlbums = result.getExportedData().getAlbums();
    assertThat(actualAlbums.stream().map(PhotoAlbum::getId).collect(Collectors.toList())).containsExactly(ALBUM_ID);
    // Check photos field of container (should be empty, even though there is a photo in the
    // original album)
    Collection<PhotoModel> actualPhotos = result.getExportedData().getPhotos();
    assertThat(actualPhotos).isEmpty();
    // Should be one container in the resource list
    List<ContainerResource> actualResources = continuationData.getContainerResources();
    assertThat(actualResources.stream().map(a -> ((IdOnlyContainerResource) a).getId()).collect(Collectors.toList())).containsExactly(ALBUM_ID);
}
Also used : ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) ALBUM_TOKEN_PREFIX(org.datatransferproject.datatransfer.google.photos.GooglePhotosExporter.ALBUM_TOKEN_PREFIX) GoogleMediaItem(org.datatransferproject.datatransfer.google.mediaModels.GoogleMediaItem) ExportResult(org.datatransferproject.spi.transfer.provider.ExportResult) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) ArgumentMatchers.eq(org.mockito.ArgumentMatchers.eq) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PermissionDeniedException(org.datatransferproject.spi.transfer.types.PermissionDeniedException) Mockito.verifyNoInteractions(org.mockito.Mockito.verifyNoInteractions) AlbumListResponse(org.datatransferproject.datatransfer.google.mediaModels.AlbumListResponse) MediaMetadata(org.datatransferproject.datatransfer.google.mediaModels.MediaMetadata) TemporaryPerJobDataStore(org.datatransferproject.spi.cloud.storage.TemporaryPerJobDataStore) ArgumentCaptor(org.mockito.ArgumentCaptor) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) InputStreamWrapper(org.datatransferproject.spi.cloud.storage.TemporaryPerJobDataStore.InputStreamWrapper) GoogleCredentialFactory(org.datatransferproject.datatransfer.google.common.GoogleCredentialFactory) Before(org.junit.Before) JacksonFactory(com.google.api.client.json.jackson2.JacksonFactory) Collection(java.util.Collection) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) IOException(java.io.IOException) ContainerResource(org.datatransferproject.types.common.models.ContainerResource) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) UUID(java.util.UUID) PaginationData(org.datatransferproject.types.common.PaginationData) Truth.assertThat(com.google.common.truth.Truth.assertThat) Photo(org.datatransferproject.datatransfer.google.mediaModels.Photo) Collectors(java.util.stream.Collectors) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) Mockito.verify(org.mockito.Mockito.verify) PHOTO_TOKEN_PREFIX(org.datatransferproject.datatransfer.google.photos.GooglePhotosExporter.PHOTO_TOKEN_PREFIX) List(java.util.List) Assert.assertNull(org.junit.Assert.assertNull) Monitor(org.datatransferproject.api.launcher.Monitor) MediaItemSearchResponse(org.datatransferproject.datatransfer.google.mediaModels.MediaItemSearchResponse) TempPhotosData(org.datatransferproject.spi.transfer.types.TempPhotosData) Optional(java.util.Optional) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) GoogleAlbum(org.datatransferproject.datatransfer.google.mediaModels.GoogleAlbum) InvalidTokenException(org.datatransferproject.spi.transfer.types.InvalidTokenException) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) Mockito.mock(org.mockito.Mockito.mock) InputStream(java.io.InputStream) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) PhotosContainerResource(org.datatransferproject.types.common.models.photos.PhotosContainerResource) ContainerResource(org.datatransferproject.types.common.models.ContainerResource) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) PhotoModel(org.datatransferproject.types.common.models.photos.PhotoModel) IdOnlyContainerResource(org.datatransferproject.types.common.models.IdOnlyContainerResource) ContinuationData(org.datatransferproject.spi.transfer.types.ContinuationData) PhotoAlbum(org.datatransferproject.types.common.models.photos.PhotoAlbum) StringPaginationToken(org.datatransferproject.types.common.StringPaginationToken) Test(org.junit.Test)

Example 2 with PermissionDeniedException

use of org.datatransferproject.spi.transfer.types.PermissionDeniedException in project data-transfer-project by google.

the class MicrosoftPhotosImporter method createOneDriveFolder.

@SuppressWarnings("unchecked")
private String createOneDriveFolder(PhotoAlbum album) throws IOException, CopyExceptionWithFailureReason {
    Map<String, Object> rawFolder = new LinkedHashMap<>();
    // clean up album name for microsoft specifically
    // Note that PhotoAlbum.getName() can return an empty string or null depending
    // on the results of PhotoAlbum.cleanName(), e.g. if a Google Photos album has
    // title=" ", its cleaned name will be "". See PhotoAlbum.cleanName for further
    // details on what forms the name can take.
    String albumName = Strings.isNullOrEmpty(album.getName()) ? "Untitled" : album.getName();
    rawFolder.put("name", albumName);
    rawFolder.put("folder", new LinkedHashMap());
    rawFolder.put("@microsoft.graph.conflictBehavior", "rename");
    Request.Builder requestBuilder = new Request.Builder().url(createFolderUrl);
    requestBuilder.header("Authorization", "Bearer " + credential.getAccessToken());
    requestBuilder.post(RequestBody.create(MediaType.parse("application/json"), objectMapper.writeValueAsString(rawFolder)));
    try (Response response = client.newCall(requestBuilder.build()).execute()) {
        int code = response.code();
        ResponseBody body = response.body();
        if (code == 401) {
            // If there was an unauthorized error, then try refreshing the creds
            credentialFactory.refreshCredential(credential);
            monitor.info(() -> "Refreshed authorization token successfuly");
            requestBuilder.header("Authorization", "Bearer " + credential.getAccessToken());
            Response newResponse = client.newCall(requestBuilder.build()).execute();
            code = newResponse.code();
            body = newResponse.body();
        }
        if (code == 403 && response.message().contains("Access Denied")) {
            throw new PermissionDeniedException("User access to microsoft onedrive was denied", new IOException(String.format("Got error code %d  with message: %s", code, response.message())));
        } else if (code < 200 || code > 299) {
            throw new IOException("Got error code: " + code + " message: " + response.message() + " body: " + response.body().string());
        } else if (body == null) {
            throw new IOException("Got null body");
        }
        Map<String, Object> responseData = objectMapper.readValue(body.bytes(), Map.class);
        String folderId = (String) responseData.get("id");
        Preconditions.checkState(!Strings.isNullOrEmpty(folderId), "Expected id value to be present in %s", responseData);
        return folderId;
    }
}
Also used : Response(okhttp3.Response) Request(okhttp3.Request) PermissionDeniedException(org.datatransferproject.spi.transfer.types.PermissionDeniedException) IOException(java.io.IOException) LinkedHashMap(java.util.LinkedHashMap) ResponseBody(okhttp3.ResponseBody)

Example 3 with PermissionDeniedException

use of org.datatransferproject.spi.transfer.types.PermissionDeniedException in project data-transfer-project by google.

the class MicrosoftPhotosImporter method createUploadSession.

// Request an upload session to the OneDrive api so that we can upload chunks
// to the returned URL
private String createUploadSession(PhotoModel photo, IdempotentImportExecutor idempotentImportExecutor) throws IOException, CopyExceptionWithFailureReason {
    // Forming the URL to create an upload session
    String createSessionUrl;
    if (Strings.isNullOrEmpty(photo.getAlbumId())) {
        createSessionUrl = String.format(albumlessPhotoUrlTemplate, photo.getTitle(), UPLOAD_PARAMS);
    } else {
        String oneDriveFolderId = idempotentImportExecutor.getCachedValue(photo.getAlbumId());
        createSessionUrl = String.format(uploadPhotoUrlTemplate, oneDriveFolderId, photo.getTitle(), UPLOAD_PARAMS);
    }
    // create upload session
    // POST to /me/drive/items/{folder_id}:/{file_name}:/createUploadSession OR /me/drive/items/root:/Photos/{file_name}:/createUploadSession
    // get {uploadurl} from response
    Request.Builder createSessionRequestBuilder = new Request.Builder().url(createSessionUrl);
    // Auth headers
    createSessionRequestBuilder.header("Authorization", "Bearer " + credential.getAccessToken());
    createSessionRequestBuilder.header("Content-Type", "application/json");
    // Post request with empty body. If you don't include an empty body, you'll have problems
    createSessionRequestBuilder.post(RequestBody.create(MediaType.parse("application/json"), objectMapper.writeValueAsString(ImmutableMap.of())));
    // Make the call, we should get an upload url for photo data in a 200 response
    Response response = client.newCall(createSessionRequestBuilder.build()).execute();
    int code = response.code();
    ResponseBody responseBody = response.body();
    // If there was an unauthorized error, then try refreshing the creds
    if (code == 401) {
        this.credentialFactory.refreshCredential(credential);
        monitor.info(() -> "Refreshed authorization token successfuly");
        createSessionRequestBuilder.header("Authorization", "Bearer " + credential.getAccessToken());
        Response newResponse = client.newCall(createSessionRequestBuilder.build()).execute();
        code = newResponse.code();
        responseBody = newResponse.body();
    }
    if (code == 403 && response.message().contains("Access Denied")) {
        throw new PermissionDeniedException("User access to Microsoft One Drive was denied", new IOException(String.format("Got error code %d  with message: %s", code, response.message())));
    } else if (code == 507 && response.message().contains("Insufficient Storage")) {
        throw new DestinationMemoryFullException("Microsoft destination storage limit reached", new IOException(String.format("Got error code %d  with message: %s", code, response.message())));
    } else if (code < 200 || code > 299) {
        throw new IOException(String.format("Got error code: %s\n" + "message: %s\n" + "body: %s\n" + "request url: %s\n" + "bearer token: %s\n" + // For debugging 404s on upload
        " photo: %s\n", code, response.message(), response.body().string(), createSessionUrl, credential.getAccessToken(), photo));
    } else if (code != 200) {
        monitor.info(() -> String.format("Got an unexpected non-200, non-error response code"));
    }
    // make sure we have a non-null response body
    Preconditions.checkState(responseBody != null, "Got Null Body when creating photo upload session %s", photo);
    // convert to a map
    final Map<String, Object> responseData = objectMapper.readValue(responseBody.bytes(), Map.class);
    // return the session's upload url
    Preconditions.checkState(responseData.containsKey("uploadUrl"), "No uploadUrl :(");
    return (String) responseData.get("uploadUrl");
}
Also used : Response(okhttp3.Response) DestinationMemoryFullException(org.datatransferproject.spi.transfer.types.DestinationMemoryFullException) Request(okhttp3.Request) PermissionDeniedException(org.datatransferproject.spi.transfer.types.PermissionDeniedException) IOException(java.io.IOException) ResponseBody(okhttp3.ResponseBody)

Aggregations

IOException (java.io.IOException)3 PermissionDeniedException (org.datatransferproject.spi.transfer.types.PermissionDeniedException)3 Request (okhttp3.Request)2 Response (okhttp3.Response)2 ResponseBody (okhttp3.ResponseBody)2 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1 JacksonFactory (com.google.api.client.json.jackson2.JacksonFactory)1 Truth.assertThat (com.google.common.truth.Truth.assertThat)1 InputStream (java.io.InputStream)1 Collection (java.util.Collection)1 LinkedHashMap (java.util.LinkedHashMap)1 List (java.util.List)1 Optional (java.util.Optional)1 UUID (java.util.UUID)1 Collectors (java.util.stream.Collectors)1 Monitor (org.datatransferproject.api.launcher.Monitor)1 GoogleCredentialFactory (org.datatransferproject.datatransfer.google.common.GoogleCredentialFactory)1 AlbumListResponse (org.datatransferproject.datatransfer.google.mediaModels.AlbumListResponse)1 GoogleAlbum (org.datatransferproject.datatransfer.google.mediaModels.GoogleAlbum)1 GoogleMediaItem (org.datatransferproject.datatransfer.google.mediaModels.GoogleMediaItem)1