Search in sources :

Example 76 with AccessToken

use of org.glassfish.jersey.client.oauth1.AccessToken in project data-transfer-project by google.

the class GoogleVideosImporter method importItem.

@Override
public ImportResult importItem(UUID jobId, IdempotentImportExecutor executor, TokensAndUrlAuthData authData, VideosContainerResource data) throws Exception {
    if (data == null) {
        // Nothing to do
        return ImportResult.OK;
    }
    PhotosLibraryClient client;
    if (clientsMap.containsKey(jobId)) {
        client = clientsMap.get(jobId);
    } else {
        PhotosLibrarySettings settings = PhotosLibrarySettings.newBuilder().setCredentialsProvider(FixedCredentialsProvider.create(UserCredentials.newBuilder().setClientId(appCredentials.getKey()).setClientSecret(appCredentials.getSecret()).setAccessToken(new AccessToken(authData.getAccessToken(), new Date())).setRefreshToken(authData.getRefreshToken()).build())).build();
        client = PhotosLibraryClient.initialize(settings);
        clientsMap.put(jobId, client);
    }
    long bytes = 0L;
    // Uploads videos
    final Collection<VideoModel> videos = data.getVideos();
    if (videos != null && videos.size() > 0) {
        Stream<VideoModel> stream = videos.stream().filter(video -> shouldImport(video, executor)).map(this::transformVideoName);
        // We partition into groups of 49 as 50 is the maximum number of items that can be created in
        // one call. (We use 49 to avoid potential off by one errors)
        // https://developers.google.com/photos/library/guides/upload-media#creating-media-item
        final UnmodifiableIterator<List<VideoModel>> batches = Iterators.partition(stream.iterator(), 49);
        while (batches.hasNext()) {
            long batchBytes = importVideoBatch(batches.next(), client, executor);
            bytes += batchBytes;
        }
    }
    final ImportResult result = ImportResult.OK;
    return result.copyWithBytes(bytes);
}
Also used : RandomAccessFile(java.io.RandomAccessFile) UnauthenticatedException(com.google.api.gax.rpc.UnauthenticatedException) Error(com.google.photos.library.v1.upload.UploadMediaItemResponse.Error) Date(java.util.Date) PhotosLibrarySettings(com.google.photos.library.v1.PhotosLibrarySettings) FixedCredentialsProvider(com.google.api.gax.core.FixedCredentialsProvider) AppCredentials(org.datatransferproject.types.transfer.auth.AppCredentials) Pair(org.apache.commons.lang3.tuple.Pair) Map(java.util.Map) UploadMediaItemRequest(com.google.photos.library.v1.upload.UploadMediaItemRequest) ImageStreamProvider(org.datatransferproject.transfer.ImageStreamProvider) IdempotentImportExecutor(org.datatransferproject.spi.transfer.idempotentexecutor.IdempotentImportExecutor) UploadErrorException(org.datatransferproject.spi.transfer.types.UploadErrorException) Collection(java.util.Collection) Status(com.google.rpc.Status) UUID(java.util.UUID) FileNotFoundException(java.io.FileNotFoundException) VideosContainerResource(org.datatransferproject.types.common.models.videos.VideosContainerResource) BatchCreateMediaItemsResponse(com.google.photos.library.v1.proto.BatchCreateMediaItemsResponse) List(java.util.List) Stream(java.util.stream.Stream) Monitor(org.datatransferproject.api.launcher.Monitor) TokensAndUrlAuthData(org.datatransferproject.types.transfer.auth.TokensAndUrlAuthData) VideoModel(org.datatransferproject.types.common.models.videos.VideoModel) UploadMediaItemResponse(com.google.photos.library.v1.upload.UploadMediaItemResponse) DestinationMemoryFullException(org.datatransferproject.spi.transfer.types.DestinationMemoryFullException) ImportResult(org.datatransferproject.spi.transfer.provider.ImportResult) HashMap(java.util.HashMap) Iterators(com.google.common.collect.Iterators) ArrayList(java.util.ArrayList) TemporaryPerJobDataStore(org.datatransferproject.spi.cloud.storage.TemporaryPerJobDataStore) Strings(com.google.common.base.Strings) NewMediaItem(com.google.photos.library.v1.proto.NewMediaItem) Importer(org.datatransferproject.spi.transfer.provider.Importer) NewMediaItemFactory(com.google.photos.library.v1.util.NewMediaItemFactory) Code(com.google.rpc.Code) PhotosLibraryClient(com.google.photos.library.v1.PhotosLibraryClient) MediaObject(org.datatransferproject.types.common.models.MediaObject) IOException(java.io.IOException) NewMediaItemResult(com.google.photos.library.v1.proto.NewMediaItemResult) UserCredentials(com.google.auth.oauth2.UserCredentials) File(java.io.File) InvalidArgumentException(com.google.api.gax.rpc.InvalidArgumentException) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) UnmodifiableIterator(com.google.common.collect.UnmodifiableIterator) InvalidTokenException(org.datatransferproject.spi.transfer.types.InvalidTokenException) AccessToken(com.google.auth.oauth2.AccessToken) InputStream(java.io.InputStream) ImportResult(org.datatransferproject.spi.transfer.provider.ImportResult) AccessToken(com.google.auth.oauth2.AccessToken) PhotosLibrarySettings(com.google.photos.library.v1.PhotosLibrarySettings) List(java.util.List) ArrayList(java.util.ArrayList) PhotosLibraryClient(com.google.photos.library.v1.PhotosLibraryClient) VideoModel(org.datatransferproject.types.common.models.videos.VideoModel) Date(java.util.Date)

Example 77 with AccessToken

use of org.glassfish.jersey.client.oauth1.AccessToken in project cdap by caskdata.

the class GCPRemoteAuthenticatorTest method testRemoteAuthenticatorRefreshesExpiredAccessToken.

@Test
public void testRemoteAuthenticatorRefreshesExpiredAccessToken() throws Exception {
    String expiredAccessTokenValue = "expired-access-token";
    String accessTokenValue = "access-token";
    // This is just an arbitrary fixed point in time.
    Instant fixedInstant = Instant.ofEpochSecond(1646358109);
    Clock fixedClock = Clock.fixed(fixedInstant, ZoneId.systemDefault());
    GoogleCredentials mockGoogleCredentials = mock(GoogleCredentials.class);
    AccessToken expiredAccessToken = new AccessToken(expiredAccessTokenValue, Date.from(fixedInstant.minus(Duration.ofHours(1))));
    AccessToken accessToken = new AccessToken(accessTokenValue, Date.from(fixedInstant.plus(Duration.ofHours(1))));
    when(mockGoogleCredentials.refreshAccessToken()).thenReturn(accessToken);
    GCPRemoteAuthenticator gcpRemoteAuthenticator = new GCPRemoteAuthenticator(mockGoogleCredentials, fixedClock, expiredAccessToken);
    // Verify expected credential value and that refresh was called exactly once.
    Credential credential = gcpRemoteAuthenticator.getCredentials();
    Assert.assertEquals(accessTokenValue, credential.getValue());
    verify(mockGoogleCredentials, times(1)).refreshAccessToken();
}
Also used : Credential(io.cdap.cdap.proto.security.Credential) AccessToken(com.google.auth.oauth2.AccessToken) Instant(java.time.Instant) GoogleCredentials(com.google.auth.oauth2.GoogleCredentials) Clock(java.time.Clock) Test(org.junit.Test)

Example 78 with AccessToken

use of org.glassfish.jersey.client.oauth1.AccessToken in project cdap by caskdata.

the class GCPRemoteAuthenticatorTest method testRemoteAuthenticatorReturnsValidAccessToken.

@Test
public void testRemoteAuthenticatorReturnsValidAccessToken() throws Exception {
    String accessTokenValue = "access-token";
    // This is just an arbitrary fixed point in time.
    Instant fixedInstant = Instant.ofEpochSecond(1646358109);
    Clock fixedClock = Clock.fixed(fixedInstant, ZoneId.systemDefault());
    GoogleCredentials mockGoogleCredentials = mock(GoogleCredentials.class);
    AccessToken accessToken = new AccessToken(accessTokenValue, Date.from(fixedInstant.plus(Duration.ofHours(1))));
    when(mockGoogleCredentials.refreshAccessToken()).thenReturn(accessToken);
    GCPRemoteAuthenticator gcpRemoteAuthenticator = new GCPRemoteAuthenticator(mockGoogleCredentials, fixedClock, accessToken);
    // Verify expected credential value and that refresh was not called.
    Credential credential = gcpRemoteAuthenticator.getCredentials();
    Assert.assertEquals(accessTokenValue, credential.getValue());
    verify(mockGoogleCredentials, times(0)).refreshAccessToken();
}
Also used : Credential(io.cdap.cdap.proto.security.Credential) AccessToken(com.google.auth.oauth2.AccessToken) Instant(java.time.Instant) GoogleCredentials(com.google.auth.oauth2.GoogleCredentials) Clock(java.time.Clock) Test(org.junit.Test)

Example 79 with AccessToken

use of org.glassfish.jersey.client.oauth1.AccessToken in project terra-workspace-manager by DataBiosphere.

the class EnablePet method doUserJourney.

@Override
protected void doUserJourney(TestUserSpecification testUser, WorkspaceApi userWorkspaceApi) throws Exception {
    // Validate that the user cannot impersonate their pet before calling this endpoint.
    GoogleApi samGoogleApi = SamClientUtils.samGoogleApi(testUser, server);
    String petSaEmail = SamRetry.retry(() -> samGoogleApi.getPetServiceAccount(projectId));
    Iam userIamClient = ClientTestUtils.getGcpIamClient(testUser);
    assertFalse(canImpersonateSa(userIamClient, petSaEmail));
    userWorkspaceApi.enablePet(getWorkspaceId());
    assertTrue(canImpersonateSa(userIamClient, petSaEmail));
    // Validate that calling this endpoint as the pet does not grant the pet permission to
    // impersonate itself.
    String rawPetSaToken = SamRetry.retry(() -> samGoogleApi.getPetServiceAccountToken(projectId, ClientTestUtils.TEST_USER_SCOPES));
    AccessToken petSaToken = new AccessToken(rawPetSaToken, null);
    WorkspaceApi petSaWorkspaceApi = ClientTestUtils.getWorkspaceClientFromToken(petSaToken, server);
    petSaWorkspaceApi.enablePet(getWorkspaceId());
    // Add second user to the workspace as a reader.
    userWorkspaceApi.grantRole(new GrantRoleRequestBody().memberEmail(secondUser.userEmail), getWorkspaceId(), IamRole.READER);
    // Validate the second user cannot impersonate either user's pet.
    GoogleApi secondUserSamGoogleApi = SamClientUtils.samGoogleApi(secondUser, server);
    String secondUserPetSaEmail = SamRetry.retry(() -> secondUserSamGoogleApi.getPetServiceAccount(projectId));
    Iam secondUserIamClient = ClientTestUtils.getGcpIamClient(secondUser);
    assertFalse(canImpersonateSa(secondUserIamClient, secondUserPetSaEmail));
    assertFalse(canImpersonateSa(secondUserIamClient, petSaEmail));
    // Enable the second user to impersonate their pet
    WorkspaceApi secondUserWorkspaceApi = ClientTestUtils.getWorkspaceClient(secondUser, server);
    secondUserWorkspaceApi.enablePet(getWorkspaceId());
    assertTrue(canImpersonateSa(secondUserIamClient, secondUserPetSaEmail));
    // Second user still cannot impersonate first user's pet
    assertFalse(canImpersonateSa(secondUserIamClient, petSaEmail));
    // Remove second user from workspace. This should revoke their permission to impersonate their
    // pet.
    userWorkspaceApi.removeRole(getWorkspaceId(), IamRole.READER, secondUser.userEmail);
    assertTrue(ClientTestUtils.getWithRetryOnException(() -> assertCannotImpersonateSa(secondUserIamClient, secondUserPetSaEmail)));
}
Also used : GrantRoleRequestBody(bio.terra.workspace.model.GrantRoleRequestBody) Iam(com.google.api.services.iam.v1.Iam) GoogleApi(org.broadinstitute.dsde.workbench.client.sam.api.GoogleApi) WorkspaceApi(bio.terra.workspace.api.WorkspaceApi) AccessToken(com.google.auth.oauth2.AccessToken)

Example 80 with AccessToken

use of org.glassfish.jersey.client.oauth1.AccessToken in project terra-workspace-manager by DataBiosphere.

the class BufferServiceConfiguration method getAccessToken.

public String getAccessToken() throws IOException {
    try (FileInputStream fileInputStream = new FileInputStream(clientCredentialFilePath)) {
        GoogleCredentials credentials = ServiceAccountCredentials.fromStream(fileInputStream).createScoped(BUFFER_SCOPES);
        AccessToken token = credentials.refreshAccessToken();
        return token.getTokenValue();
    }
}
Also used : AccessToken(com.google.auth.oauth2.AccessToken) GoogleCredentials(com.google.auth.oauth2.GoogleCredentials) FileInputStream(java.io.FileInputStream)

Aggregations

AccessToken (com.google.auth.oauth2.AccessToken)78 Test (org.junit.Test)44 GoogleCredentials (com.google.auth.oauth2.GoogleCredentials)33 Date (java.util.Date)23 IOException (java.io.IOException)20 AccessToken (io.vertx.ext.auth.oauth2.AccessToken)16 Instant (java.time.Instant)10 Client (javax.ws.rs.client.Client)10 AccessToken (org.glassfish.jersey.client.oauth1.AccessToken)10 ConsumerCredentials (org.glassfish.jersey.client.oauth1.ConsumerCredentials)10 JsonObject (io.vertx.core.json.JsonObject)9 URI (java.net.URI)9 Feature (javax.ws.rs.core.Feature)8 JerseyTest (org.glassfish.jersey.test.JerseyTest)8 MockHttpTransport (com.google.api.client.testing.http.MockHttpTransport)6 ServiceAccountCredentials (com.google.auth.oauth2.ServiceAccountCredentials)6 Credential (io.cdap.cdap.proto.security.Credential)6 InputStreamReader (java.io.InputStreamReader)6 Clock (java.time.Clock)6 WebTarget (javax.ws.rs.client.WebTarget)6