Search in sources :

Example 91 with AmazonS3Client

use of com.talend.shaded.com.amazonaws.services.s3.AmazonS3Client in project herd by FINRAOS.

the class S3DaoTest method testGetAmazonS3AssertProxyIsSet.

@Test
public void testGetAmazonS3AssertProxyIsSet() {
    S3Operations originalS3Operations = (S3Operations) ReflectionTestUtils.getField(s3Dao, "s3Operations");
    S3Operations mockS3Operations = mock(S3Operations.class);
    ReflectionTestUtils.setField(s3Dao, "s3Operations", mockS3Operations);
    try {
        String s3BucketName = "s3BucketName";
        String s3KeyPrefix = "s3KeyPrefix";
        String httpProxyHost = "httpProxyHost";
        Integer httpProxyPort = 1234;
        S3FileTransferRequestParamsDto s3FileTransferRequestParamsDto = new S3FileTransferRequestParamsDto();
        s3FileTransferRequestParamsDto.setS3BucketName(s3BucketName);
        s3FileTransferRequestParamsDto.setS3KeyPrefix(s3KeyPrefix);
        s3FileTransferRequestParamsDto.setHttpProxyHost(httpProxyHost);
        s3FileTransferRequestParamsDto.setHttpProxyPort(httpProxyPort);
        when(mockS3Operations.putObject(any(), any())).then(new Answer<PutObjectResult>() {

            @Override
            public PutObjectResult answer(InvocationOnMock invocation) throws Throwable {
                AmazonS3Client amazonS3Client = invocation.getArgument(1);
                ClientConfiguration clientConfiguration = (ClientConfiguration) ReflectionTestUtils.getField(amazonS3Client, "clientConfiguration");
                assertEquals(httpProxyHost, clientConfiguration.getProxyHost());
                assertEquals(httpProxyPort.intValue(), clientConfiguration.getProxyPort());
                return new PutObjectResult();
            }
        });
        s3Dao.createDirectory(s3FileTransferRequestParamsDto);
    } finally {
        ReflectionTestUtils.setField(s3Dao, "s3Operations", originalS3Operations);
    }
}
Also used : AmazonS3Client(com.amazonaws.services.s3.AmazonS3Client) S3FileTransferRequestParamsDto(org.finra.herd.model.dto.S3FileTransferRequestParamsDto) PutObjectResult(com.amazonaws.services.s3.model.PutObjectResult) InvocationOnMock(org.mockito.invocation.InvocationOnMock) ClientConfiguration(com.amazonaws.ClientConfiguration) Test(org.junit.Test)

Example 92 with AmazonS3Client

use of com.talend.shaded.com.amazonaws.services.s3.AmazonS3Client in project herd by FINRAOS.

the class S3DaoTest method testGetAmazonS3AssertProxyIsNotSetWhenProxyPortIsNull.

@Test
public void testGetAmazonS3AssertProxyIsNotSetWhenProxyPortIsNull() {
    S3Operations originalS3Operations = (S3Operations) ReflectionTestUtils.getField(s3Dao, "s3Operations");
    S3Operations mockS3Operations = mock(S3Operations.class);
    ReflectionTestUtils.setField(s3Dao, "s3Operations", mockS3Operations);
    try {
        String s3BucketName = "s3BucketName";
        String s3KeyPrefix = "s3KeyPrefix";
        String httpProxyHost = "httpProxyHost";
        Integer httpProxyPort = null;
        S3FileTransferRequestParamsDto s3FileTransferRequestParamsDto = new S3FileTransferRequestParamsDto();
        s3FileTransferRequestParamsDto.setS3BucketName(s3BucketName);
        s3FileTransferRequestParamsDto.setS3KeyPrefix(s3KeyPrefix);
        s3FileTransferRequestParamsDto.setHttpProxyHost(httpProxyHost);
        s3FileTransferRequestParamsDto.setHttpProxyPort(httpProxyPort);
        when(mockS3Operations.putObject(any(), any())).then(new Answer<PutObjectResult>() {

            @Override
            public PutObjectResult answer(InvocationOnMock invocation) throws Throwable {
                AmazonS3Client amazonS3Client = invocation.getArgument(1);
                ClientConfiguration clientConfiguration = (ClientConfiguration) ReflectionTestUtils.getField(amazonS3Client, "clientConfiguration");
                assertNull(clientConfiguration.getProxyHost());
                return new PutObjectResult();
            }
        });
        s3Dao.createDirectory(s3FileTransferRequestParamsDto);
    } finally {
        ReflectionTestUtils.setField(s3Dao, "s3Operations", originalS3Operations);
    }
}
Also used : AmazonS3Client(com.amazonaws.services.s3.AmazonS3Client) S3FileTransferRequestParamsDto(org.finra.herd.model.dto.S3FileTransferRequestParamsDto) PutObjectResult(com.amazonaws.services.s3.model.PutObjectResult) InvocationOnMock(org.mockito.invocation.InvocationOnMock) ClientConfiguration(com.amazonaws.ClientConfiguration) Test(org.junit.Test)

Example 93 with AmazonS3Client

use of com.talend.shaded.com.amazonaws.services.s3.AmazonS3Client in project herd by FINRAOS.

the class S3DaoTest method testGetAWSCredentialsProviderAssertStaticCredentialsIsNotSetWhenAccessKeyIsNull.

@Test
public void testGetAWSCredentialsProviderAssertStaticCredentialsIsNotSetWhenAccessKeyIsNull() {
    S3Operations originalS3Operations = (S3Operations) ReflectionTestUtils.getField(s3Dao, "s3Operations");
    S3Operations mockS3Operations = mock(S3Operations.class);
    ReflectionTestUtils.setField(s3Dao, "s3Operations", mockS3Operations);
    try {
        String s3BucketName = "s3BucketName";
        String s3KeyPrefix = "s3KeyPrefix";
        String s3AccessKey = null;
        String s3SecretKey = "s3SecretKey";
        S3FileTransferRequestParamsDto s3FileTransferRequestParamsDto = new S3FileTransferRequestParamsDto();
        s3FileTransferRequestParamsDto.setS3BucketName(s3BucketName);
        s3FileTransferRequestParamsDto.setS3KeyPrefix(s3KeyPrefix);
        s3FileTransferRequestParamsDto.setAwsAccessKeyId(s3AccessKey);
        s3FileTransferRequestParamsDto.setAwsSecretKey(s3SecretKey);
        when(mockS3Operations.putObject(any(), any())).then(new Answer<PutObjectResult>() {

            @SuppressWarnings("unchecked")
            @Override
            public PutObjectResult answer(InvocationOnMock invocation) throws Throwable {
                AmazonS3Client amazonS3Client = invocation.getArgument(1);
                AWSCredentialsProviderChain awsCredentialsProviderChain = (AWSCredentialsProviderChain) ReflectionTestUtils.getField(amazonS3Client, "awsCredentialsProvider");
                List<AWSCredentialsProvider> credentialsProviders = (List<AWSCredentialsProvider>) ReflectionTestUtils.getField(awsCredentialsProviderChain, "credentialsProviders");
                assertEquals(1, credentialsProviders.size());
                assertEquals(DefaultAWSCredentialsProviderChain.class, credentialsProviders.get(0).getClass());
                return new PutObjectResult();
            }
        });
        s3Dao.createDirectory(s3FileTransferRequestParamsDto);
    } finally {
        ReflectionTestUtils.setField(s3Dao, "s3Operations", originalS3Operations);
    }
}
Also used : DefaultAWSCredentialsProviderChain(com.amazonaws.auth.DefaultAWSCredentialsProviderChain) S3FileTransferRequestParamsDto(org.finra.herd.model.dto.S3FileTransferRequestParamsDto) PutObjectResult(com.amazonaws.services.s3.model.PutObjectResult) DefaultAWSCredentialsProviderChain(com.amazonaws.auth.DefaultAWSCredentialsProviderChain) AWSCredentialsProviderChain(com.amazonaws.auth.AWSCredentialsProviderChain) AmazonS3Client(com.amazonaws.services.s3.AmazonS3Client) InvocationOnMock(org.mockito.invocation.InvocationOnMock) List(java.util.List) ArrayList(java.util.ArrayList) HerdAWSCredentialsProvider(org.finra.herd.model.dto.HerdAWSCredentialsProvider) AWSCredentialsProvider(com.amazonaws.auth.AWSCredentialsProvider) Test(org.junit.Test)

Example 94 with AmazonS3Client

use of com.talend.shaded.com.amazonaws.services.s3.AmazonS3Client in project herd by FINRAOS.

the class S3DaoTest method testGetAmazonS3AssertProxyIsNotSetWhenProxyHostIsBlank.

@Test
public void testGetAmazonS3AssertProxyIsNotSetWhenProxyHostIsBlank() {
    S3Operations originalS3Operations = (S3Operations) ReflectionTestUtils.getField(s3Dao, "s3Operations");
    S3Operations mockS3Operations = mock(S3Operations.class);
    ReflectionTestUtils.setField(s3Dao, "s3Operations", mockS3Operations);
    try {
        String s3BucketName = "s3BucketName";
        String s3KeyPrefix = "s3KeyPrefix";
        String httpProxyHost = "";
        Integer httpProxyPort = 1234;
        S3FileTransferRequestParamsDto s3FileTransferRequestParamsDto = new S3FileTransferRequestParamsDto();
        s3FileTransferRequestParamsDto.setS3BucketName(s3BucketName);
        s3FileTransferRequestParamsDto.setS3KeyPrefix(s3KeyPrefix);
        s3FileTransferRequestParamsDto.setHttpProxyHost(httpProxyHost);
        s3FileTransferRequestParamsDto.setHttpProxyPort(httpProxyPort);
        when(mockS3Operations.putObject(any(), any())).then(new Answer<PutObjectResult>() {

            @Override
            public PutObjectResult answer(InvocationOnMock invocation) throws Throwable {
                AmazonS3Client amazonS3Client = invocation.getArgument(1);
                ClientConfiguration clientConfiguration = (ClientConfiguration) ReflectionTestUtils.getField(amazonS3Client, "clientConfiguration");
                assertNull(clientConfiguration.getProxyHost());
                return new PutObjectResult();
            }
        });
        s3Dao.createDirectory(s3FileTransferRequestParamsDto);
    } finally {
        ReflectionTestUtils.setField(s3Dao, "s3Operations", originalS3Operations);
    }
}
Also used : AmazonS3Client(com.amazonaws.services.s3.AmazonS3Client) S3FileTransferRequestParamsDto(org.finra.herd.model.dto.S3FileTransferRequestParamsDto) PutObjectResult(com.amazonaws.services.s3.model.PutObjectResult) InvocationOnMock(org.mockito.invocation.InvocationOnMock) ClientConfiguration(com.amazonaws.ClientConfiguration) Test(org.junit.Test)

Example 95 with AmazonS3Client

use of com.talend.shaded.com.amazonaws.services.s3.AmazonS3Client in project photon-model by vmware.

the class AWSRemoteCleanup method cleanUpAWSS3.

@Test
public void cleanUpAWSS3() {
    if (this.isMock) {
        return;
    }
    List<Bucket> buckets = this.s3Clients.get(Regions.DEFAULT_REGION.getName()).listBuckets();
    for (Bucket bucket : buckets) {
        long bucketCreationTimeMicros = TimeUnit.MILLISECONDS.toMicros(bucket.getCreationDate().getTime());
        long timeDifference = Utils.getNowMicrosUtc() - bucketCreationTimeMicros;
        if (bucket.getName().contains(ENUMTEST_BUCKET) && timeDifference > TimeUnit.HOURS.toMicros(1) && !bucket.getName().contains(ENUMTEST_BUCKET_TAG)) {
            for (AmazonS3Client s3Client : this.s3Clients.values()) {
                try {
                    s3Client.deleteBucket(bucket.getName());
                    this.host.log(Level.INFO, "Deleting stale bucket %s", bucket.getName());
                } catch (Exception e) {
                    continue;
                }
            }
        }
    }
}
Also used : AmazonS3Client(com.amazonaws.services.s3.AmazonS3Client) Bucket(com.amazonaws.services.s3.model.Bucket) Test(org.junit.Test)

Aggregations

AmazonS3Client (com.amazonaws.services.s3.AmazonS3Client)107 Test (org.junit.Test)23 BasicAWSCredentials (com.amazonaws.auth.BasicAWSCredentials)20 AmazonClientException (com.amazonaws.AmazonClientException)16 ClientConfiguration (com.amazonaws.ClientConfiguration)15 ArrayList (java.util.ArrayList)13 HashMap (java.util.HashMap)13 AmazonS3 (com.amazonaws.services.s3.AmazonS3)12 File (java.io.File)12 InvocationOnMock (org.mockito.invocation.InvocationOnMock)12 PutObjectResult (com.amazonaws.services.s3.model.PutObjectResult)11 UploadPartRequest (com.amazonaws.services.s3.model.UploadPartRequest)11 AWSCredentials (com.amazonaws.auth.AWSCredentials)10 AWSCredentialsProvider (com.amazonaws.auth.AWSCredentialsProvider)10 ObjectMetadata (com.amazonaws.services.s3.model.ObjectMetadata)10 AmazonServiceException (com.amazonaws.AmazonServiceException)9 AmazonS3Exception (com.amazonaws.services.s3.model.AmazonS3Exception)9 InternalEvent (com.nextdoor.bender.InternalEvent)9 TestContext (com.nextdoor.bender.aws.TestContext)9 IOException (java.io.IOException)9