Search in sources :

Example 1 with Grant

use of software.amazon.awssdk.services.s3.model.Grant in project druid by druid-io.

the class S3DataSegmentPusherTest method testPushInternal.

private void testPushInternal(boolean useUniquePath, String matcher) throws Exception {
    ServerSideEncryptingAmazonS3 s3Client = EasyMock.createStrictMock(ServerSideEncryptingAmazonS3.class);
    final AccessControlList acl = new AccessControlList();
    acl.setOwner(new Owner("ownerId", "owner"));
    acl.grantAllPermissions(new Grant(new CanonicalGrantee(acl.getOwner().getId()), Permission.FullControl));
    EasyMock.expect(s3Client.getBucketAcl(EasyMock.eq("bucket"))).andReturn(acl).once();
    EasyMock.expect(s3Client.putObject(EasyMock.anyObject())).andReturn(new PutObjectResult()).once();
    EasyMock.replay(s3Client);
    S3DataSegmentPusherConfig config = new S3DataSegmentPusherConfig();
    config.setBucket("bucket");
    config.setBaseKey("key");
    S3DataSegmentPusher pusher = new S3DataSegmentPusher(s3Client, config);
    // Create a mock segment on disk
    File tmp = tempFolder.newFile("version.bin");
    final byte[] data = new byte[] { 0x0, 0x0, 0x0, 0x1 };
    Files.write(data, tmp);
    final long size = data.length;
    DataSegment segmentToPush = new DataSegment("foo", Intervals.of("2015/2016"), "0", new HashMap<>(), new ArrayList<>(), new ArrayList<>(), NoneShardSpec.instance(), 0, size);
    DataSegment segment = pusher.push(tempFolder.getRoot(), segmentToPush, useUniquePath);
    Assert.assertEquals(segmentToPush.getSize(), segment.getSize());
    Assert.assertEquals(1, (int) segment.getBinaryVersion());
    Assert.assertEquals("bucket", segment.getLoadSpec().get("bucket"));
    Assert.assertTrue(segment.getLoadSpec().get("key").toString(), Pattern.compile(matcher).matcher(segment.getLoadSpec().get("key").toString()).matches());
    Assert.assertEquals("s3_zip", segment.getLoadSpec().get("type"));
    EasyMock.verify(s3Client);
}
Also used : AccessControlList(com.amazonaws.services.s3.model.AccessControlList) Grant(com.amazonaws.services.s3.model.Grant) Owner(com.amazonaws.services.s3.model.Owner) PutObjectResult(com.amazonaws.services.s3.model.PutObjectResult) DataSegment(org.apache.druid.timeline.DataSegment) CanonicalGrantee(com.amazonaws.services.s3.model.CanonicalGrantee) File(java.io.File)

Example 2 with Grant

use of software.amazon.awssdk.services.s3.model.Grant in project aws-doc-sdk-examples by awsdocs.

the class SetAcl method main.

public static void main(String[] args) {
    final String USAGE = "\n" + "Usage:\n" + "  <bucketName> <id> \n\n" + "Where:\n" + "  bucketName - the Amazon S3 bucket to grant permissions on. \n" + "  id - the ID of the owner of this bucket (you can get this value from the AWS Management Console).\n";
    if (args.length != 3) {
        System.out.println(USAGE);
        System.exit(1);
    }
    String bucketName = args[0];
    String id = args[1];
    System.out.format("Setting access \n");
    System.out.println(" in bucket: " + bucketName);
    Region region = Region.US_WEST_2;
    S3Client s3 = S3Client.builder().region(region).build();
    setBucketAcl(s3, bucketName, id);
    System.out.println("Done!");
    s3.close();
}
Also used : Region(software.amazon.awssdk.regions.Region) S3Client(software.amazon.awssdk.services.s3.S3Client)

Example 3 with Grant

use of software.amazon.awssdk.services.s3.model.Grant in project aws-doc-sdk-examples by awsdocs.

the class SetAcl method setBucketAcl.

// snippet-start:[s3.java2.set_acl.main]
public static void setBucketAcl(S3Client s3, String bucketName, String id) {
    try {
        Grant ownerGrant = Grant.builder().grantee(builder -> {
            builder.id(id).type(Type.CANONICAL_USER);
        }).permission(Permission.FULL_CONTROL).build();
        List<Grant> grantList2 = new ArrayList<>();
        grantList2.add(ownerGrant);
        AccessControlPolicy acl = AccessControlPolicy.builder().owner(builder -> builder.id(id)).grants(grantList2).build();
        PutBucketAclRequest putAclReq = PutBucketAclRequest.builder().bucket(bucketName).accessControlPolicy(acl).build();
        s3.putBucketAcl(putAclReq);
    } catch (S3Exception e) {
        e.printStackTrace();
        System.exit(1);
    }
}
Also used : Grant(software.amazon.awssdk.services.s3.model.Grant) AccessControlPolicy(software.amazon.awssdk.services.s3.model.AccessControlPolicy) S3Exception(software.amazon.awssdk.services.s3.model.S3Exception) PutBucketAclRequest(software.amazon.awssdk.services.s3.model.PutBucketAclRequest) ArrayList(java.util.ArrayList)

Example 4 with Grant

use of software.amazon.awssdk.services.s3.model.Grant in project aws-doc-sdk-examples by awsdocs.

the class GetAcl method getBucketACL.

// snippet-start:[s3.java2.get_acl.main]
public static String getBucketACL(S3Client s3, String objectKey, String bucketName) {
    try {
        GetObjectAclRequest aclReq = GetObjectAclRequest.builder().bucket(bucketName).key(objectKey).build();
        GetObjectAclResponse aclRes = s3.getObjectAcl(aclReq);
        List<Grant> grants = aclRes.grants();
        String grantee = "";
        for (Grant grant : grants) {
            System.out.format("  %s: %s\n", grant.grantee().id(), grant.permission());
            grantee = grant.grantee().id();
        }
        return grantee;
    } catch (S3Exception e) {
        System.err.println(e.awsErrorDetails().errorMessage());
        System.exit(1);
    }
    return "";
}
Also used : Grant(software.amazon.awssdk.services.s3.model.Grant) GetObjectAclRequest(software.amazon.awssdk.services.s3.model.GetObjectAclRequest) GetObjectAclResponse(software.amazon.awssdk.services.s3.model.GetObjectAclResponse) S3Exception(software.amazon.awssdk.services.s3.model.S3Exception)

Example 5 with Grant

use of software.amazon.awssdk.services.s3.model.Grant in project aws-doc-sdk-examples by awsdocs.

the class PersonalizeDemoOnMovieLens20M method main.

public static void main(String[] args) throws Exception {
    // Pre-flight step - 1
    // Set credentials provider, s3 and personalize clients
    // AWSStaticCredentialsProvider cred = getCredentialsProvider();
    S3Client s3Client = S3Client.builder().region(region).build();
    PersonalizeEventsClient personalizeEventsClient = PersonalizeEventsClient.builder().region(region).build();
    PersonalizeClient personalizeClient = PersonalizeClient.builder().region(region).build();
    PersonalizeRuntimeClient personalizeRuntimeClient = PersonalizeRuntimeClient.builder().region(region).build();
    IamClient iamClient = IamClient.builder().region(Region.AWS_GLOBAL).build();
    // Pre-flight step - 2
    // Identify your datasets and export them to S3 bucket
    DatasetProvider datasetProvider = new MovieLensDatasetProvider();
    datasetProvider.exportDatasetToS3(DatasetType.INTERACTIONS, s3Client, S3_BUCKET, true);
    // Pre-flight step - 3
    // Personalize needs the ability to assume Roles in AWS in order to have the permissions to execute certain tasks, the lines below grant that.
    DemoUtils.ensurePersonalizePermissionsOnS3Bucket(s3Client, S3_BUCKET);
    // Pre-flight step - 4
    // Ensure S3 bucket is accessible by Personalize
    String role = PREFIX + "-role";
    String roleArn = DemoUtils.createPersonalizeRole(iamClient, role);
    // Step 1
    // Create dataset group
    // Create dataset schemas
    // Create datasets
    String datasetGroupArn = createSchemaAndDatasets(personalizeClient, datasetProvider, roleArn);
    // step 2
    // create solution and solution version
    final String userPersonalizationRecipeArn = "arn:aws:personalize:::recipe/aws-user-personalization";
    final String userPersonalizationSolutionName = PREFIX + "-user-personalization-solution";
    final String awsUserPersonalizeSVArn = createSolutionAndSolutionVersion(personalizeClient, datasetGroupArn, userPersonalizationRecipeArn, userPersonalizationSolutionName);
    System.out.println("AWS User Personalization solution and solution version created");
    final String simsRecipeArn = "arn:aws:personalize:::recipe/aws-sims";
    final String simsSolutionName = PREFIX + "-sims-solution";
    final String simsSVArn = createSolutionAndSolutionVersion(personalizeClient, datasetGroupArn, simsRecipeArn, simsSolutionName);
    System.out.println("SIMS solution and solution version created");
    // step 3
    // setup campaign
    final String userPersonalizationCampaignName = PREFIX + "-user-personalization-campaign";
    final CampaignManager userPersonalizationCM = new CampaignManager(personalizeClient, userPersonalizationCampaignName, awsUserPersonalizeSVArn);
    final String userPersonalizationCampaignArn = userPersonalizationCM.createAndWaitForResource(true);
    System.out.println("AWS User Personalization campaign deployed");
    final String simsCampaignName = PREFIX + "-sims-campaign";
    final CampaignManager simsCM = new CampaignManager(personalizeClient, simsCampaignName, simsSVArn);
    final String simsCampaignArn = simsCM.createAndWaitForResource(true);
    System.out.println("SIMS campaign deployed");
    // Step 4
    // Create event tracker for real time events
    // AmazonPersonalizeEvents personalizeEvents = AmazonPersonalizeEventsClientBuilder.standard().withCredentials(cred).build();
    final String eventTrackerName = PREFIX + "-event-tracker";
    final EventTrackerManager etm = new EventTrackerManager(personalizeClient, eventTrackerName, datasetGroupArn);
    final String eventTackerArn = etm.createAndWaitForResource(true);
    final String eventTrackingId = etm.getTrackingId(eventTackerArn);
    System.out.println("Event tracker created");
    // step 5
    // create runtime client for demo
    runWebDemo(personalizeRuntimeClient, personalizeEventsClient, userPersonalizationCampaignArn, simsCampaignArn, eventTrackingId, datasetProvider);
}
Also used : PersonalizeClient(software.amazon.awssdk.services.personalize.PersonalizeClient) PersonalizeRuntimeClient(software.amazon.awssdk.services.personalizeruntime.PersonalizeRuntimeClient) IamClient(software.amazon.awssdk.services.iam.IamClient) CampaignManager(com.amazonaws.personalize.client.resource.CampaignManager) MovieLensDatasetProvider(com.amazonaws.personalize.client.datasets.MovieLensDatasetProvider) S3Client(software.amazon.awssdk.services.s3.S3Client) PersonalizeEventsClient(software.amazon.awssdk.services.personalizeevents.PersonalizeEventsClient) EventTrackerManager(com.amazonaws.personalize.client.resource.EventTrackerManager) DatasetProvider(com.amazonaws.personalize.client.datasets.DatasetProvider) MovieLensDatasetProvider(com.amazonaws.personalize.client.datasets.MovieLensDatasetProvider)

Aggregations

Grant (com.amazonaws.services.s3.model.Grant)6 AccessControlList (com.amazonaws.services.s3.model.AccessControlList)4 AmazonServiceException (com.amazonaws.AmazonServiceException)2 AmazonS3 (com.amazonaws.services.s3.AmazonS3)2 CanonicalGrantee (com.amazonaws.services.s3.model.CanonicalGrantee)2 S3Client (software.amazon.awssdk.services.s3.S3Client)2 Grant (software.amazon.awssdk.services.s3.model.Grant)2 S3Exception (software.amazon.awssdk.services.s3.model.S3Exception)2 DatasetProvider (com.amazonaws.personalize.client.datasets.DatasetProvider)1 MovieLensDatasetProvider (com.amazonaws.personalize.client.datasets.MovieLensDatasetProvider)1 CampaignManager (com.amazonaws.personalize.client.resource.CampaignManager)1 EventTrackerManager (com.amazonaws.personalize.client.resource.EventTrackerManager)1 Grantee (com.amazonaws.services.s3.model.Grantee)1 GroupGrantee (com.amazonaws.services.s3.model.GroupGrantee)1 Owner (com.amazonaws.services.s3.model.Owner)1 Permission (com.amazonaws.services.s3.model.Permission)1 PutObjectResult (com.amazonaws.services.s3.model.PutObjectResult)1 File (java.io.File)1 ArrayList (java.util.ArrayList)1 DataSegment (org.apache.druid.timeline.DataSegment)1