Search in sources :

Example 36 with Tag

use of software.amazon.awssdk.services.s3.model.Tag in project aws-doc-sdk-examples by awsdocs.

the class ManagingObjectTags method putS3ObjectTags.

// snippet-start:[s3.java2.s3_object_manage_tags.main]
public static void putS3ObjectTags(S3Client s3, String bucketName, String objectKey, String objectPath) {
    try {
        // Define the tags.
        Tag tag1 = Tag.builder().key("Tag 1").value("This is tag 1").build();
        Tag tag2 = Tag.builder().key("Tag 2").value("This is tag 2").build();
        List<Tag> tags = new ArrayList<Tag>();
        tags.add(tag1);
        tags.add(tag2);
        Tagging allTags = Tagging.builder().tagSet(tags).build();
        PutObjectRequest putOb = PutObjectRequest.builder().bucket(bucketName).key(objectKey).tagging(allTags).build();
        s3.putObject(putOb, RequestBody.fromBytes(getObjectFile(objectPath)));
    } catch (S3Exception e) {
        System.err.println(e.getMessage());
        System.exit(1);
    }
}
Also used : S3Exception(software.amazon.awssdk.services.s3.model.S3Exception) ArrayList(java.util.ArrayList) Tagging(software.amazon.awssdk.services.s3.model.Tagging) Tag(software.amazon.awssdk.services.s3.model.Tag) PutObjectRequest(software.amazon.awssdk.services.s3.model.PutObjectRequest)

Example 37 with Tag

use of software.amazon.awssdk.services.s3.model.Tag in project aws-doc-sdk-examples by awsdocs.

the class PutObjectMetadata method main.

public static void main(String[] args) {
    final String USAGE = "\n" + "Usage:\n" + "  <bucketName> <objectKey> <objectPath> \n\n" + "Where:\n" + "  bucketName - the Amazon S3 bucket to upload an object into.\n" + "  objectKey - the object to upload (for example, book.pdf).\n" + "  objectPath - the path where the file is located (for example, C:/AWS/book2.pdf). \n\n";
    if (args.length != 3) {
        System.out.println(USAGE);
        System.exit(1);
    }
    String bucketName = args[0];
    String objectKey = args[1];
    String objectPath = args[2];
    System.out.println("Putting object " + objectKey + " into bucket " + bucketName);
    System.out.println("  in bucket: " + bucketName);
    Region region = Region.US_EAST_1;
    S3Client s3 = S3Client.builder().region(region).build();
    String result = putS3Object(s3, bucketName, objectKey, objectPath);
    System.out.println("Tag information: " + result);
    s3.close();
}
Also used : Region(software.amazon.awssdk.regions.Region) S3Client(software.amazon.awssdk.services.s3.S3Client)

Example 38 with Tag

use of software.amazon.awssdk.services.s3.model.Tag in project XRTB by benmfaul.

the class Configuration method processDirectory.

public void processDirectory(AmazonS3Client s3, ObjectListing listing, String bucket) throws Exception {
    for (S3ObjectSummary objectSummary : listing.getObjectSummaries()) {
        long size = objectSummary.getSize();
        logger.info("*** Processing S3 {}, size: {}", objectSummary.getKey(), size);
        S3Object object = s3.getObject(new GetObjectRequest(bucket, objectSummary.getKey()));
        String bucketName = object.getBucketName();
        String keyName = object.getKey();
        GetObjectTaggingRequest request = new GetObjectTaggingRequest(bucketName, keyName);
        GetObjectTaggingResult result = s3.getObjectTagging(request);
        List<Tag> tags = result.getTagSet();
        String type = null;
        String name = null;
        if (tags.isEmpty()) {
            System.err.println("Error: " + keyName + " has no tags");
        } else {
            for (Tag tag : tags) {
                String key = tag.getKey();
                String value = tag.getValue();
                if (key.equals("type")) {
                    type = value;
                }
                if (key.equals("name")) {
                    name = value;
                }
            }
            if (name == null)
                throw new Exception("Error: " + keyName + " is missing a name tag");
            if (name.contains(" "))
                throw new Exception("Error: " + keyName + " has a name attribute with a space in it");
            if (type == null)
                throw new Exception("Error: " + keyName + " has no type tag");
            if (!name.startsWith("$"))
                name = "$" + name;
            readData(type, name, object, size);
        }
    }
}
Also used : GetObjectTaggingRequest(com.amazonaws.services.s3.model.GetObjectTaggingRequest) S3ObjectSummary(com.amazonaws.services.s3.model.S3ObjectSummary) S3Object(com.amazonaws.services.s3.model.S3Object) GeoTag(com.xrtb.geo.GeoTag) Tag(com.amazonaws.services.s3.model.Tag) GetObjectRequest(com.amazonaws.services.s3.model.GetObjectRequest) GetObjectTaggingResult(com.amazonaws.services.s3.model.GetObjectTaggingResult)

Example 39 with Tag

use of software.amazon.awssdk.services.s3.model.Tag in project crate by crate.

the class AwsEc2SeedHostsProvider method fetchDynamicNodes.

private List<TransportAddress> fetchDynamicNodes() {
    final List<TransportAddress> dynamicHosts = new ArrayList<>();
    final DescribeInstancesResult descInstances;
    try (AmazonEc2Reference clientReference = awsEc2Service.client()) {
        // Query EC2 API based on AZ, instance state, and tag.
        // NOTE: we don't filter by security group during the describe instances request for two reasons:
        // 1. differences in VPCs require different parameters during query (ID vs Name)
        // 2. We want to use two different strategies: (all security groups vs. any security groups)
        descInstances = clientReference.client().describeInstances(buildDescribeInstancesRequest());
    } catch (final AmazonClientException e) {
        LOGGER.info("Exception while retrieving instance list from AWS API: {}", e.getMessage());
        LOGGER.debug("Full exception:", e);
        return dynamicHosts;
    }
    LOGGER.trace("finding seed nodes...");
    for (final Reservation reservation : descInstances.getReservations()) {
        for (final Instance instance : reservation.getInstances()) {
            // lets see if we can filter based on groups
            if (!groups.isEmpty()) {
                final List<GroupIdentifier> instanceSecurityGroups = instance.getSecurityGroups();
                final List<String> securityGroupNames = new ArrayList<>(instanceSecurityGroups.size());
                final List<String> securityGroupIds = new ArrayList<>(instanceSecurityGroups.size());
                for (final GroupIdentifier sg : instanceSecurityGroups) {
                    securityGroupNames.add(sg.getGroupName());
                    securityGroupIds.add(sg.getGroupId());
                }
                if (bindAnyGroup) {
                    // We check if we can find at least one group name or one group id in groups.
                    if (disjoint(securityGroupNames, groups) && disjoint(securityGroupIds, groups)) {
                        LOGGER.trace("filtering out instance {} based on groups {}, not part of {}", instance.getInstanceId(), instanceSecurityGroups, groups);
                        // continue to the next instance
                        continue;
                    }
                } else {
                    // We need tp match all group names or group ids, otherwise we ignore this instance
                    if (!(securityGroupNames.containsAll(groups) || securityGroupIds.containsAll(groups))) {
                        LOGGER.trace("filtering out instance {} based on groups {}, does not include all of {}", instance.getInstanceId(), instanceSecurityGroups, groups);
                        // continue to the next instance
                        continue;
                    }
                }
            }
            String address = null;
            if (hostType.equals(PRIVATE_DNS)) {
                address = instance.getPrivateDnsName();
            } else if (hostType.equals(PRIVATE_IP)) {
                address = instance.getPrivateIpAddress();
            } else if (hostType.equals(PUBLIC_DNS)) {
                address = instance.getPublicDnsName();
            } else if (hostType.equals(PUBLIC_IP)) {
                address = instance.getPublicIpAddress();
            } else if (hostType.startsWith(TAG_PREFIX)) {
                // Reading the node host from its metadata
                final String tagName = hostType.substring(TAG_PREFIX.length());
                LOGGER.debug("reading hostname from [{}] instance tag", tagName);
                final List<Tag> tags = instance.getTags();
                for (final Tag tag : tags) {
                    if (tag.getKey().equals(tagName)) {
                        address = tag.getValue();
                        LOGGER.debug("using [{}] as the instance address", address);
                    }
                }
            } else {
                throw new IllegalArgumentException(hostType + " is unknown for discovery.ec2.host_type");
            }
            if (address != null) {
                try {
                    final TransportAddress[] addresses = transportService.addressesFromString(address);
                    for (int i = 0; i < addresses.length; i++) {
                        LOGGER.trace("adding {}, address {}, transport_address {}", instance.getInstanceId(), address, addresses[i]);
                        dynamicHosts.add(addresses[i]);
                    }
                } catch (final Exception e) {
                    final String finalAddress = address;
                    LOGGER.warn((Supplier<?>) () -> new ParameterizedMessage("failed to add {}, address {}", instance.getInstanceId(), finalAddress), e);
                }
            } else {
                LOGGER.trace("not adding {}, address is null, host_type {}", instance.getInstanceId(), hostType);
            }
        }
    }
    LOGGER.debug("using dynamic transport addresses {}", dynamicHosts);
    return dynamicHosts;
}
Also used : Instance(com.amazonaws.services.ec2.model.Instance) TransportAddress(org.elasticsearch.common.transport.TransportAddress) AmazonClientException(com.amazonaws.AmazonClientException) ArrayList(java.util.ArrayList) Collections.disjoint(java.util.Collections.disjoint) AmazonClientException(com.amazonaws.AmazonClientException) GroupIdentifier(com.amazonaws.services.ec2.model.GroupIdentifier) DescribeInstancesResult(com.amazonaws.services.ec2.model.DescribeInstancesResult) Reservation(com.amazonaws.services.ec2.model.Reservation) Supplier(org.apache.logging.log4j.util.Supplier) ParameterizedMessage(org.apache.logging.log4j.message.ParameterizedMessage) Tag(com.amazonaws.services.ec2.model.Tag)

Example 40 with Tag

use of software.amazon.awssdk.services.s3.model.Tag in project elasticsearch by elastic.

the class AmazonEC2Mock method describeInstances.

@Override
public DescribeInstancesResult describeInstances(DescribeInstancesRequest describeInstancesRequest) throws AmazonServiceException, AmazonClientException {
    Collection<Instance> filteredInstances = new ArrayList<>();
    logger.debug("--> mocking describeInstances");
    for (Instance instance : instances) {
        boolean tagFiltered = false;
        boolean instanceFound = false;
        Map<String, List<String>> expectedTags = new HashMap<>();
        Map<String, List<String>> instanceTags = new HashMap<>();
        for (Tag tag : instance.getTags()) {
            List<String> tags = instanceTags.get(tag.getKey());
            if (tags == null) {
                tags = new ArrayList<>();
                instanceTags.put(tag.getKey(), tags);
            }
            tags.add(tag.getValue());
        }
        for (Filter filter : describeInstancesRequest.getFilters()) {
            // If we have the same tag name and one of the values, we add the instance
            if (filter.getName().startsWith("tag:")) {
                tagFiltered = true;
                String tagName = filter.getName().substring(4);
                // if we have more than one value for the same key, then the key is appended with .x
                Pattern p = Pattern.compile("\\.\\d+", Pattern.DOTALL);
                Matcher m = p.matcher(tagName);
                if (m.find()) {
                    int i = tagName.lastIndexOf(".");
                    tagName = tagName.substring(0, i);
                }
                List<String> tags = expectedTags.get(tagName);
                if (tags == null) {
                    tags = new ArrayList<>();
                    expectedTags.put(tagName, tags);
                }
                tags.addAll(filter.getValues());
            }
        }
        if (tagFiltered) {
            logger.debug("--> expected tags: [{}]", expectedTags);
            logger.debug("--> instance tags: [{}]", instanceTags);
            instanceFound = true;
            for (Map.Entry<String, List<String>> expectedTagsEntry : expectedTags.entrySet()) {
                List<String> instanceTagValues = instanceTags.get(expectedTagsEntry.getKey());
                if (instanceTagValues == null) {
                    instanceFound = false;
                    break;
                }
                for (String expectedValue : expectedTagsEntry.getValue()) {
                    boolean valueFound = false;
                    for (String instanceTagValue : instanceTagValues) {
                        if (instanceTagValue.equals(expectedValue)) {
                            valueFound = true;
                        }
                    }
                    if (valueFound == false) {
                        instanceFound = false;
                    }
                }
            }
        }
        if (tagFiltered == false || instanceFound) {
            logger.debug("--> instance added");
            filteredInstances.add(instance);
        } else {
            logger.debug("--> instance filtered");
        }
    }
    return new DescribeInstancesResult().withReservations(new Reservation().withInstances(filteredInstances));
}
Also used : Pattern(java.util.regex.Pattern) Instance(com.amazonaws.services.ec2.model.Instance) HashMap(java.util.HashMap) Matcher(java.util.regex.Matcher) ArrayList(java.util.ArrayList) DescribeInstancesResult(com.amazonaws.services.ec2.model.DescribeInstancesResult) Reservation(com.amazonaws.services.ec2.model.Reservation) Filter(com.amazonaws.services.ec2.model.Filter) ArrayList(java.util.ArrayList) List(java.util.List) Tag(com.amazonaws.services.ec2.model.Tag) Map(java.util.Map) HashMap(java.util.HashMap)

Aggregations

Tag (com.amazonaws.services.ec2.model.Tag)38 ArrayList (java.util.ArrayList)27 Tag (com.amazonaws.services.s3.model.Tag)18 HashMap (java.util.HashMap)17 Test (org.junit.Test)16 Instance (com.amazonaws.services.ec2.model.Instance)15 List (java.util.List)15 S3FileTransferRequestParamsDto (org.finra.herd.model.dto.S3FileTransferRequestParamsDto)14 File (java.io.File)11 Map (java.util.Map)10 GetObjectTaggingRequest (com.amazonaws.services.s3.model.GetObjectTaggingRequest)9 GetObjectTaggingResult (com.amazonaws.services.s3.model.GetObjectTaggingResult)9 HashSet (java.util.HashSet)9 Utils (com.vmware.xenon.common.Utils)8 CreateTagsRequest (com.amazonaws.services.ec2.model.CreateTagsRequest)7 Reservation (com.amazonaws.services.ec2.model.Reservation)7 S3ObjectSummary (com.amazonaws.services.s3.model.S3ObjectSummary)5 TagsUtil.newTagState (com.vmware.photon.controller.model.adapters.util.TagsUtil.newTagState)5 TagState (com.vmware.photon.controller.model.resources.TagService.TagState)5 DeferredResult (com.vmware.xenon.common.DeferredResult)5