Search in sources :

Example 1 with ConditionalCheckFailedException

use of software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException in project iceberg by apache.

the class DynamoDbCatalog method createNamespace.

@Override
public void createNamespace(Namespace namespace, Map<String, String> metadata) {
    validateNamespace(namespace);
    Map<String, AttributeValue> values = namespacePrimaryKey(namespace);
    setNewCatalogEntryMetadata(values);
    metadata.forEach((key, value) -> values.put(toPropertyCol(key), AttributeValue.builder().s(value).build()));
    try {
        dynamo.putItem(PutItemRequest.builder().tableName(awsProperties.dynamoDbTableName()).conditionExpression("attribute_not_exists(" + DynamoDbCatalog.COL_VERSION + ")").item(values).build());
    } catch (ConditionalCheckFailedException e) {
        throw new AlreadyExistsException("Cannot create namespace %s: already exists", namespace);
    }
}
Also used : AttributeValue(software.amazon.awssdk.services.dynamodb.model.AttributeValue) AlreadyExistsException(org.apache.iceberg.exceptions.AlreadyExistsException) ConditionalCheckFailedException(software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException)

Example 2 with ConditionalCheckFailedException

use of software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException in project iceberg by apache.

the class DynamoDbCatalog method updateProperties.

private boolean updateProperties(Namespace namespace, String updateExpression, Map<String, AttributeValue> attributeValues, Map<String, String> attributeNames) {
    validateNamespace(namespace);
    Map<String, AttributeValue> key = namespacePrimaryKey(namespace);
    try {
        GetItemResponse response = dynamo.getItem(GetItemRequest.builder().tableName(awsProperties.dynamoDbTableName()).consistentRead(true).key(key).build());
        if (!response.hasItem()) {
            throw new NoSuchNamespaceException("Cannot find namespace %s", namespace);
        }
        attributeValues.put(":v", response.item().get(COL_VERSION));
        dynamo.updateItem(UpdateItemRequest.builder().tableName(awsProperties.dynamoDbTableName()).key(key).conditionExpression(COL_VERSION + " = :v").updateExpression(updateExpression).expressionAttributeValues(attributeValues).expressionAttributeNames(attributeNames).build());
        return true;
    } catch (ConditionalCheckFailedException e) {
        return false;
    }
}
Also used : AttributeValue(software.amazon.awssdk.services.dynamodb.model.AttributeValue) NoSuchNamespaceException(org.apache.iceberg.exceptions.NoSuchNamespaceException) GetItemResponse(software.amazon.awssdk.services.dynamodb.model.GetItemResponse) ConditionalCheckFailedException(software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException)

Example 3 with ConditionalCheckFailedException

use of software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException in project iceberg by apache.

the class DynamoDbCatalog method dropTable.

@Override
public boolean dropTable(TableIdentifier identifier, boolean purge) {
    Map<String, AttributeValue> key = tablePrimaryKey(identifier);
    try {
        GetItemResponse response = dynamo.getItem(GetItemRequest.builder().tableName(awsProperties.dynamoDbTableName()).consistentRead(true).key(key).build());
        if (!response.hasItem()) {
            throw new NoSuchTableException("Cannot find table %s to drop", identifier);
        }
        TableOperations ops = newTableOps(identifier);
        TableMetadata lastMetadata = ops.current();
        dynamo.deleteItem(DeleteItemRequest.builder().tableName(awsProperties.dynamoDbTableName()).key(tablePrimaryKey(identifier)).conditionExpression(COL_VERSION + " = :v").expressionAttributeValues(ImmutableMap.of(":v", response.item().get(COL_VERSION))).build());
        LOG.info("Successfully dropped table {} from DynamoDb catalog", identifier);
        if (purge && lastMetadata != null) {
            CatalogUtil.dropTableData(ops.io(), lastMetadata);
            LOG.info("Table {} data purged", identifier);
        }
        LOG.info("Dropped table: {}", identifier);
        return true;
    } catch (ConditionalCheckFailedException e) {
        LOG.error("Cannot complete drop table operation for {}: commit conflict", identifier, e);
        return false;
    } catch (Exception e) {
        LOG.error("Cannot complete drop table operation for {}: unexpected exception", identifier, e);
        throw e;
    }
}
Also used : TableMetadata(org.apache.iceberg.TableMetadata) AttributeValue(software.amazon.awssdk.services.dynamodb.model.AttributeValue) TableOperations(org.apache.iceberg.TableOperations) NoSuchTableException(org.apache.iceberg.exceptions.NoSuchTableException) GetItemResponse(software.amazon.awssdk.services.dynamodb.model.GetItemResponse) ConditionalCheckFailedException(software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException) AlreadyExistsException(org.apache.iceberg.exceptions.AlreadyExistsException) NoSuchNamespaceException(org.apache.iceberg.exceptions.NoSuchNamespaceException) NoSuchTableException(org.apache.iceberg.exceptions.NoSuchTableException) NamespaceNotEmptyException(org.apache.iceberg.exceptions.NamespaceNotEmptyException) ConditionalCheckFailedException(software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException) IOException(java.io.IOException) ResourceNotFoundException(software.amazon.awssdk.services.dynamodb.model.ResourceNotFoundException) ValidationException(org.apache.iceberg.exceptions.ValidationException)

Example 4 with ConditionalCheckFailedException

use of software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException in project iceberg by apache.

the class DynamoDbTableOperations method doCommit.

@Override
protected void doCommit(TableMetadata base, TableMetadata metadata) {
    String newMetadataLocation = writeNewMetadata(metadata, currentVersion() + 1);
    CommitStatus commitStatus = CommitStatus.FAILURE;
    Map<String, AttributeValue> tableKey = DynamoDbCatalog.tablePrimaryKey(tableIdentifier);
    try {
        GetItemResponse table = dynamo.getItem(GetItemRequest.builder().tableName(awsProperties.dynamoDbTableName()).consistentRead(true).key(tableKey).build());
        checkMetadataLocation(table, base);
        Map<String, String> properties = prepareProperties(table, newMetadataLocation);
        persistTable(tableKey, table, properties);
        commitStatus = CommitStatus.SUCCESS;
    } catch (ConditionalCheckFailedException e) {
        throw new CommitFailedException(e, "Cannot commit %s: concurrent update detected", tableName());
    } catch (RuntimeException persistFailure) {
        LOG.error("Confirming if commit to {} indeed failed to persist, attempting to reconnect and check.", fullTableName, persistFailure);
        commitStatus = checkCommitStatus(newMetadataLocation, metadata);
        switch(commitStatus) {
            case SUCCESS:
                break;
            case FAILURE:
                throw new CommitFailedException(persistFailure, "Cannot commit %s due to unexpected exception", tableName());
            case UNKNOWN:
                throw new CommitStateUnknownException(persistFailure);
        }
    } finally {
        try {
            if (commitStatus == CommitStatus.FAILURE) {
                // if anything went wrong, clean up the uncommitted metadata file
                io().deleteFile(newMetadataLocation);
            }
        } catch (RuntimeException e) {
            LOG.error("Fail to cleanup metadata file at {}", newMetadataLocation, e);
            throw e;
        }
    }
}
Also used : AttributeValue(software.amazon.awssdk.services.dynamodb.model.AttributeValue) CommitStateUnknownException(org.apache.iceberg.exceptions.CommitStateUnknownException) GetItemResponse(software.amazon.awssdk.services.dynamodb.model.GetItemResponse) ConditionalCheckFailedException(software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException) CommitFailedException(org.apache.iceberg.exceptions.CommitFailedException)

Example 5 with ConditionalCheckFailedException

use of software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException in project iep by Netflix.

the class DynamoDbLeaderDatabase method removeLeadershipFor.

@Override
public boolean removeLeadershipFor(ResourceId resourceId) {
    boolean removed;
    try {
        final Map<String, AttributeValue> resourceRecordKey = Collections.singletonMap(hashKeyName, AttributeValue.builder().s(resourceId.getId()).build());
        final Map<String, AttributeValue> expressionAttributeValues = new HashMap<>(2, 1.0f);
        expressionAttributeValues.put(LEADER_ID_PLACEHOLDER, AttributeValue.builder().s(leaderId.getId()).build());
        expressionAttributeValues.put(NO_LEADER_PLACEHOLDER, AttributeValue.builder().s(LeaderId.NO_LEADER.getId()).build());
        final UpdateItemRequest updateRequest = UpdateItemRequest.builder().tableName(tableName).key(resourceRecordKey).updateExpression(removeLeadershipExpression).conditionExpression(removeLeadershipConditionExpression).expressionAttributeValues(expressionAttributeValues).build();
        removed = db.updateItem(updateRequest).sdkHttpResponse().isSuccessful();
    } catch (ConditionalCheckFailedException e) {
        logger.debug("Didn't remove leader attribute for {}; this {} is not its leader.", resourceId, leaderId.getId());
        removed = false;
    }
    return removed;
}
Also used : AttributeValue(software.amazon.awssdk.services.dynamodb.model.AttributeValue) UpdateItemRequest(software.amazon.awssdk.services.dynamodb.model.UpdateItemRequest) HashMap(java.util.HashMap) ConditionalCheckFailedException(software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException)

Aggregations

ConditionalCheckFailedException (software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException)19 AttributeValue (software.amazon.awssdk.services.dynamodb.model.AttributeValue)13 UpdateItemRequest (software.amazon.awssdk.services.dynamodb.model.UpdateItemRequest)8 HashMap (java.util.HashMap)7 GetItemResponse (software.amazon.awssdk.services.dynamodb.model.GetItemResponse)3 Delete (com.scalar.db.api.Delete)2 Put (com.scalar.db.api.Put)2 TableMetadata (com.scalar.db.api.TableMetadata)2 ExecutionException (com.scalar.db.exception.storage.ExecutionException)2 NoMutationException (com.scalar.db.exception.storage.NoMutationException)2 RetriableExecutionException (com.scalar.db.exception.storage.RetriableExecutionException)2 AbstractProcessInstance (io.automatiko.engine.workflow.AbstractProcessInstance)2 Instant (java.time.Instant)2 ArrayList (java.util.ArrayList)2 CompletionException (java.util.concurrent.CompletionException)2 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)2 Nonnull (javax.annotation.Nonnull)2 AlreadyExistsException (org.apache.iceberg.exceptions.AlreadyExistsException)2 NoSuchNamespaceException (org.apache.iceberg.exceptions.NoSuchNamespaceException)2 Test (org.junit.Test)2