Search in sources :

Example 11 with DocumentClientException

use of com.microsoft.azure.cosmosdb.DocumentClientException in project ambry by linkedin.

the class CosmosDataAccessor method queryMetadata.

/**
 * Get the list of blobs in the specified partition matching the specified DocumentDB query spec.
 * @param partitionPath the partition to query.
 * @param querySpec the DocumentDB {@link SqlQuerySpec} to execute.
 * @param timer the {@link Timer} to use to record query time (excluding waiting).
 * @return a List of {@link CloudBlobMetadata} referencing the matching blobs.
 */
List<CloudBlobMetadata> queryMetadata(String partitionPath, SqlQuerySpec querySpec, Timer timer) throws DocumentClientException {
    FeedOptions feedOptions = new FeedOptions();
    // TODO: set maxItemCount
    feedOptions.setResponseContinuationTokenLimitInKb(continuationTokenLimitKb);
    feedOptions.setPartitionKey(new PartitionKey(partitionPath));
    // TODO: consolidate error count here
    try {
        Iterator<FeedResponse<Document>> iterator = executeCosmosQuery(partitionPath, querySpec, feedOptions, timer).getIterator();
        List<CloudBlobMetadata> metadataList = new ArrayList<>();
        double requestCharge = 0.0;
        while (iterator.hasNext()) {
            FeedResponse<Document> response = iterator.next();
            requestCharge += response.getRequestCharge();
            response.getResults().iterator().forEachRemaining(doc -> metadataList.add(createMetadataFromDocument(doc)));
        }
        if (requestCharge >= requestChargeThreshold) {
            logger.info("Query partition {} request charge {} for {} records", partitionPath, requestCharge, metadataList.size());
        }
        return metadataList;
    } catch (RuntimeException rex) {
        if (rex.getCause() instanceof DocumentClientException) {
            logger.warn("Query {} on partition {} got {}", querySpec.getQueryText(), partitionPath, ((DocumentClientException) rex.getCause()).getStatusCode());
            throw (DocumentClientException) rex.getCause();
        }
        throw rex;
    }
}
Also used : ChangeFeedOptions(com.microsoft.azure.cosmosdb.ChangeFeedOptions) FeedOptions(com.microsoft.azure.cosmosdb.FeedOptions) CloudBlobMetadata(com.github.ambry.cloud.CloudBlobMetadata) ArrayList(java.util.ArrayList) PartitionKey(com.microsoft.azure.cosmosdb.PartitionKey) FeedResponse(com.microsoft.azure.cosmosdb.FeedResponse) Document(com.microsoft.azure.cosmosdb.Document) DocumentClientException(com.microsoft.azure.cosmosdb.DocumentClientException)

Example 12 with DocumentClientException

use of com.microsoft.azure.cosmosdb.DocumentClientException in project ambry by linkedin.

the class AzureStorageCompactorTest method testPurgeWithCosmosBulkDeleteError.

/**
 * Test purgeBlobs with Cosmos bulk delete error
 */
@Test
public void testPurgeWithCosmosBulkDeleteError() throws Exception {
    Exception mockException = new RuntimeException(new DocumentClientException(HttpConstants.StatusCodes.TOO_MANY_REQUESTS));
    doThrow(mockException).when(mockumentClient).executeStoredProcedure(anyString(), any(RequestOptions.class), any());
    try {
        AzureCompactionUtil.purgeBlobs(blobMetadataList, azureBlobDataAccessor, azureMetrics, cosmosDataAccessor);
        fail("Expected CloudStorageException");
    } catch (CloudStorageException bex) {
    }
    assertEquals(0, azureMetrics.blobDeletedCount.getCount());
    assertEquals(numBlobsPerQuery, azureMetrics.blobDeleteErrorCount.getCount());
}
Also used : RequestOptions(com.microsoft.azure.cosmosdb.RequestOptions) CloudStorageException(com.github.ambry.cloud.CloudStorageException) DocumentClientException(com.microsoft.azure.cosmosdb.DocumentClientException) BlobStorageException(com.azure.storage.blob.models.BlobStorageException) CloudStorageException(com.github.ambry.cloud.CloudStorageException) IOException(java.io.IOException) DocumentClientException(com.microsoft.azure.cosmosdb.DocumentClientException) Test(org.junit.Test)

Aggregations

DocumentClientException (com.microsoft.azure.cosmosdb.DocumentClientException)12 Document (com.microsoft.azure.cosmosdb.Document)10 ChangeFeedOptions (com.microsoft.azure.cosmosdb.ChangeFeedOptions)8 FeedResponse (com.microsoft.azure.cosmosdb.FeedResponse)7 CloudBlobMetadata (com.github.ambry.cloud.CloudBlobMetadata)6 FeedOptions (com.microsoft.azure.cosmosdb.FeedOptions)6 PartitionKey (com.microsoft.azure.cosmosdb.PartitionKey)6 RequestOptions (com.microsoft.azure.cosmosdb.RequestOptions)6 CloudStorageException (com.github.ambry.cloud.CloudStorageException)5 SqlParameter (com.microsoft.azure.cosmosdb.SqlParameter)5 SqlParameterCollection (com.microsoft.azure.cosmosdb.SqlParameterCollection)5 SqlQuerySpec (com.microsoft.azure.cosmosdb.SqlQuerySpec)5 ArrayList (java.util.ArrayList)5 Timer (com.codahale.metrics.Timer)3 AccessCondition (com.microsoft.azure.cosmosdb.AccessCondition)3 Date (java.util.Date)3 HashSet (java.util.HashSet)3 SecretClient (com.azure.security.keyvault.secrets.SecretClient)2 SecretClientBuilder (com.azure.security.keyvault.secrets.SecretClientBuilder)2 Container (com.github.ambry.account.Container)2