Search in sources :

Example 1 with GlueInputConverter

use of com.facebook.presto.hive.metastore.glue.converter.GlueInputConverter in project presto by prestodb.

the class GlueHiveMetastore method addPartitions.

@Override
public void addPartitions(MetastoreContext metastoreContext, String databaseName, String tableName, List<PartitionWithStatistics> partitions) {
    try {
        List<Future<BatchCreatePartitionResult>> futures = new ArrayList<>();
        for (List<PartitionWithStatistics> partitionBatch : Lists.partition(partitions, BATCH_CREATE_PARTITION_MAX_PAGE_SIZE)) {
            List<PartitionInput> partitionInputs = mappedCopy(partitionBatch, GlueInputConverter::convertPartition);
            futures.add(glueClient.batchCreatePartitionAsync(new BatchCreatePartitionRequest().withCatalogId(catalogId).withDatabaseName(databaseName).withTableName(tableName).withPartitionInputList(partitionInputs), stats.getBatchCreatePartitions().metricsAsyncHandler()));
        }
        for (Future<BatchCreatePartitionResult> future : futures) {
            BatchCreatePartitionResult result = future.get();
            propagatePartitionErrorToPrestoException(databaseName, tableName, result.getErrors());
        }
    } catch (AmazonServiceException | InterruptedException | ExecutionException e) {
        if (e instanceof InterruptedException) {
            Thread.currentThread().interrupt();
        }
        throw new PrestoException(HIVE_METASTORE_ERROR, e);
    }
}
Also used : ArrayList(java.util.ArrayList) PrestoException(com.facebook.presto.spi.PrestoException) PartitionInput(com.amazonaws.services.glue.model.PartitionInput) GlueInputConverter(com.facebook.presto.hive.metastore.glue.converter.GlueInputConverter) BatchCreatePartitionRequest(com.amazonaws.services.glue.model.BatchCreatePartitionRequest) PartitionWithStatistics(com.facebook.presto.hive.metastore.PartitionWithStatistics) BatchCreatePartitionResult(com.amazonaws.services.glue.model.BatchCreatePartitionResult) AmazonServiceException(com.amazonaws.AmazonServiceException) Future(java.util.concurrent.Future) ExecutionException(java.util.concurrent.ExecutionException)

Aggregations

AmazonServiceException (com.amazonaws.AmazonServiceException)1 BatchCreatePartitionRequest (com.amazonaws.services.glue.model.BatchCreatePartitionRequest)1 BatchCreatePartitionResult (com.amazonaws.services.glue.model.BatchCreatePartitionResult)1 PartitionInput (com.amazonaws.services.glue.model.PartitionInput)1 PartitionWithStatistics (com.facebook.presto.hive.metastore.PartitionWithStatistics)1 GlueInputConverter (com.facebook.presto.hive.metastore.glue.converter.GlueInputConverter)1 PrestoException (com.facebook.presto.spi.PrestoException)1 ArrayList (java.util.ArrayList)1 ExecutionException (java.util.concurrent.ExecutionException)1 Future (java.util.concurrent.Future)1