Search in sources :

Example 31 with CacheConfig

use of com.facebook.presto.cache.CacheConfig in project presto by prestodb.

the class TestHivePartitionManager method testUsesBucketingWithBucketFilters.

@Test
public void testUsesBucketingWithBucketFilters() {
    ConnectorSession session = new TestingConnectorSession(new HiveSessionProperties(new HiveClientConfig().setMaxBucketsForGroupedExecution(100), new OrcFileWriterConfig(), new ParquetFileWriterConfig(), new CacheConfig()).getSessionProperties());
    HiveTableHandle tableHandle = new HiveTableHandle(SCHEMA_NAME, TABLE_NAME);
    HivePartitionResult result = hivePartitionManager.getPartitions(metastore, tableHandle, new Constraint<>(TupleDomain.withColumnDomains(ImmutableMap.of(new HiveColumnHandle(BUCKET_COLUMN.getName(), BUCKET_COLUMN.getType(), parseTypeSignature(StandardTypes.VARCHAR), 0, REGULAR, Optional.empty(), Optional.empty()), Domain.singleValue(INTEGER, 1L)))), session);
    assertTrue(result.getBucketHandle().isPresent(), "bucketHandle is not present");
    assertTrue(result.getBucketFilter().isPresent(), "bucketFilter is present");
}
Also used : TestingConnectorSession(com.facebook.presto.testing.TestingConnectorSession) TestingConnectorSession(com.facebook.presto.testing.TestingConnectorSession) ConnectorSession(com.facebook.presto.spi.ConnectorSession) CacheConfig(com.facebook.presto.cache.CacheConfig) Test(org.testng.annotations.Test)

Example 32 with CacheConfig

use of com.facebook.presto.cache.CacheConfig in project presto by prestodb.

the class TestHivePartitionManager method testIgnoresBucketingWhenConfigured.

@Test
public void testIgnoresBucketingWhenConfigured() {
    ConnectorSession session = new TestingConnectorSession(new HiveSessionProperties(new HiveClientConfig().setIgnoreTableBucketing(true), new OrcFileWriterConfig(), new ParquetFileWriterConfig(), new CacheConfig()).getSessionProperties());
    HivePartitionResult result = hivePartitionManager.getPartitions(metastore, new HiveTableHandle(SCHEMA_NAME, TABLE_NAME), Constraint.alwaysTrue(), session);
    assertFalse(result.getBucketHandle().isPresent(), "bucketHandle is present");
    assertFalse(result.getBucketFilter().isPresent(), "bucketFilter is present");
}
Also used : TestingConnectorSession(com.facebook.presto.testing.TestingConnectorSession) TestingConnectorSession(com.facebook.presto.testing.TestingConnectorSession) ConnectorSession(com.facebook.presto.spi.ConnectorSession) CacheConfig(com.facebook.presto.cache.CacheConfig) Test(org.testng.annotations.Test)

Example 33 with CacheConfig

use of com.facebook.presto.cache.CacheConfig in project presto by prestodb.

the class TestHivePartitionManager method testUsesBucketingWithBucketColumn.

@Test
public void testUsesBucketingWithBucketColumn() {
    ConnectorSession session = new TestingConnectorSession(new HiveSessionProperties(new HiveClientConfig().setMaxBucketsForGroupedExecution(1), new OrcFileWriterConfig(), new ParquetFileWriterConfig(), new CacheConfig()).getSessionProperties());
    HiveTableHandle tableHandle = new HiveTableHandle(SCHEMA_NAME, TABLE_NAME);
    HivePartitionResult result = hivePartitionManager.getPartitions(metastore, tableHandle, new Constraint<>(TupleDomain.withColumnDomains(ImmutableMap.of(bucketColumnHandle(), Domain.singleValue(INTEGER, 1L)))), session);
    assertTrue(result.getBucketHandle().isPresent(), "bucketHandle is not present");
    assertTrue(result.getBucketFilter().isPresent(), "bucketFilter is present");
}
Also used : TestingConnectorSession(com.facebook.presto.testing.TestingConnectorSession) TestingConnectorSession(com.facebook.presto.testing.TestingConnectorSession) ConnectorSession(com.facebook.presto.spi.ConnectorSession) CacheConfig(com.facebook.presto.cache.CacheConfig) Test(org.testng.annotations.Test)

Example 34 with CacheConfig

use of com.facebook.presto.cache.CacheConfig in project presto by prestodb.

the class TestHivePartitionManager method testUsesBucketingWithPartitionFilters.

@Test
public void testUsesBucketingWithPartitionFilters() {
    ConnectorSession session = new TestingConnectorSession(new HiveSessionProperties(new HiveClientConfig().setMaxBucketsForGroupedExecution(100), new OrcFileWriterConfig(), new ParquetFileWriterConfig(), new CacheConfig()).getSessionProperties());
    HiveTableHandle tableHandle = new HiveTableHandle(SCHEMA_NAME, TABLE_NAME);
    HivePartitionResult result = hivePartitionManager.getPartitions(metastore, tableHandle, new Constraint<>(TupleDomain.withColumnDomains(ImmutableMap.of(new HiveColumnHandle(PARTITION_COLUMN.getName(), PARTITION_COLUMN.getType(), parseTypeSignature(StandardTypes.VARCHAR), MAX_PARTITION_KEY_COLUMN_INDEX, PARTITION_KEY, Optional.empty(), Optional.empty()), Domain.singleValue(VARCHAR, utf8Slice("2019-07-23"))))), session);
    assertTrue(result.getBucketHandle().isPresent(), "bucketHandle is not present");
    assertFalse(result.getBucketFilter().isPresent(), "bucketFilter is present");
}
Also used : TestingConnectorSession(com.facebook.presto.testing.TestingConnectorSession) TestingConnectorSession(com.facebook.presto.testing.TestingConnectorSession) ConnectorSession(com.facebook.presto.spi.ConnectorSession) CacheConfig(com.facebook.presto.cache.CacheConfig) Test(org.testng.annotations.Test)

Example 35 with CacheConfig

use of com.facebook.presto.cache.CacheConfig in project presto by prestodb.

the class TestHivePartitionManager method testUsesBucketingIfSmallEnough.

@Test
public void testUsesBucketingIfSmallEnough() {
    HiveTableHandle tableHandle = new HiveTableHandle(SCHEMA_NAME, TABLE_NAME);
    HivePartitionResult result = hivePartitionManager.getPartitions(metastore, tableHandle, Constraint.alwaysTrue(), new TestingConnectorSession(new HiveSessionProperties(new HiveClientConfig(), new OrcFileWriterConfig(), new ParquetFileWriterConfig(), new CacheConfig()).getSessionProperties()));
    assertTrue(result.getBucketHandle().isPresent(), "bucketHandle is not present");
    assertFalse(result.getBucketFilter().isPresent(), "bucketFilter is present");
}
Also used : TestingConnectorSession(com.facebook.presto.testing.TestingConnectorSession) CacheConfig(com.facebook.presto.cache.CacheConfig) Test(org.testng.annotations.Test)

Aggregations

CacheConfig (com.facebook.presto.cache.CacheConfig)45 Test (org.testng.annotations.Test)33 TestingConnectorSession (com.facebook.presto.testing.TestingConnectorSession)31 ConnectorSession (com.facebook.presto.spi.ConnectorSession)18 Configuration (org.apache.hadoop.conf.Configuration)12 SchemaTableName (com.facebook.presto.spi.SchemaTableName)9 ConnectorPageSource (com.facebook.presto.spi.ConnectorPageSource)8 HiveClientConfig (com.facebook.presto.hive.HiveClientConfig)7 HiveSessionProperties (com.facebook.presto.hive.HiveSessionProperties)7 OrcFileWriterConfig (com.facebook.presto.hive.OrcFileWriterConfig)7 ParquetFileWriterConfig (com.facebook.presto.hive.ParquetFileWriterConfig)7 TestingCacheUtils.stressTest (com.facebook.presto.cache.TestingCacheUtils.stressTest)6 Storage (com.facebook.presto.hive.metastore.Storage)6 File (java.io.File)6 StorageFormat (com.facebook.presto.hive.metastore.StorageFormat)5 OutputStreamDataSinkFactory (com.facebook.presto.hive.datasink.OutputStreamDataSinkFactory)4 RcFilePageSourceFactory (com.facebook.presto.hive.rcfile.RcFilePageSourceFactory)4 DataSize (io.airlift.units.DataSize)4 BeforeClass (org.testng.annotations.BeforeClass)4 CounterStat (com.facebook.airlift.stats.CounterStat)3