Search in sources :

Example 41 with SchemaTableName

use of io.trino.spi.connector.SchemaTableName in project trino by trinodb.

the class AbstractTestHive method assertGetRecords.

protected void assertGetRecords(String tableName, HiveStorageFormat hiveStorageFormat) throws Exception {
    try (Transaction transaction = newTransaction()) {
        ConnectorSession session = newSession();
        ConnectorMetadata metadata = transaction.getMetadata();
        metadata.beginQuery(session);
        ConnectorTableHandle tableHandle = getTableHandle(metadata, new SchemaTableName(database, tableName));
        ConnectorTableMetadata tableMetadata = metadata.getTableMetadata(session, tableHandle);
        HiveSplit hiveSplit = getHiveSplit(tableHandle, transaction, session);
        List<ColumnHandle> columnHandles = ImmutableList.copyOf(metadata.getColumnHandles(session, tableHandle).values());
        ConnectorPageSource pageSource = pageSourceProvider.createPageSource(transaction.getTransactionHandle(), session, hiveSplit, tableHandle, columnHandles, DynamicFilter.EMPTY);
        assertGetRecords(hiveStorageFormat, tableMetadata, hiveSplit, pageSource, columnHandles);
    }
}
Also used : HiveColumnHandle.bucketColumnHandle(io.trino.plugin.hive.HiveColumnHandle.bucketColumnHandle) ColumnHandle(io.trino.spi.connector.ColumnHandle) ConnectorSession(io.trino.spi.connector.ConnectorSession) TestingConnectorSession(io.trino.testing.TestingConnectorSession) ConnectorMetadata(io.trino.spi.connector.ConnectorMetadata) ConnectorPageSource(io.trino.spi.connector.ConnectorPageSource) CatalogSchemaTableName(io.trino.spi.connector.CatalogSchemaTableName) SchemaTableName(io.trino.spi.connector.SchemaTableName) ConnectorTableMetadata(io.trino.spi.connector.ConnectorTableMetadata) ConnectorTableHandle(io.trino.spi.connector.ConnectorTableHandle)

Example 42 with SchemaTableName

use of io.trino.spi.connector.SchemaTableName in project trino by trinodb.

the class AbstractTestHive method testTableCreationRollback.

@Test
public void testTableCreationRollback() throws Exception {
    SchemaTableName temporaryCreateRollbackTable = temporaryTable("create_rollback");
    try {
        Path stagingPathRoot;
        try (Transaction transaction = newTransaction()) {
            ConnectorSession session = newSession();
            ConnectorMetadata metadata = transaction.getMetadata();
            // begin creating the table
            ConnectorTableMetadata tableMetadata = new ConnectorTableMetadata(temporaryCreateRollbackTable, CREATE_TABLE_COLUMNS, createTableProperties(RCBINARY));
            ConnectorOutputTableHandle outputHandle = metadata.beginCreateTable(session, tableMetadata, Optional.empty(), NO_RETRIES);
            // write the data
            ConnectorPageSink sink = pageSinkProvider.createPageSink(transaction.getTransactionHandle(), session, outputHandle);
            sink.appendPage(CREATE_TABLE_DATA.toPage());
            getFutureValue(sink.finish());
            // verify we have data files
            stagingPathRoot = getStagingPathRoot(outputHandle);
            HdfsContext context = new HdfsContext(session);
            assertFalse(listAllDataFiles(context, stagingPathRoot).isEmpty());
            // rollback the table
            transaction.rollback();
        }
        // verify all files have been deleted
        HdfsContext context = new HdfsContext(newSession());
        assertTrue(listAllDataFiles(context, stagingPathRoot).isEmpty());
        // verify table is not in the metastore
        try (Transaction transaction = newTransaction()) {
            ConnectorSession session = newSession();
            ConnectorMetadata metadata = transaction.getMetadata();
            assertNull(metadata.getTableHandle(session, temporaryCreateRollbackTable));
        }
    } finally {
        dropTable(temporaryCreateRollbackTable);
    }
}
Also used : Path(org.apache.hadoop.fs.Path) HiveSessionProperties.getTemporaryStagingDirectoryPath(io.trino.plugin.hive.HiveSessionProperties.getTemporaryStagingDirectoryPath) ConnectorOutputTableHandle(io.trino.spi.connector.ConnectorOutputTableHandle) ConnectorSession(io.trino.spi.connector.ConnectorSession) TestingConnectorSession(io.trino.testing.TestingConnectorSession) ConnectorMetadata(io.trino.spi.connector.ConnectorMetadata) HdfsContext(io.trino.plugin.hive.HdfsEnvironment.HdfsContext) ConnectorPageSink(io.trino.spi.connector.ConnectorPageSink) CatalogSchemaTableName(io.trino.spi.connector.CatalogSchemaTableName) SchemaTableName(io.trino.spi.connector.SchemaTableName) ConnectorTableMetadata(io.trino.spi.connector.ConnectorTableMetadata) Test(org.testng.annotations.Test)

Example 43 with SchemaTableName

use of io.trino.spi.connector.SchemaTableName in project trino by trinodb.

the class AbstractTestHive method testUpdateBasicTableStatistics.

@Test
public void testUpdateBasicTableStatistics() throws Exception {
    SchemaTableName tableName = temporaryTable("update_basic_table_statistics");
    try {
        doCreateEmptyTable(tableName, ORC, STATISTICS_TABLE_COLUMNS);
        testUpdateTableStatistics(tableName, EMPTY_TABLE_STATISTICS, BASIC_STATISTICS_1, BASIC_STATISTICS_2);
    } finally {
        dropTable(tableName);
    }
}
Also used : CatalogSchemaTableName(io.trino.spi.connector.CatalogSchemaTableName) SchemaTableName(io.trino.spi.connector.SchemaTableName) Test(org.testng.annotations.Test)

Example 44 with SchemaTableName

use of io.trino.spi.connector.SchemaTableName in project trino by trinodb.

the class AbstractTestHive method testPartitionStatisticsSampling.

protected void testPartitionStatisticsSampling(List<ColumnMetadata> columns, PartitionStatistics statistics) throws Exception {
    SchemaTableName tableName = temporaryTable("test_partition_statistics_sampling");
    try {
        createDummyPartitionedTable(tableName, columns);
        HiveMetastoreClosure metastoreClient = new HiveMetastoreClosure(getMetastoreClient());
        metastoreClient.updatePartitionStatistics(tableName.getSchemaName(), tableName.getTableName(), "ds=2016-01-01", actualStatistics -> statistics);
        metastoreClient.updatePartitionStatistics(tableName.getSchemaName(), tableName.getTableName(), "ds=2016-01-02", actualStatistics -> statistics);
        try (Transaction transaction = newTransaction()) {
            ConnectorSession session = newSession();
            ConnectorMetadata metadata = transaction.getMetadata();
            ConnectorTableHandle tableHandle = metadata.getTableHandle(session, tableName);
            TableStatistics unsampledStatistics = metadata.getTableStatistics(sampleSize(2), tableHandle, Constraint.alwaysTrue());
            TableStatistics sampledStatistics = metadata.getTableStatistics(sampleSize(1), tableHandle, Constraint.alwaysTrue());
            assertEquals(sampledStatistics, unsampledStatistics);
        }
    } finally {
        dropTable(tableName);
    }
}
Also used : ConnectorSession(io.trino.spi.connector.ConnectorSession) TestingConnectorSession(io.trino.testing.TestingConnectorSession) TableStatistics(io.trino.spi.statistics.TableStatistics) ConnectorMetadata(io.trino.spi.connector.ConnectorMetadata) CatalogSchemaTableName(io.trino.spi.connector.CatalogSchemaTableName) SchemaTableName(io.trino.spi.connector.SchemaTableName) ConnectorTableHandle(io.trino.spi.connector.ConnectorTableHandle)

Example 45 with SchemaTableName

use of io.trino.spi.connector.SchemaTableName in project trino by trinodb.

the class AbstractTestHive method testUpdateBasicPartitionStatistics.

@Test
public void testUpdateBasicPartitionStatistics() throws Exception {
    SchemaTableName tableName = temporaryTable("update_basic_partition_statistics");
    try {
        createDummyPartitionedTable(tableName, STATISTICS_PARTITIONED_TABLE_COLUMNS);
        testUpdatePartitionStatistics(tableName, EMPTY_TABLE_STATISTICS, ImmutableList.of(BASIC_STATISTICS_1, BASIC_STATISTICS_2), ImmutableList.of(BASIC_STATISTICS_2, BASIC_STATISTICS_1));
    } finally {
        dropTable(tableName);
    }
}
Also used : CatalogSchemaTableName(io.trino.spi.connector.CatalogSchemaTableName) SchemaTableName(io.trino.spi.connector.SchemaTableName) Test(org.testng.annotations.Test)

Aggregations

SchemaTableName (io.trino.spi.connector.SchemaTableName)446 Test (org.testng.annotations.Test)212 ImmutableList (com.google.common.collect.ImmutableList)131 ImmutableMap (com.google.common.collect.ImmutableMap)106 List (java.util.List)102 TrinoException (io.trino.spi.TrinoException)100 ImmutableList.toImmutableList (com.google.common.collect.ImmutableList.toImmutableList)98 ConnectorSession (io.trino.spi.connector.ConnectorSession)98 CatalogSchemaTableName (io.trino.spi.connector.CatalogSchemaTableName)92 TableNotFoundException (io.trino.spi.connector.TableNotFoundException)89 ColumnMetadata (io.trino.spi.connector.ColumnMetadata)86 ConnectorTableMetadata (io.trino.spi.connector.ConnectorTableMetadata)86 Optional (java.util.Optional)78 Map (java.util.Map)67 ColumnHandle (io.trino.spi.connector.ColumnHandle)66 TupleDomain (io.trino.spi.predicate.TupleDomain)59 ConnectorMetadata (io.trino.spi.connector.ConnectorMetadata)58 Path (org.apache.hadoop.fs.Path)55 ConnectorTableHandle (io.trino.spi.connector.ConnectorTableHandle)53 ImmutableSet (com.google.common.collect.ImmutableSet)52