Search in sources :

Example 1 with DeltaLakeTableHandle

use of io.trino.plugin.deltalake.DeltaLakeTableHandle in project trino by trinodb.

the class TestDeltaLakeMetastoreStatistics method testStatisticsNaNWithMultipleFiles.

@Test
public void testStatisticsNaNWithMultipleFiles() {
    // Stats with NaN values cannot be used. This transaction combines a file with NaN min/max values with one with 0.0 min/max values
    DeltaLakeTableHandle tableHandle = registerTable("nan_multi_file");
    TableStatistics stats = deltaLakeMetastore.getTableStatistics(SESSION, tableHandle, Constraint.alwaysTrue());
    ColumnStatistics columnStatistics = stats.getColumnStatistics().get(COLUMN_HANDLE);
    assertEquals(columnStatistics.getRange(), Optional.empty());
}
Also used : ColumnStatistics(io.trino.spi.statistics.ColumnStatistics) TableStatistics(io.trino.spi.statistics.TableStatistics) DeltaLakeTableHandle(io.trino.plugin.deltalake.DeltaLakeTableHandle) Test(org.testng.annotations.Test)

Example 2 with DeltaLakeTableHandle

use of io.trino.plugin.deltalake.DeltaLakeTableHandle in project trino by trinodb.

the class TestDeltaLakeMetastoreStatistics method testStatisticsZeroAndNegativeInfinity.

@Test
public void testStatisticsZeroAndNegativeInfinity() {
    DeltaLakeTableHandle tableHandle = registerTable("zero_negative_infinity");
    TableStatistics stats = deltaLakeMetastore.getTableStatistics(SESSION, tableHandle, Constraint.alwaysTrue());
    ColumnStatistics columnStatistics = stats.getColumnStatistics().get(COLUMN_HANDLE);
    assertEquals(columnStatistics.getRange().get().getMin(), NEGATIVE_INFINITY);
    assertEquals(columnStatistics.getRange().get().getMax(), 0.0);
}
Also used : ColumnStatistics(io.trino.spi.statistics.ColumnStatistics) TableStatistics(io.trino.spi.statistics.TableStatistics) DeltaLakeTableHandle(io.trino.plugin.deltalake.DeltaLakeTableHandle) Test(org.testng.annotations.Test)

Example 3 with DeltaLakeTableHandle

use of io.trino.plugin.deltalake.DeltaLakeTableHandle in project trino by trinodb.

the class TestDeltaLakeMetastoreStatistics method testStatisticsZeroAndNaN.

@Test
public void testStatisticsZeroAndNaN() {
    // Stats with NaN values cannot be used
    DeltaLakeTableHandle tableHandle = registerTable("zero_nan");
    TableStatistics stats = deltaLakeMetastore.getTableStatistics(SESSION, tableHandle, Constraint.alwaysTrue());
    ColumnStatistics columnStatistics = stats.getColumnStatistics().get(COLUMN_HANDLE);
    assertEquals(columnStatistics.getRange().get().getMin(), 0.0);
    assertEquals(columnStatistics.getRange().get().getMax(), POSITIVE_INFINITY);
}
Also used : ColumnStatistics(io.trino.spi.statistics.ColumnStatistics) TableStatistics(io.trino.spi.statistics.TableStatistics) DeltaLakeTableHandle(io.trino.plugin.deltalake.DeltaLakeTableHandle) Test(org.testng.annotations.Test)

Example 4 with DeltaLakeTableHandle

use of io.trino.plugin.deltalake.DeltaLakeTableHandle in project trino by trinodb.

the class TestDeltaLakeMetastoreStatistics method testStatisticsZeroAndInfinity.

@Test
public void testStatisticsZeroAndInfinity() {
    DeltaLakeTableHandle tableHandle = registerTable("zero_infinity");
    TableStatistics stats = deltaLakeMetastore.getTableStatistics(SESSION, tableHandle, Constraint.alwaysTrue());
    ColumnStatistics columnStatistics = stats.getColumnStatistics().get(COLUMN_HANDLE);
    assertEquals(columnStatistics.getRange().get().getMin(), 0.0);
    assertEquals(columnStatistics.getRange().get().getMax(), POSITIVE_INFINITY);
}
Also used : ColumnStatistics(io.trino.spi.statistics.ColumnStatistics) TableStatistics(io.trino.spi.statistics.TableStatistics) DeltaLakeTableHandle(io.trino.plugin.deltalake.DeltaLakeTableHandle) Test(org.testng.annotations.Test)

Example 5 with DeltaLakeTableHandle

use of io.trino.plugin.deltalake.DeltaLakeTableHandle in project trino by trinodb.

the class TestDeltaLakeMetastoreStatistics method testStatisticsMultipleFiles.

@Test
public void testStatisticsMultipleFiles() {
    DeltaLakeTableHandle tableHandle = registerTable("basic_multi_file");
    TableStatistics stats = deltaLakeMetastore.getTableStatistics(SESSION, tableHandle, Constraint.alwaysTrue());
    ColumnStatistics columnStatistics = stats.getColumnStatistics().get(COLUMN_HANDLE);
    assertEquals(columnStatistics.getRange().get().getMin(), -42.0);
    assertEquals(columnStatistics.getRange().get().getMax(), 42.0);
    DeltaLakeTableHandle tableHandleWithUnenforcedConstraint = new DeltaLakeTableHandle(tableHandle.getSchemaName(), tableHandle.getTableName(), tableHandle.getLocation(), Optional.of(tableHandle.getMetadataEntry()), TupleDomain.all(), TupleDomain.withColumnDomains(ImmutableMap.of((DeltaLakeColumnHandle) COLUMN_HANDLE, Domain.singleValue(DOUBLE, 42.0))), tableHandle.getWriteType(), tableHandle.getProjectedColumns(), tableHandle.getUpdatedColumns(), tableHandle.getUpdateRowIdColumns(), tableHandle.getAnalyzeHandle(), 0);
    stats = deltaLakeMetastore.getTableStatistics(SESSION, tableHandleWithUnenforcedConstraint, Constraint.alwaysTrue());
    columnStatistics = stats.getColumnStatistics().get(COLUMN_HANDLE);
    assertEquals(columnStatistics.getRange().get().getMin(), 0.0);
    assertEquals(columnStatistics.getRange().get().getMax(), 42.0);
}
Also used : ColumnStatistics(io.trino.spi.statistics.ColumnStatistics) TableStatistics(io.trino.spi.statistics.TableStatistics) DeltaLakeTableHandle(io.trino.plugin.deltalake.DeltaLakeTableHandle) Test(org.testng.annotations.Test)

Aggregations

DeltaLakeTableHandle (io.trino.plugin.deltalake.DeltaLakeTableHandle)18 ColumnStatistics (io.trino.spi.statistics.ColumnStatistics)15 TableStatistics (io.trino.spi.statistics.TableStatistics)15 Test (org.testng.annotations.Test)15 DeltaLakeColumnHandle (io.trino.plugin.deltalake.DeltaLakeColumnHandle)4 ColumnHandle (io.trino.spi.connector.ColumnHandle)3 ImmutableList.toImmutableList (com.google.common.collect.ImmutableList.toImmutableList)2 ImmutableSet.toImmutableSet (com.google.common.collect.ImmutableSet.toImmutableSet)2 AddFileEntry (io.trino.plugin.deltalake.transactionlog.AddFileEntry)2 TableSnapshot (io.trino.plugin.deltalake.transactionlog.TableSnapshot)2 TransactionLogAccess (io.trino.plugin.deltalake.transactionlog.TransactionLogAccess)2 TrinoException (io.trino.spi.TrinoException)2 ConnectorSession (io.trino.spi.connector.ConnectorSession)2 SchemaTableName (io.trino.spi.connector.SchemaTableName)2 IOException (java.io.IOException)2 String.format (java.lang.String.format)2 List (java.util.List)2 Objects.requireNonNull (java.util.Objects.requireNonNull)2 Set (java.util.Set)2 Path (org.apache.hadoop.fs.Path)2