Search in sources :

Example 1 with ClassLoaderSafeConnectorSplitSource

use of io.trino.plugin.base.classloader.ClassLoaderSafeConnectorSplitSource in project trino by trinodb.

the class IcebergSplitManager method getSplits.

@Override
public ConnectorSplitSource getSplits(ConnectorTransactionHandle transaction, ConnectorSession session, ConnectorTableHandle handle, SplitSchedulingStrategy splitSchedulingStrategy, DynamicFilter dynamicFilter, Constraint constraint) {
    IcebergTableHandle table = (IcebergTableHandle) handle;
    if (table.getSnapshotId().isEmpty()) {
        if (table.isRecordScannedFiles()) {
            return new FixedSplitSource(ImmutableList.of(), ImmutableList.of());
        }
        return new FixedSplitSource(ImmutableList.of());
    }
    Table icebergTable = transactionManager.get(transaction, session.getIdentity()).getIcebergTable(session, table.getSchemaTableName());
    Duration dynamicFilteringWaitTimeout = getDynamicFilteringWaitTimeout(session);
    TableScan tableScan = icebergTable.newScan().useSnapshot(table.getSnapshotId().get());
    IcebergSplitSource splitSource = new IcebergSplitSource(table, tableScan, table.getMaxScannedFileSize(), dynamicFilter, dynamicFilteringWaitTimeout, constraint, typeManager, table.isRecordScannedFiles());
    return new ClassLoaderSafeConnectorSplitSource(splitSource, Thread.currentThread().getContextClassLoader());
}
Also used : TableScan(org.apache.iceberg.TableScan) Table(org.apache.iceberg.Table) ClassLoaderSafeConnectorSplitSource(io.trino.plugin.base.classloader.ClassLoaderSafeConnectorSplitSource) FixedSplitSource(io.trino.spi.connector.FixedSplitSource) Duration(io.airlift.units.Duration)

Aggregations

Duration (io.airlift.units.Duration)1 ClassLoaderSafeConnectorSplitSource (io.trino.plugin.base.classloader.ClassLoaderSafeConnectorSplitSource)1 FixedSplitSource (io.trino.spi.connector.FixedSplitSource)1 Table (org.apache.iceberg.Table)1 TableScan (org.apache.iceberg.TableScan)1