Search in sources :

Example 6 with ConnectorTransactionHandle

use of com.facebook.presto.spi.connector.ConnectorTransactionHandle in project presto by prestodb.

the class RaptorPageSourceProvider method createPageSource.

@Override
public ConnectorPageSource createPageSource(ConnectorTransactionHandle transactionHandle, ConnectorSession session, ConnectorSplit split, List<ColumnHandle> columns, SplitContext splitContext) {
    RaptorSplit raptorSplit = (RaptorSplit) split;
    OptionalInt bucketNumber = raptorSplit.getBucketNumber();
    TupleDomain<RaptorColumnHandle> predicate = raptorSplit.getEffectivePredicate();
    ReaderAttributes attributes = ReaderAttributes.from(session);
    OptionalLong transactionId = raptorSplit.getTransactionId();
    Optional<Map<String, Type>> columnTypes = raptorSplit.getColumnTypes();
    boolean tableSupportsDeltaDelete = raptorSplit.isTableSupportsDeltaDelete();
    HdfsContext context = new HdfsContext(session);
    Map<UUID, UUID> shardDeltaMap = raptorSplit.getShardDeltaMap();
    if (raptorSplit.getShardUuids().size() == 1) {
        UUID shardUuid = raptorSplit.getShardUuids().iterator().next();
        return createPageSource(context, DEFAULT_HIVE_FILE_CONTEXT, shardUuid, Optional.ofNullable(shardDeltaMap.get(shardUuid)), tableSupportsDeltaDelete, bucketNumber, columns, predicate, attributes, transactionId, columnTypes);
    }
    Iterator<ConnectorPageSource> iterator = raptorSplit.getShardUuids().stream().map(shardUuid -> createPageSource(context, DEFAULT_HIVE_FILE_CONTEXT, shardUuid, Optional.ofNullable(shardDeltaMap.get(shardUuid)), tableSupportsDeltaDelete, bucketNumber, columns, predicate, attributes, transactionId, columnTypes)).iterator();
    return new ConcatPageSource(iterator);
}
Also used : OptionalInt(java.util.OptionalInt) ConnectorTransactionHandle(com.facebook.presto.spi.connector.ConnectorTransactionHandle) Inject(javax.inject.Inject) OptionalLong(java.util.OptionalLong) SplitContext(com.facebook.presto.spi.SplitContext) Map(java.util.Map) Objects.requireNonNull(java.util.Objects.requireNonNull) ReaderAttributes(com.facebook.presto.raptor.storage.ReaderAttributes) DEFAULT_HIVE_FILE_CONTEXT(com.facebook.presto.hive.HiveFileContext.DEFAULT_HIVE_FILE_CONTEXT) StorageManager(com.facebook.presto.raptor.storage.StorageManager) HdfsContext(com.facebook.presto.hive.HdfsContext) ConnectorPageSourceProvider(com.facebook.presto.spi.connector.ConnectorPageSourceProvider) Type(com.facebook.presto.common.type.Type) ConcatPageSource(com.facebook.presto.raptor.util.ConcatPageSource) Iterator(java.util.Iterator) HiveFileContext(com.facebook.presto.hive.HiveFileContext) UUID(java.util.UUID) TupleDomain(com.facebook.presto.common.predicate.TupleDomain) ConnectorSession(com.facebook.presto.spi.ConnectorSession) ConnectorSplit(com.facebook.presto.spi.ConnectorSplit) List(java.util.List) Collectors.toList(java.util.stream.Collectors.toList) ConnectorPageSource(com.facebook.presto.spi.ConnectorPageSource) ColumnHandle(com.facebook.presto.spi.ColumnHandle) Optional(java.util.Optional) ConcatPageSource(com.facebook.presto.raptor.util.ConcatPageSource) OptionalInt(java.util.OptionalInt) ConnectorPageSource(com.facebook.presto.spi.ConnectorPageSource) ReaderAttributes(com.facebook.presto.raptor.storage.ReaderAttributes) OptionalLong(java.util.OptionalLong) HdfsContext(com.facebook.presto.hive.HdfsContext) UUID(java.util.UUID) Map(java.util.Map)

Example 7 with ConnectorTransactionHandle

use of com.facebook.presto.spi.connector.ConnectorTransactionHandle in project presto by prestodb.

the class TestJdbcRecordSetProvider method testGetRecordSet.

@Test
public void testGetRecordSet() {
    ConnectorTransactionHandle transaction = new JdbcTransactionHandle();
    JdbcRecordSetProvider recordSetProvider = new JdbcRecordSetProvider(jdbcClient);
    RecordSet recordSet = recordSetProvider.getRecordSet(transaction, SESSION, split, ImmutableList.of(textColumn, textShortColumn, valueColumn));
    assertNotNull(recordSet, "recordSet is null");
    RecordCursor cursor = recordSet.cursor();
    assertNotNull(cursor, "cursor is null");
    Map<String, Long> data = new LinkedHashMap<>();
    while (cursor.advanceNextPosition()) {
        data.put(cursor.getSlice(0).toStringUtf8(), cursor.getLong(2));
        assertEquals(cursor.getSlice(0), cursor.getSlice(1));
    }
    assertEquals(data, ImmutableMap.<String, Long>builder().put("one", 1L).put("two", 2L).put("three", 3L).put("ten", 10L).put("eleven", 11L).put("twelve", 12L).build());
}
Also used : RecordCursor(com.facebook.presto.spi.RecordCursor) ConnectorTransactionHandle(com.facebook.presto.spi.connector.ConnectorTransactionHandle) RecordSet(com.facebook.presto.spi.RecordSet) LinkedHashMap(java.util.LinkedHashMap) Test(org.testng.annotations.Test)

Example 8 with ConnectorTransactionHandle

use of com.facebook.presto.spi.connector.ConnectorTransactionHandle in project presto by prestodb.

the class MetadataManager method getNewTableLayout.

@Override
public Optional<NewTableLayout> getNewTableLayout(Session session, String catalogName, ConnectorTableMetadata tableMetadata) {
    CatalogMetadata catalogMetadata = getCatalogMetadataForWrite(session, catalogName);
    ConnectorId connectorId = catalogMetadata.getConnectorId();
    ConnectorMetadata metadata = catalogMetadata.getMetadata();
    ConnectorTransactionHandle transactionHandle = catalogMetadata.getTransactionHandleFor(connectorId);
    ConnectorSession connectorSession = session.toConnectorSession(connectorId);
    return metadata.getNewTableLayout(connectorSession, tableMetadata).map(layout -> new NewTableLayout(connectorId, transactionHandle, layout));
}
Also used : ConnectorTransactionHandle(com.facebook.presto.spi.connector.ConnectorTransactionHandle) ConnectorSession(com.facebook.presto.spi.ConnectorSession) ConnectorMetadata(com.facebook.presto.spi.connector.ConnectorMetadata) ConnectorId(com.facebook.presto.spi.ConnectorId)

Example 9 with ConnectorTransactionHandle

use of com.facebook.presto.spi.connector.ConnectorTransactionHandle in project presto by prestodb.

the class MetadataManager method getPreferredShuffleLayoutForNewTable.

@Override
public Optional<NewTableLayout> getPreferredShuffleLayoutForNewTable(Session session, String catalogName, ConnectorTableMetadata tableMetadata) {
    CatalogMetadata catalogMetadata = getCatalogMetadataForWrite(session, catalogName);
    ConnectorId connectorId = catalogMetadata.getConnectorId();
    ConnectorMetadata metadata = catalogMetadata.getMetadata();
    ConnectorTransactionHandle transactionHandle = catalogMetadata.getTransactionHandleFor(connectorId);
    ConnectorSession connectorSession = session.toConnectorSession(connectorId);
    return metadata.getPreferredShuffleLayoutForNewTable(connectorSession, tableMetadata).map(layout -> new NewTableLayout(connectorId, transactionHandle, layout));
}
Also used : ConnectorTransactionHandle(com.facebook.presto.spi.connector.ConnectorTransactionHandle) ConnectorSession(com.facebook.presto.spi.ConnectorSession) ConnectorMetadata(com.facebook.presto.spi.connector.ConnectorMetadata) ConnectorId(com.facebook.presto.spi.ConnectorId)

Example 10 with ConnectorTransactionHandle

use of com.facebook.presto.spi.connector.ConnectorTransactionHandle in project presto by prestodb.

the class MetadataManager method beginInsert.

@Override
public InsertTableHandle beginInsert(Session session, TableHandle tableHandle) {
    ConnectorId connectorId = tableHandle.getConnectorId();
    CatalogMetadata catalogMetadata = getCatalogMetadataForWrite(session, connectorId);
    ConnectorMetadata metadata = catalogMetadata.getMetadata();
    ConnectorTransactionHandle transactionHandle = catalogMetadata.getTransactionHandleFor(connectorId);
    ConnectorInsertTableHandle handle = metadata.beginInsert(session.toConnectorSession(connectorId), tableHandle.getConnectorHandle());
    return new InsertTableHandle(tableHandle.getConnectorId(), transactionHandle, handle);
}
Also used : ConnectorInsertTableHandle(com.facebook.presto.spi.ConnectorInsertTableHandle) ConnectorTransactionHandle(com.facebook.presto.spi.connector.ConnectorTransactionHandle) ConnectorMetadata(com.facebook.presto.spi.connector.ConnectorMetadata) ConnectorInsertTableHandle(com.facebook.presto.spi.ConnectorInsertTableHandle) ConnectorId(com.facebook.presto.spi.ConnectorId)

Aggregations

ConnectorTransactionHandle (com.facebook.presto.spi.connector.ConnectorTransactionHandle)40 ConnectorSession (com.facebook.presto.spi.ConnectorSession)18 ColumnHandle (com.facebook.presto.spi.ColumnHandle)13 ConnectorId (com.facebook.presto.spi.ConnectorId)12 ConnectorSplit (com.facebook.presto.spi.ConnectorSplit)12 List (java.util.List)12 Objects.requireNonNull (java.util.Objects.requireNonNull)12 ImmutableList (com.google.common.collect.ImmutableList)11 ConnectorMetadata (com.facebook.presto.spi.connector.ConnectorMetadata)10 Inject (javax.inject.Inject)9 ConnectorTableHandle (com.facebook.presto.spi.ConnectorTableHandle)8 Optional (java.util.Optional)8 ConnectorTableLayoutHandle (com.facebook.presto.spi.ConnectorTableLayoutHandle)7 SchemaTableName (com.facebook.presto.spi.SchemaTableName)7 ImmutableMap (com.google.common.collect.ImmutableMap)7 TupleDomain (com.facebook.presto.common.predicate.TupleDomain)6 Type (com.facebook.presto.common.type.Type)6 Map (java.util.Map)5 ConnectorTableMetadata (com.facebook.presto.spi.ConnectorTableMetadata)4 NodeManager (com.facebook.presto.spi.NodeManager)4