Search in sources :

Example 1 with DaoSupplier

use of com.facebook.presto.raptor.util.DaoSupplier in project presto by prestodb.

the class TestRaptorSplitManager method setup.

@BeforeMethod
public void setup() throws Exception {
    FunctionAndTypeManager functionAndTypeManager = createTestFunctionAndTypeManager();
    DBI dbi = new DBI("jdbc:h2:mem:test" + System.nanoTime() + "_" + ThreadLocalRandom.current().nextInt());
    dbi.registerMapper(new TableColumn.Mapper(functionAndTypeManager));
    dummyHandle = dbi.open();
    createTablesWithRetry(dbi);
    temporary = createTempDir();
    AssignmentLimiter assignmentLimiter = new AssignmentLimiter(ImmutableSet::of, systemTicker(), new MetadataConfig());
    shardManager = new DatabaseShardManager(dbi, new DaoSupplier<>(dbi, ShardDao.class), ImmutableSet::of, assignmentLimiter, systemTicker(), new Duration(0, MINUTES));
    TestingNodeManager nodeManager = new TestingNodeManager();
    NodeSupplier nodeSupplier = nodeManager::getWorkerNodes;
    String nodeName = UUID.randomUUID().toString();
    nodeManager.addNode(new InternalNode(nodeName, new URI("http://127.0.0.1/"), NodeVersion.UNKNOWN, false));
    RaptorConnectorId connectorId = new RaptorConnectorId("raptor");
    metadata = new RaptorMetadata(connectorId.toString(), dbi, shardManager, createTestFunctionAndTypeManager());
    metadata.createTable(SESSION, TEST_TABLE, false);
    tableHandle = metadata.getTableHandle(SESSION, TEST_TABLE.getTable());
    List<ShardInfo> shards = ImmutableList.<ShardInfo>builder().add(shardInfo(UUID.randomUUID(), nodeName)).add(shardInfo(UUID.randomUUID(), nodeName)).add(shardInfo(UUID.randomUUID(), nodeName)).add(shardInfo(UUID.randomUUID(), nodeName)).build();
    tableId = ((RaptorTableHandle) tableHandle).getTableId();
    List<ColumnInfo> columns = metadata.getColumnHandles(SESSION, tableHandle).values().stream().map(RaptorColumnHandle.class::cast).map(ColumnInfo::fromHandle).collect(toList());
    long transactionId = shardManager.beginTransaction();
    shardManager.commitShards(transactionId, tableId, columns, shards, Optional.empty(), 0);
    raptorSplitManager = new RaptorSplitManager(connectorId, nodeSupplier, shardManager, false);
}
Also used : RaptorColumnHandle(com.facebook.presto.raptor.RaptorColumnHandle) RaptorMetadata(com.facebook.presto.raptor.RaptorMetadata) DBI(org.skife.jdbi.v2.DBI) Duration(io.airlift.units.Duration) DaoSupplier(com.facebook.presto.raptor.util.DaoSupplier) URI(java.net.URI) FunctionAndTypeManager(com.facebook.presto.metadata.FunctionAndTypeManager) FunctionAndTypeManager.createTestFunctionAndTypeManager(com.facebook.presto.metadata.FunctionAndTypeManager.createTestFunctionAndTypeManager) ImmutableSet(com.google.common.collect.ImmutableSet) TestingNodeManager(com.facebook.presto.testing.TestingNodeManager) InternalNode(com.facebook.presto.metadata.InternalNode) RaptorConnectorId(com.facebook.presto.raptor.RaptorConnectorId) NodeSupplier(com.facebook.presto.raptor.NodeSupplier) RaptorSplitManager(com.facebook.presto.raptor.RaptorSplitManager) BeforeMethod(org.testng.annotations.BeforeMethod)

Example 2 with DaoSupplier

use of com.facebook.presto.raptor.util.DaoSupplier in project presto by prestodb.

the class TestShardCleaner method setup.

@BeforeMethod
public void setup() {
    dbi = new DBI("jdbc:h2:mem:test" + System.nanoTime() + "_" + ThreadLocalRandom.current().nextInt());
    dummyHandle = dbi.open();
    createTablesWithRetry(dbi);
    temporary = createTempDir();
    File directory = new File(temporary, "data");
    storageService = new LocalFileStorageService(new LocalOrcDataEnvironment(), directory.toURI());
    storageService.start();
    File backupDirectory = new File(temporary, "backup");
    backupStore = new FileBackupStore(backupDirectory);
    ((FileBackupStore) backupStore).start();
    ticker = new TestingTicker();
    ShardCleanerConfig config = new ShardCleanerConfig();
    cleaner = new ShardCleaner(new DaoSupplier<>(dbi, H2ShardDao.class), "node1", true, ticker, storageService, Optional.of(backupStore), new LocalOrcDataEnvironment(), config.getMaxTransactionAge(), config.getTransactionCleanerInterval(), config.getLocalCleanerInterval(), config.getLocalCleanTime(), config.getBackupCleanerInterval(), config.getBackupCleanTime(), config.getBackupDeletionThreads(), config.getMaxCompletedTransactionAge());
}
Also used : FileBackupStore(com.facebook.presto.raptor.backup.FileBackupStore) TestingTicker(com.facebook.airlift.testing.TestingTicker) LocalOrcDataEnvironment(com.facebook.presto.raptor.filesystem.LocalOrcDataEnvironment) IDBI(org.skife.jdbi.v2.IDBI) DBI(org.skife.jdbi.v2.DBI) DaoSupplier(com.facebook.presto.raptor.util.DaoSupplier) File(java.io.File) LocalFileStorageService(com.facebook.presto.raptor.filesystem.LocalFileStorageService) BeforeMethod(org.testng.annotations.BeforeMethod)

Example 3 with DaoSupplier

use of com.facebook.presto.raptor.util.DaoSupplier in project presto by prestodb.

the class TestDatabaseShardManager method createShardManager.

public static ShardManager createShardManager(IDBI dbi, NodeSupplier nodeSupplier, Ticker ticker) {
    DaoSupplier<ShardDao> shardDaoSupplier = new DaoSupplier<>(dbi, H2ShardDao.class);
    AssignmentLimiter assignmentLimiter = new AssignmentLimiter(nodeSupplier, ticker, new MetadataConfig());
    return new DatabaseShardManager(dbi, shardDaoSupplier, nodeSupplier, assignmentLimiter, ticker, new Duration(1, DAYS));
}
Also used : Duration(io.airlift.units.Duration) DaoSupplier(com.facebook.presto.raptor.util.DaoSupplier)

Aggregations

DaoSupplier (com.facebook.presto.raptor.util.DaoSupplier)3 Duration (io.airlift.units.Duration)2 DBI (org.skife.jdbi.v2.DBI)2 BeforeMethod (org.testng.annotations.BeforeMethod)2 TestingTicker (com.facebook.airlift.testing.TestingTicker)1 FunctionAndTypeManager (com.facebook.presto.metadata.FunctionAndTypeManager)1 FunctionAndTypeManager.createTestFunctionAndTypeManager (com.facebook.presto.metadata.FunctionAndTypeManager.createTestFunctionAndTypeManager)1 InternalNode (com.facebook.presto.metadata.InternalNode)1 NodeSupplier (com.facebook.presto.raptor.NodeSupplier)1 RaptorColumnHandle (com.facebook.presto.raptor.RaptorColumnHandle)1 RaptorConnectorId (com.facebook.presto.raptor.RaptorConnectorId)1 RaptorMetadata (com.facebook.presto.raptor.RaptorMetadata)1 RaptorSplitManager (com.facebook.presto.raptor.RaptorSplitManager)1 FileBackupStore (com.facebook.presto.raptor.backup.FileBackupStore)1 LocalFileStorageService (com.facebook.presto.raptor.filesystem.LocalFileStorageService)1 LocalOrcDataEnvironment (com.facebook.presto.raptor.filesystem.LocalOrcDataEnvironment)1 TestingNodeManager (com.facebook.presto.testing.TestingNodeManager)1 ImmutableSet (com.google.common.collect.ImmutableSet)1 File (java.io.File)1 URI (java.net.URI)1