Search in sources :

Example 1 with DBI

use of org.skife.jdbi.v2.DBI in project druid by druid-io.

the class SQLMetadataSegmentManager method enableDatasource.

@Override
public boolean enableDatasource(final String ds) {
    try {
        final IDBI dbi = connector.getDBI();
        VersionedIntervalTimeline<String, DataSegment> segmentTimeline = connector.inReadOnlyTransaction(new TransactionCallback<VersionedIntervalTimeline<String, DataSegment>>() {

            @Override
            public VersionedIntervalTimeline<String, DataSegment> inTransaction(Handle handle, TransactionStatus status) throws Exception {
                return handle.createQuery(String.format("SELECT payload FROM %s WHERE dataSource = :dataSource", getSegmentsTable())).setFetchSize(connector.getStreamingFetchSize()).bind("dataSource", ds).map(ByteArrayMapper.FIRST).fold(new VersionedIntervalTimeline<String, DataSegment>(Ordering.natural()), new Folder3<VersionedIntervalTimeline<String, DataSegment>, byte[]>() {

                    @Override
                    public VersionedIntervalTimeline<String, DataSegment> fold(VersionedIntervalTimeline<String, DataSegment> timeline, byte[] payload, FoldController foldController, StatementContext statementContext) throws SQLException {
                        try {
                            final DataSegment segment = DATA_SEGMENT_INTERNER.intern(jsonMapper.readValue(payload, DataSegment.class));
                            timeline.add(segment.getInterval(), segment.getVersion(), segment.getShardSpec().createChunk(segment));
                            return timeline;
                        } catch (Exception e) {
                            throw new SQLException(e.toString());
                        }
                    }
                });
            }
        });
        final List<DataSegment> segments = Lists.newArrayList();
        for (TimelineObjectHolder<String, DataSegment> objectHolder : segmentTimeline.lookup(new Interval("0000-01-01/3000-01-01"))) {
            for (PartitionChunk<DataSegment> partitionChunk : objectHolder.getObject()) {
                segments.add(partitionChunk.getObject());
            }
        }
        if (segments.isEmpty()) {
            log.warn("No segments found in the database!");
            return false;
        }
        dbi.withHandle(new HandleCallback<Void>() {

            @Override
            public Void withHandle(Handle handle) throws Exception {
                Batch batch = handle.createBatch();
                for (DataSegment segment : segments) {
                    batch.add(String.format("UPDATE %s SET used=true WHERE id = '%s'", getSegmentsTable(), segment.getIdentifier()));
                }
                batch.execute();
                return null;
            }
        });
    } catch (Exception e) {
        log.error(e, "Exception enabling datasource %s", ds);
        return false;
    }
    return true;
}
Also used : IDBI(org.skife.jdbi.v2.IDBI) SQLException(java.sql.SQLException) TransactionStatus(org.skife.jdbi.v2.TransactionStatus) DataSegment(io.druid.timeline.DataSegment) SQLException(java.sql.SQLException) IOException(java.io.IOException) Handle(org.skife.jdbi.v2.Handle) StatementContext(org.skife.jdbi.v2.StatementContext) FoldController(org.skife.jdbi.v2.FoldController) Batch(org.skife.jdbi.v2.Batch) VersionedIntervalTimeline(io.druid.timeline.VersionedIntervalTimeline) Folder3(org.skife.jdbi.v2.Folder3) Interval(org.joda.time.Interval)

Example 2 with DBI

use of org.skife.jdbi.v2.DBI in project druid by druid-io.

the class SQLMetadataSegmentPublisher method publishSegment.

@VisibleForTesting
void publishSegment(final String identifier, final String dataSource, final String createdDate, final String start, final String end, final boolean partitioned, final String version, final boolean used, final byte[] payload) {
    try {
        final DBI dbi = connector.getDBI();
        List<Map<String, Object>> exists = dbi.withHandle(new HandleCallback<List<Map<String, Object>>>() {

            @Override
            public List<Map<String, Object>> withHandle(Handle handle) throws Exception {
                return handle.createQuery(String.format("SELECT id FROM %s WHERE id=:id", config.getSegmentsTable())).bind("id", identifier).list();
            }
        });
        if (!exists.isEmpty()) {
            log.info("Found [%s] in DB, not updating DB", identifier);
            return;
        }
        dbi.withHandle(new HandleCallback<Void>() {

            @Override
            public Void withHandle(Handle handle) throws Exception {
                handle.createStatement(statement).bind("id", identifier).bind("dataSource", dataSource).bind("created_date", createdDate).bind("start", start).bind("end", end).bind("partitioned", partitioned).bind("version", version).bind("used", used).bind("payload", payload).execute();
                return null;
            }
        });
    } catch (Exception e) {
        log.error(e, "Exception inserting into DB");
        throw new RuntimeException(e);
    }
}
Also used : DBI(org.skife.jdbi.v2.DBI) IOException(java.io.IOException) Handle(org.skife.jdbi.v2.Handle) List(java.util.List) Map(java.util.Map) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 3 with DBI

use of org.skife.jdbi.v2.DBI in project druid by druid-io.

the class JDBCExtractionNamespaceCacheFactory method lastUpdates.

private Long lastUpdates(CacheScheduler.EntryImpl<JDBCExtractionNamespace> id, JDBCExtractionNamespace namespace) {
    final DBI dbi = ensureDBI(id, namespace);
    final String table = namespace.getTable();
    final String tsColumn = namespace.getTsColumn();
    if (tsColumn == null) {
        return null;
    }
    final Timestamp update = dbi.withHandle(new HandleCallback<Timestamp>() {

        @Override
        public Timestamp withHandle(Handle handle) throws Exception {
            final String query = String.format("SELECT MAX(%s) FROM %s", tsColumn, table);
            return handle.createQuery(query).map(TimestampMapper.FIRST).first();
        }
    });
    return update.getTime();
}
Also used : DBI(org.skife.jdbi.v2.DBI) Timestamp(java.sql.Timestamp) SQLException(java.sql.SQLException) Handle(org.skife.jdbi.v2.Handle)

Example 4 with DBI

use of org.skife.jdbi.v2.DBI in project dropwizard by dropwizard.

the class DBIHealthCheckTest method testItTimesOutProperly.

@Test
public void testItTimesOutProperly() throws Exception {
    String validationQuery = "select 1";
    DBI dbi = mock(DBI.class);
    Handle handle = mock(Handle.class);
    when(dbi.open()).thenReturn(handle);
    Mockito.doAnswer(invocation -> {
        try {
            TimeUnit.SECONDS.sleep(10);
        } catch (Exception ignored) {
        }
        return null;
    }).when(handle).execute(validationQuery);
    ExecutorService executorService = Executors.newSingleThreadExecutor();
    DBIHealthCheck dbiHealthCheck = new DBIHealthCheck(executorService, Duration.milliseconds(5), dbi, validationQuery);
    HealthCheck.Result result = dbiHealthCheck.check();
    executorService.shutdown();
    assertThat("is unhealthy", false, is(result.isHealthy()));
}
Also used : ExecutorService(java.util.concurrent.ExecutorService) HealthCheck(com.codahale.metrics.health.HealthCheck) DBI(org.skife.jdbi.v2.DBI) Handle(org.skife.jdbi.v2.Handle) Test(org.junit.Test)

Example 5 with DBI

use of org.skife.jdbi.v2.DBI in project dropwizard by dropwizard.

the class OptionalDoubleTest method setupTests.

@Before
public void setupTests() throws IOException {
    final DataSourceFactory dataSourceFactory = new DataSourceFactory();
    dataSourceFactory.setDriverClass("org.h2.Driver");
    dataSourceFactory.setUrl("jdbc:h2:mem:optional-double-" + System.currentTimeMillis() + "?user=sa");
    dataSourceFactory.setInitialSize(1);
    final DBI dbi = new DBIFactory().build(env, dataSourceFactory, "test");
    try (Handle h = dbi.open()) {
        h.execute("CREATE TABLE test (id INT PRIMARY KEY, optional DOUBLE)");
    }
    dao = dbi.onDemand(TestDao.class);
}
Also used : DataSourceFactory(io.dropwizard.db.DataSourceFactory) DBI(org.skife.jdbi.v2.DBI) DBIFactory(io.dropwizard.jdbi.DBIFactory) Handle(org.skife.jdbi.v2.Handle) Before(org.junit.Before)

Aggregations

DBI (org.skife.jdbi.v2.DBI)53 Handle (org.skife.jdbi.v2.Handle)26 Before (org.junit.Before)17 DBIFactory (io.dropwizard.jdbi.DBIFactory)16 DataSourceFactory (io.dropwizard.db.DataSourceFactory)15 IDBI (org.skife.jdbi.v2.IDBI)15 BeforeMethod (org.testng.annotations.BeforeMethod)15 Test (org.junit.Test)7 MetadataDao (com.facebook.presto.raptor.metadata.MetadataDao)6 Namespace (net.sourceforge.argparse4j.inf.Namespace)6 ShardManager (com.facebook.presto.raptor.metadata.ShardManager)5 TypeRegistry (com.facebook.presto.type.TypeRegistry)5 Duration (io.airlift.units.Duration)5 RaptorMetadata (com.facebook.presto.raptor.RaptorMetadata)4 NodeManager (com.facebook.presto.spi.NodeManager)4 SQLException (java.sql.SQLException)4 List (java.util.List)4 NodeSupplier (com.facebook.presto.raptor.NodeSupplier)3 FileBackupStore (com.facebook.presto.raptor.backup.FileBackupStore)3 ForMetadata (com.facebook.presto.raptor.metadata.ForMetadata)3