Search in sources :

Example 1 with AsyncCassandraExecutor

use of com.netflix.titus.ext.cassandra.executor.AsyncCassandraExecutor in project titus-control-plane by Netflix.

the class CassandraUtils method writeIntoTwoColumnTable.

public static long writeIntoTwoColumnTable(Session targetSession, String table, Observable<Pair<Object, Object>> sourceData) {
    Pair<String, String> columnNames = resolveColumnNamesInTwoColumnTable(targetSession, table);
    String primaryKey = columnNames.getLeft();
    String valueColumn = columnNames.getRight();
    PreparedStatement insertStatement = targetSession.prepare(String.format("INSERT INTO %s (%s, %s) VALUES (?, ?)", table, primaryKey, valueColumn));
    AsyncCassandraExecutor executor = new AsyncCassandraExecutor(targetSession, PAGE_SIZE, SPLIT);
    long recordCount = sourceData.flatMap(pair -> {
        BoundStatement boundStatement = insertStatement.bind(pair.getLeft(), pair.getRight());
        return executor.executeUpdate(boundStatement).cast(Long.class).concatWith(Observable.just(1L));
    }, MAX_CONCURRENCY).reduce(0L, (acc, v) -> acc + v).toBlocking().firstOrDefault(null);
    return recordCount;
}
Also used : TableMetadata(com.datastax.driver.core.TableMetadata) Logger(org.slf4j.Logger) ColumnMetadata(com.datastax.driver.core.ColumnMetadata) LoggerFactory(org.slf4j.LoggerFactory) Collectors(java.util.stream.Collectors) Observable(rx.Observable) PreparedStatement(com.datastax.driver.core.PreparedStatement) List(java.util.List) BoundStatement(com.datastax.driver.core.BoundStatement) Pair(com.netflix.titus.common.util.tuple.Pair) TruncateException(com.datastax.driver.core.exceptions.TruncateException) Session(com.datastax.driver.core.Session) Preconditions(com.google.common.base.Preconditions) AsyncCassandraExecutor(com.netflix.titus.ext.cassandra.executor.AsyncCassandraExecutor) AsyncCassandraExecutor(com.netflix.titus.ext.cassandra.executor.AsyncCassandraExecutor) PreparedStatement(com.datastax.driver.core.PreparedStatement) BoundStatement(com.datastax.driver.core.BoundStatement)

Example 2 with AsyncCassandraExecutor

use of com.netflix.titus.ext.cassandra.executor.AsyncCassandraExecutor in project titus-control-plane by Netflix.

the class CassandraUtils method readTwoColumnTable.

public static Observable<Pair<Object, Object>> readTwoColumnTable(Session sourceSession, String table) {
    Pair<String, String> columnNames = resolveColumnNamesInTwoColumnTable(sourceSession, table);
    String primaryKey = columnNames.getLeft();
    String valueColumn = columnNames.getRight();
    PreparedStatement queryAllStatement = sourceSession.prepare(String.format("SELECT * FROM %s WHERE token(%s) > :min AND token(%s) <= :max", table, primaryKey, primaryKey));
    AsyncCassandraExecutor executor = new AsyncCassandraExecutor(sourceSession, PAGE_SIZE, SPLIT);
    return executor.rawRangeQuery2(primaryKey, valueColumn, queryAllStatement);
}
Also used : AsyncCassandraExecutor(com.netflix.titus.ext.cassandra.executor.AsyncCassandraExecutor) PreparedStatement(com.datastax.driver.core.PreparedStatement)

Aggregations

PreparedStatement (com.datastax.driver.core.PreparedStatement)2 AsyncCassandraExecutor (com.netflix.titus.ext.cassandra.executor.AsyncCassandraExecutor)2 BoundStatement (com.datastax.driver.core.BoundStatement)1 ColumnMetadata (com.datastax.driver.core.ColumnMetadata)1 Session (com.datastax.driver.core.Session)1 TableMetadata (com.datastax.driver.core.TableMetadata)1 TruncateException (com.datastax.driver.core.exceptions.TruncateException)1 Preconditions (com.google.common.base.Preconditions)1 Pair (com.netflix.titus.common.util.tuple.Pair)1 List (java.util.List)1 Collectors (java.util.stream.Collectors)1 Logger (org.slf4j.Logger)1 LoggerFactory (org.slf4j.LoggerFactory)1 Observable (rx.Observable)1