Search in sources :

Example 1 with FQLQueryIterator

use of org.apache.cassandra.fqltool.FQLQueryIterator in project cassandra by apache.

the class Compare method compare.

public static void compare(String querylog, List<String> arguments) {
    List<ChronicleQueue> readQueues = null;
    try (ResultHandler rh = new ResultHandler(arguments, null, null);
        ChronicleQueue queryQ = SingleChronicleQueueBuilder.single(querylog).readOnly(true).build();
        FQLQueryIterator queries = new FQLQueryIterator(queryQ.createTailer(), 1)) {
        readQueues = arguments.stream().map(s -> SingleChronicleQueueBuilder.single(s).readOnly(true).build()).collect(Collectors.toList());
        List<Iterator<ResultHandler.ComparableResultSet>> its = readQueues.stream().map(q -> new StoredResultSetIterator(q.createTailer())).collect(Collectors.toList());
        while (queries.hasNext()) rh.handleResults(queries.next(), resultSets(its));
    } catch (Exception e) {
        throw new RuntimeException(e);
    } finally {
        if (readQueues != null)
            readQueues.forEach(Closeable::close);
    }
}
Also used : ExcerptTailer(net.openhft.chronicle.queue.ExcerptTailer) Iterator(java.util.Iterator) ChronicleQueue(net.openhft.chronicle.queue.ChronicleQueue) AbstractIterator(com.google.common.collect.AbstractIterator) Closeable(net.openhft.chronicle.core.io.Closeable) StoredResultSet(org.apache.cassandra.fqltool.StoredResultSet) Collectors(java.util.stream.Collectors) ArrayList(java.util.ArrayList) Command(io.airlift.airline.Command) SingleChronicleQueueBuilder(net.openhft.chronicle.queue.impl.single.SingleChronicleQueueBuilder) List(java.util.List) Option(io.airlift.airline.Option) ResultHandler(org.apache.cassandra.fqltool.ResultHandler) VisibleForTesting(com.google.common.annotations.VisibleForTesting) FQLQueryIterator(org.apache.cassandra.fqltool.FQLQueryIterator) Arguments(io.airlift.airline.Arguments) ChronicleQueue(net.openhft.chronicle.queue.ChronicleQueue) Iterator(java.util.Iterator) AbstractIterator(com.google.common.collect.AbstractIterator) FQLQueryIterator(org.apache.cassandra.fqltool.FQLQueryIterator) ResultHandler(org.apache.cassandra.fqltool.ResultHandler) FQLQueryIterator(org.apache.cassandra.fqltool.FQLQueryIterator)

Example 2 with FQLQueryIterator

use of org.apache.cassandra.fqltool.FQLQueryIterator in project cassandra by apache.

the class Replay method replay.

public static void replay(String keyspace, List<String> arguments, List<String> targetHosts, List<File> resultPaths, String queryStorePath, boolean replayDDLStatements) {
    // how many fql queries should we read in to memory to be able to sort them?
    int readAhead = 200;
    List<ChronicleQueue> readQueues = null;
    List<FQLQueryIterator> iterators = null;
    List<Predicate<FQLQuery>> filters = new ArrayList<>();
    if (keyspace != null)
        filters.add(fqlQuery -> fqlQuery.keyspace() == null || fqlQuery.keyspace().equals(keyspace));
    if (!replayDDLStatements)
        filters.add(fqlQuery -> {
            boolean notDDLStatement = !fqlQuery.isDDLStatement();
            if (!notDDLStatement)
                logger.info("Excluding DDL statement from replaying: {}", ((FQLQuery.Single) fqlQuery).query);
            return notDDLStatement;
        });
    try {
        readQueues = arguments.stream().map(s -> SingleChronicleQueueBuilder.single(s).readOnly(true).build()).collect(Collectors.toList());
        iterators = readQueues.stream().map(ChronicleQueue::createTailer).map(tailer -> new FQLQueryIterator(tailer, readAhead)).collect(Collectors.toList());
        try (MergeIterator<FQLQuery, List<FQLQuery>> iter = MergeIterator.get(iterators, FQLQuery::compareTo, new Reducer());
            QueryReplayer replayer = new QueryReplayer(iter, targetHosts, resultPaths, filters, queryStorePath)) {
            replayer.replay();
        }
    } catch (Exception e) {
        throw new RuntimeException(e);
    } finally {
        if (iterators != null)
            iterators.forEach(AbstractIterator::close);
        if (readQueues != null)
            readQueues.forEach(Closeable::close);
    }
}
Also used : MergeIterator(org.apache.cassandra.utils.MergeIterator) Logger(org.slf4j.Logger) Predicate(java.util.function.Predicate) ChronicleQueue(net.openhft.chronicle.queue.ChronicleQueue) LoggerFactory(org.slf4j.LoggerFactory) Closeable(net.openhft.chronicle.core.io.Closeable) Collectors(java.util.stream.Collectors) QueryReplayer(org.apache.cassandra.fqltool.QueryReplayer) File(java.io.File) ArrayList(java.util.ArrayList) Command(io.airlift.airline.Command) SingleChronicleQueueBuilder(net.openhft.chronicle.queue.impl.single.SingleChronicleQueueBuilder) List(java.util.List) Option(io.airlift.airline.Option) FQLQuery(org.apache.cassandra.fqltool.FQLQuery) VisibleForTesting(com.google.common.annotations.VisibleForTesting) FQLQueryIterator(org.apache.cassandra.fqltool.FQLQueryIterator) Arguments(io.airlift.airline.Arguments) AbstractIterator(org.apache.cassandra.utils.AbstractIterator) FQLQuery(org.apache.cassandra.fqltool.FQLQuery) ArrayList(java.util.ArrayList) Predicate(java.util.function.Predicate) ChronicleQueue(net.openhft.chronicle.queue.ChronicleQueue) ArrayList(java.util.ArrayList) List(java.util.List) QueryReplayer(org.apache.cassandra.fqltool.QueryReplayer) FQLQueryIterator(org.apache.cassandra.fqltool.FQLQueryIterator)

Aggregations

VisibleForTesting (com.google.common.annotations.VisibleForTesting)2 Arguments (io.airlift.airline.Arguments)2 Command (io.airlift.airline.Command)2 Option (io.airlift.airline.Option)2 ArrayList (java.util.ArrayList)2 List (java.util.List)2 Collectors (java.util.stream.Collectors)2 Closeable (net.openhft.chronicle.core.io.Closeable)2 ChronicleQueue (net.openhft.chronicle.queue.ChronicleQueue)2 SingleChronicleQueueBuilder (net.openhft.chronicle.queue.impl.single.SingleChronicleQueueBuilder)2 FQLQueryIterator (org.apache.cassandra.fqltool.FQLQueryIterator)2 AbstractIterator (com.google.common.collect.AbstractIterator)1 File (java.io.File)1 Iterator (java.util.Iterator)1 Predicate (java.util.function.Predicate)1 ExcerptTailer (net.openhft.chronicle.queue.ExcerptTailer)1 FQLQuery (org.apache.cassandra.fqltool.FQLQuery)1 QueryReplayer (org.apache.cassandra.fqltool.QueryReplayer)1 ResultHandler (org.apache.cassandra.fqltool.ResultHandler)1 StoredResultSet (org.apache.cassandra.fqltool.StoredResultSet)1