Search in sources :

Example 1 with AbstractIterator

use of org.apache.cassandra.utils.AbstractIterator in project cassandra by apache.

the class HintsBuffer method consumingHintsIterator.

/**
     * Coverts the queue of offsets for the selected host id into an iterator of hints encoded as ByteBuffers.
     */
Iterator<ByteBuffer> consumingHintsIterator(UUID hostId) {
    final Queue<Integer> bufferOffsets = offsets.get(hostId);
    if (bufferOffsets == null)
        return Collections.emptyIterator();
    return new AbstractIterator<ByteBuffer>() {

        private final ByteBuffer flyweight = slab.duplicate();

        protected ByteBuffer computeNext() {
            Integer offset = bufferOffsets.poll();
            if (offset == null)
                return endOfData();
            int totalSize = slab.getInt(offset) + ENTRY_OVERHEAD_SIZE;
            return (ByteBuffer) flyweight.clear().position(offset).limit(offset + totalSize);
        }
    };
}
Also used : AtomicInteger(java.util.concurrent.atomic.AtomicInteger) AbstractIterator(org.apache.cassandra.utils.AbstractIterator) ByteBuffer(java.nio.ByteBuffer)

Example 2 with AbstractIterator

use of org.apache.cassandra.utils.AbstractIterator in project cassandra by apache.

the class Replay method replay.

public static void replay(String keyspace, List<String> arguments, List<String> targetHosts, List<File> resultPaths, String queryStorePath, boolean replayDDLStatements) {
    // how many fql queries should we read in to memory to be able to sort them?
    int readAhead = 200;
    List<ChronicleQueue> readQueues = null;
    List<FQLQueryIterator> iterators = null;
    List<Predicate<FQLQuery>> filters = new ArrayList<>();
    if (keyspace != null)
        filters.add(fqlQuery -> fqlQuery.keyspace() == null || fqlQuery.keyspace().equals(keyspace));
    if (!replayDDLStatements)
        filters.add(fqlQuery -> {
            boolean notDDLStatement = !fqlQuery.isDDLStatement();
            if (!notDDLStatement)
                logger.info("Excluding DDL statement from replaying: {}", ((FQLQuery.Single) fqlQuery).query);
            return notDDLStatement;
        });
    try {
        readQueues = arguments.stream().map(s -> SingleChronicleQueueBuilder.single(s).readOnly(true).build()).collect(Collectors.toList());
        iterators = readQueues.stream().map(ChronicleQueue::createTailer).map(tailer -> new FQLQueryIterator(tailer, readAhead)).collect(Collectors.toList());
        try (MergeIterator<FQLQuery, List<FQLQuery>> iter = MergeIterator.get(iterators, FQLQuery::compareTo, new Reducer());
            QueryReplayer replayer = new QueryReplayer(iter, targetHosts, resultPaths, filters, queryStorePath)) {
            replayer.replay();
        }
    } catch (Exception e) {
        throw new RuntimeException(e);
    } finally {
        if (iterators != null)
            iterators.forEach(AbstractIterator::close);
        if (readQueues != null)
            readQueues.forEach(Closeable::close);
    }
}
Also used : MergeIterator(org.apache.cassandra.utils.MergeIterator) Logger(org.slf4j.Logger) Predicate(java.util.function.Predicate) ChronicleQueue(net.openhft.chronicle.queue.ChronicleQueue) LoggerFactory(org.slf4j.LoggerFactory) Closeable(net.openhft.chronicle.core.io.Closeable) Collectors(java.util.stream.Collectors) QueryReplayer(org.apache.cassandra.fqltool.QueryReplayer) File(java.io.File) ArrayList(java.util.ArrayList) Command(io.airlift.airline.Command) SingleChronicleQueueBuilder(net.openhft.chronicle.queue.impl.single.SingleChronicleQueueBuilder) List(java.util.List) Option(io.airlift.airline.Option) FQLQuery(org.apache.cassandra.fqltool.FQLQuery) VisibleForTesting(com.google.common.annotations.VisibleForTesting) FQLQueryIterator(org.apache.cassandra.fqltool.FQLQueryIterator) Arguments(io.airlift.airline.Arguments) AbstractIterator(org.apache.cassandra.utils.AbstractIterator) FQLQuery(org.apache.cassandra.fqltool.FQLQuery) ArrayList(java.util.ArrayList) Predicate(java.util.function.Predicate) ChronicleQueue(net.openhft.chronicle.queue.ChronicleQueue) ArrayList(java.util.ArrayList) List(java.util.List) QueryReplayer(org.apache.cassandra.fqltool.QueryReplayer) FQLQueryIterator(org.apache.cassandra.fqltool.FQLQueryIterator)

Aggregations

AbstractIterator (org.apache.cassandra.utils.AbstractIterator)2 VisibleForTesting (com.google.common.annotations.VisibleForTesting)1 Arguments (io.airlift.airline.Arguments)1 Command (io.airlift.airline.Command)1 Option (io.airlift.airline.Option)1 File (java.io.File)1 ByteBuffer (java.nio.ByteBuffer)1 ArrayList (java.util.ArrayList)1 List (java.util.List)1 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)1 Predicate (java.util.function.Predicate)1 Collectors (java.util.stream.Collectors)1 Closeable (net.openhft.chronicle.core.io.Closeable)1 ChronicleQueue (net.openhft.chronicle.queue.ChronicleQueue)1 SingleChronicleQueueBuilder (net.openhft.chronicle.queue.impl.single.SingleChronicleQueueBuilder)1 FQLQuery (org.apache.cassandra.fqltool.FQLQuery)1 FQLQueryIterator (org.apache.cassandra.fqltool.FQLQueryIterator)1 QueryReplayer (org.apache.cassandra.fqltool.QueryReplayer)1 MergeIterator (org.apache.cassandra.utils.MergeIterator)1 Logger (org.slf4j.Logger)1