Examples with Tuple com.apple.foundationdb.tuple.Tuple used on opensource projects

Example 26 with Tuple

use of com.apple.foundationdb.tuple.Tuple in project fdb-record-layer by FoundationDB.

the class IndexingByRecords method buildRange.

/**
 * Builds (transactionally) the index by adding records with primary keys within the given range.
 * This will look for gaps of keys within the given range that haven't yet been rebuilt and then
 * rebuild only those ranges. As a result, if this method is called twice, the first time, it will
 * build whatever needs to be built, and then the second time, it will notice that there are no ranges
 * that need to be built, so it will do nothing. In this way, it is idempotent and thus safe to
 * use in retry loops.
 *
 * This method will fail if there is too much work to be done in a single transaction. If one wants
 * to handle building a range that does not fit in a single transaction, one should use the
 * {@link #buildRange(Key.Evaluated, Key.Evaluated) buildRange()}
 * function that takes an {@link FDBDatabase} as its first parameter.
 *
 * @param store the record store in which to rebuild the range
 * @param start the (inclusive) beginning primary key of the range to build (or <code>null</code> to go to the end)
 * @param end the (exclusive) end primary key of the range to build (or <code>null</code> to go to the end)
 * @return a future that will be ready when the build has completed
 */
@Nonnull
public CompletableFuture<Void> buildRange(@Nonnull FDBRecordStore store, @Nullable Key.Evaluated start, @Nullable Key.Evaluated end) {
    RangeSet rangeSet = new RangeSet(store.indexRangeSubspace(common.getIndex()));
    byte[] startBytes = packOrNull(convertOrNull(start));
    byte[] endBytes = packOrNull(convertOrNull(end));
    AsyncIterator<Range> ranges = rangeSet.missingRanges(store.ensureContextActive(), startBytes, endBytes).iterator();
    return ranges.onHasNext().thenCompose(hasNext -> {
        if (hasNext) {
            return AsyncUtil.whileTrue(() -> {
                Range toBuild = ranges.next();
                Tuple startTuple = Tuple.fromBytes(toBuild.begin);
                Tuple endTuple = RangeSet.isFinalKey(toBuild.end) ? null : Tuple.fromBytes(toBuild.end);
                AtomicReference<Tuple> currStart = new AtomicReference<>(startTuple);
                return AsyncUtil.whileTrue(() -> buildUnbuiltRange(store, currStart.get(), endTuple, null).thenApply(realEnd -> {
                    if (realEnd != null && !realEnd.equals(endTuple)) {
                        currStart.set(realEnd);
                        return true;
                    } else {
                        return false;
                    }
                }), store.getExecutor()).thenCompose(vignore -> ranges.onHasNext());
            }, store.getExecutor());
        } else {
            return AsyncUtil.DONE;
        }
    });
}

Example 27 with Tuple

use of com.apple.foundationdb.tuple.Tuple in project fdb-record-layer by FoundationDB.

the class FDBRecordStoreBase method scanIndexRecordsBetween.

/**
 * Scan the records pointed to by an index between two indexed values.
 * @param indexName the name of the index
 * @param low the low value for the first indexed field
 * @param high the high value for the first indexed field
 * @return a cursor that return records pointed to by the index
 */
@Nonnull
default RecordCursor<FDBIndexedRecord<M>> scanIndexRecordsBetween(@Nonnull final String indexName, @Nullable final Object low, @Nullable final Object high) {
    final Tuple lowTuple = Tuple.from(low);
    final Tuple highTuple = Tuple.from(high);
    final TupleRange range = new TupleRange(lowTuple, highTuple, EndpointType.RANGE_INCLUSIVE, EndpointType.RANGE_INCLUSIVE);
    return scanIndexRecords(indexName, IndexScanType.BY_VALUE, range, null, ScanProperties.FORWARD_SCAN);
}

Example 28 with Tuple

use of com.apple.foundationdb.tuple.Tuple in project fdb-record-layer by FoundationDB.

the class FDBRecordStoreBase method scanIndexRecordsEqual.

/**
 * Scan the records pointed to by an index equal to indexed values.
 * @param indexName the name of the index
 * @param values a left-subset of values of indexed fields
 * @return a cursor that return records pointed to by the index
 */
@Nonnull
default RecordCursor<FDBIndexedRecord<M>> scanIndexRecordsEqual(@Nonnull final String indexName, @Nonnull final Object... values) {
    final Tuple tuple = Tuple.from(values);
    final TupleRange range = TupleRange.allOf(tuple);
    return scanIndexRecords(indexName, IndexScanType.BY_VALUE, range, null, ScanProperties.FORWARD_SCAN);
}

Example 29 with Tuple

use of com.apple.foundationdb.tuple.Tuple in project fdb-record-layer by FoundationDB.

the class TimeWindowLeaderboardIndexMaintainer method scan.

@Nonnull
@Override
public RecordCursor<IndexEntry> scan(@Nonnull IndexScanBounds scanBounds, @Nullable byte[] continuation, @Nonnull ScanProperties scanProperties) {
    final IndexScanType scanType = scanBounds.getScanType();
    if (scanType != IndexScanType.BY_VALUE && scanType != IndexScanType.BY_RANK && scanType != IndexScanType.BY_TIME_WINDOW) {
        throw new RecordCoreException("Can only scan leaderboard index by time window, rank or value.");
    }
    // Decode range arguments.
    final int type;
    final long timestamp;
    final TupleRange leaderboardRange;
    if (scanType == IndexScanType.BY_TIME_WINDOW) {
        // Get oldest leaderboard of type containing timestamp.
        if (scanBounds instanceof TimeWindowScanRange) {
            TimeWindowScanRange scanRange = (TimeWindowScanRange) scanBounds;
            type = scanRange.getLeaderboardType();
            timestamp = scanRange.getLeaderboardTimestamp();
            leaderboardRange = scanRange.getScanRange();
        } else {
            // TODO: For compatibility, accept scan with BY_TIME_WINDOW and TupleRange for a while.
            // This code can be removed when we are confident all callers have been converted.
            IndexScanRange scanRange = (IndexScanRange) scanBounds;
            TupleRange rankRange = scanRange.getScanRange();
            final Tuple lowRank = rankRange.getLow();
            final Tuple highRank = rankRange.getHigh();
            type = (int) lowRank.getLong(0);
            timestamp = lowRank.getLong(1);
            leaderboardRange = new TupleRange(Tuple.fromList(lowRank.getItems().subList(2, lowRank.size())), Tuple.fromList(highRank.getItems().subList(2, highRank.size())), rankRange.getLowEndpoint(), rankRange.getHighEndpoint());
        }
    } else {
        // Get the all-time leaderboard for unqualified rank or value.
        IndexScanRange scanRange = (IndexScanRange) scanBounds;
        type = TimeWindowLeaderboard.ALL_TIME_LEADERBOARD_TYPE;
        // Any value would do.
        timestamp = 0;
        leaderboardRange = scanRange.getScanRange();
    }
    final int groupPrefixSize = getGroupingCount();
    final CompletableFuture<TimeWindowLeaderboard> leaderboardFuture = oldestLeaderboardMatching(type, timestamp);
    final CompletableFuture<TupleRange> scoreRangeFuture;
    if (scanType == IndexScanType.BY_VALUE) {
        scoreRangeFuture = leaderboardFuture.thenApply(leaderboard -> leaderboard == null ? null : leaderboardRange);
    } else {
        scoreRangeFuture = leaderboardFuture.thenCompose(leaderboard -> {
            if (leaderboard == null) {
                return CompletableFuture.completedFuture(null);
            }
            final Subspace extraSubspace = getSecondarySubspace();
            final Subspace leaderboardSubspace = extraSubspace.subspace(leaderboard.getSubspaceKey());
            final RankedSet.Config leaderboardConfig = config.toBuilder().setNLevels(leaderboard.getNLevels()).build();
            return RankedSetIndexHelper.rankRangeToScoreRange(state, groupPrefixSize, leaderboardSubspace, leaderboardConfig, leaderboardRange);
        });
    }
    // Add leaderboard's key to the front and take it off of the results.
    return RecordCursor.flatMapPipelined(ignore -> RecordCursor.fromFuture(getExecutor(), scoreRangeFuture), (scoreRange, ignore) -> {
        if (scoreRange == null) {
            return RecordCursor.empty(getExecutor());
        }
        // Already waited in scoreRangeFuture.
        final TimeWindowLeaderboard leaderboard = state.context.joinNow(leaderboardFuture);
        final CompletableFuture<Boolean> highStoreFirstFuture;
        if (scanType == IndexScanType.BY_VALUE) {
            final Tuple lowGroup = scoreRange.getLow() != null && scoreRange.getLow().size() > groupPrefixSize ? TupleHelpers.subTuple(scoreRange.getLow(), 0, groupPrefixSize) : null;
            final Tuple highGroup = scoreRange.getHigh() != null && scoreRange.getHigh().size() > groupPrefixSize ? TupleHelpers.subTuple(scoreRange.getHigh(), 0, groupPrefixSize) : null;
            if (lowGroup != null && lowGroup.equals(highGroup)) {
                highStoreFirstFuture = isHighScoreFirst(leaderboard.getDirectory(), lowGroup);
            } else {
                highStoreFirstFuture = CompletableFuture.completedFuture(leaderboard.getDirectory().isHighScoreFirst());
            }
        } else {
            highStoreFirstFuture = AsyncUtil.READY_FALSE;
        }
        if (highStoreFirstFuture.isDone()) {
            return scanLeaderboard(leaderboard, state.context.joinNow(highStoreFirstFuture), scoreRange, continuation, scanProperties);
        } else {
            return RecordCursor.flatMapPipelined(ignore2 -> RecordCursor.fromFuture(getExecutor(), highStoreFirstFuture), (highScoreFirst, ignore2) -> scanLeaderboard(leaderboard, highScoreFirst, scoreRange, continuation, scanProperties), null, 1);
        }
    }, null, 1).mapPipelined(kv -> getIndexEntry(kv, groupPrefixSize, state.context.joinNow(leaderboardFuture).getDirectory()), 1);
}

Example 30 with Tuple

use of com.apple.foundationdb.tuple.Tuple in project fdb-record-layer by FoundationDB.

the class TimeWindowLeaderboardIndexMaintainer method evaluateEqualRange.

private CompletableFuture<Tuple> evaluateEqualRange(@Nonnull TupleRange range, @Nonnull EvaluateEqualRange function) {
    final Tuple tuple = range.getLow();
    final int type = (int) tuple.getLong(0);
    final long timestamp = tuple.getLong(1);
    final int groupingCount = getGroupingCount();
    final Tuple groupKey = TupleHelpers.subTuple(tuple, 2, 2 + groupingCount);
    final Tuple values = TupleHelpers.subTuple(tuple, 2 + groupingCount, tuple.size());
    final CompletableFuture<TimeWindowLeaderboard> leaderboardFuture = oldestLeaderboardMatching(type, timestamp);
    return leaderboardFuture.thenCompose(leaderboard -> {
        if (leaderboard == null) {
            return CompletableFuture.completedFuture(null);
        }
        final Tuple leaderboardGroupKey = leaderboard.getSubspaceKey().addAll(groupKey);
        final Subspace extraSubspace = getSecondarySubspace();
        final Subspace rankSubspace = extraSubspace.subspace(leaderboardGroupKey);
        final RankedSet.Config leaderboardConfig = config.toBuilder().setNLevels(leaderboard.getNLevels()).build();
        final RankedSet rankedSet = new RankedSetIndexHelper.InstrumentedRankedSet(state, rankSubspace, leaderboardConfig);
        return function.apply(leaderboard, rankedSet, groupKey, values);
    });
}