Search in sources :

Example 26 with Range

use of com.apple.foundationdb.Range in project fdb-record-layer by FoundationDB.

the class TupleRangeTest method toRange.

@Test
public void toRange() {
    final Tuple a = Tuple.from("a");
    final Tuple b = Tuple.from("b");
    List<Pair<TupleRange, Range>> testCases = Arrays.asList(Pair.of(TupleRange.allOf(null), new Range(new byte[0], new byte[] { (byte) 0xff })), Pair.of(TupleRange.allOf(PREFIX_TUPLE), Range.startsWith(PREFIX_BYTES)), Pair.of(new TupleRange(a, b, EndpointType.RANGE_INCLUSIVE, EndpointType.RANGE_EXCLUSIVE), new Range(a.pack(), b.pack())), Pair.of(new TupleRange(a, b, EndpointType.RANGE_INCLUSIVE, EndpointType.RANGE_INCLUSIVE), new Range(a.pack(), ByteArrayUtil.strinc(b.pack()))), Pair.of(new TupleRange(a, b, EndpointType.RANGE_EXCLUSIVE, EndpointType.RANGE_EXCLUSIVE), new Range(ByteArrayUtil.strinc(a.pack()), b.pack())), Pair.of(new TupleRange(a, b, EndpointType.RANGE_EXCLUSIVE, EndpointType.RANGE_INCLUSIVE), new Range(ByteArrayUtil.strinc(a.pack()), ByteArrayUtil.strinc(b.pack()))), Pair.of(new TupleRange(null, b, EndpointType.TREE_START, EndpointType.RANGE_EXCLUSIVE), new Range(new byte[0], b.pack())), Pair.of(new TupleRange(a, null, EndpointType.RANGE_INCLUSIVE, EndpointType.TREE_END), new Range(a.pack(), new byte[] { (byte) 0xff })), Pair.of(new TupleRange(a, b, EndpointType.CONTINUATION, EndpointType.RANGE_EXCLUSIVE), new Range(ByteArrayUtil.join(a.pack(), new byte[] { 0x00 }), b.pack())), Pair.of(new TupleRange(a, b, EndpointType.RANGE_INCLUSIVE, EndpointType.CONTINUATION), new Range(a.pack(), b.pack())), Pair.of(TupleRange.prefixedBy("a"), new Range(new byte[] { 0x02, (byte) 'a' }, new byte[] { 0x02, (byte) 'b' })), Pair.of(new TupleRange(Tuple.from("apple"), a, EndpointType.CONTINUATION, EndpointType.PREFIX_STRING), new Range(ByteArrayUtil.join(Tuple.from("apple").pack(), new byte[] { 0x00 }), new byte[] { 0x02, (byte) 'b' })), Pair.of(new TupleRange(a, Tuple.from("apple"), EndpointType.PREFIX_STRING, EndpointType.CONTINUATION), new Range(new byte[] { 0x02, (byte) 'a' }, Tuple.from("apple").pack())));
    testCases.forEach(pair -> {
        assertEquals(pair.getRight(), pair.getLeft().toRange());
        TupleRange expectedRange = pair.getLeft().prepend(PREFIX_TUPLE);
        assertEquals(expectedRange.toRange(), pair.getLeft().toRange(PREFIX_SUBSPACE));
    });
}
Also used : Range(com.apple.foundationdb.Range) Tuple(com.apple.foundationdb.tuple.Tuple) Pair(org.apache.commons.lang3.tuple.Pair) Test(org.junit.jupiter.api.Test)

Example 27 with Range

use of com.apple.foundationdb.Range in project fdb-record-layer by FoundationDB.

the class FDBRecordStoreTest method testOverlappingPrimaryKey.

@Test
public void testOverlappingPrimaryKey() {
    try (FDBRecordContext context = openContext()) {
        RecordMetaDataBuilder builder = RecordMetaData.newBuilder().setRecords(TestRecordsWithHeaderProto.getDescriptor());
        builder.getRecordType("MyRecord").setPrimaryKey(field("header").nest(concatenateFields("path", "rec_no")));
        builder.addIndex("MyRecord", "MyRecord$path_str", concat(field("header").nest("path"), field("str_value")));
        RecordMetaData metaData = builder.getRecordMetaData();
        createOrOpenRecordStore(context, metaData);
        TestRecordsWithHeaderProto.MyRecord.Builder recBuilder = TestRecordsWithHeaderProto.MyRecord.newBuilder();
        TestRecordsWithHeaderProto.HeaderRecord.Builder headerBuilder = recBuilder.getHeaderBuilder();
        headerBuilder.setPath("aaa");
        headerBuilder.setRecNo(1);
        recBuilder.setStrValue("hello");
        recordStore.saveRecord(recBuilder.build());
        headerBuilder.setPath("aaa");
        headerBuilder.setRecNo(2);
        recBuilder.setStrValue("goodbye");
        recordStore.saveRecord(recBuilder.build());
        headerBuilder.setPath("zzz");
        headerBuilder.setRecNo(3);
        recBuilder.setStrValue("end");
        recordStore.saveRecord(recBuilder.build());
        List<List<Object>> rows = new ArrayList<>();
        Index index = metaData.getIndex("MyRecord$path_str");
        ScanComparisons comparisons = ScanComparisons.from(new Comparisons.SimpleComparison(Comparisons.Type.EQUALS, "aaa"));
        TupleRange range = comparisons.toTupleRange();
        try (RecordCursor<IndexEntry> cursor = recordStore.scanIndex(index, IndexScanType.BY_VALUE, range, null, ScanProperties.FORWARD_SCAN)) {
            cursor.forEach(row -> rows.add(row.getKey().getItems())).join();
        }
        assertEquals(Arrays.asList(Arrays.asList("aaa", "goodbye", 2L), Arrays.asList("aaa", "hello", 1L)), rows);
    }
}
Also used : IndexEntry(com.apple.foundationdb.record.IndexEntry) DescriptorProtos(com.google.protobuf.DescriptorProtos) DELETE_INDEX_ENTRY(com.apple.foundationdb.record.provider.foundationdb.FDBStoreTimer.Events.DELETE_INDEX_ENTRY) Arrays(java.util.Arrays) PlanMatchers.bounds(com.apple.foundationdb.record.query.plan.match.PlanMatchers.bounds) RecordCoreStorageException(com.apple.foundationdb.record.RecordCoreStorageException) ByteBuffer(java.nio.ByteBuffer) RecordSerializer(com.apple.foundationdb.record.provider.common.RecordSerializer) RecordSerializationException(com.apple.foundationdb.record.provider.common.RecordSerializationException) IndexScanType(com.apple.foundationdb.record.IndexScanType) Tuple(com.apple.foundationdb.tuple.Tuple) Range(com.apple.foundationdb.Range) FDBError(com.apple.foundationdb.FDBError) RecordCoreException(com.apple.foundationdb.record.RecordCoreException) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Expressions.concatenateFields(com.apple.foundationdb.record.metadata.Key.Expressions.concatenateFields) RecordMetaDataProto(com.apple.foundationdb.record.RecordMetaDataProto) RecordMetaDataOptionsProto(com.apple.foundationdb.record.RecordMetaDataOptionsProto) Is.is(org.hamcrest.core.Is.is) Expressions.concat(com.apple.foundationdb.record.metadata.Key.Expressions.concat) Tag(org.junit.jupiter.api.Tag) MethodSource(org.junit.jupiter.params.provider.MethodSource) TestRecords7Proto(com.apple.foundationdb.record.TestRecords7Proto) ByteArrayUtil(com.apple.foundationdb.tuple.ByteArrayUtil) Query(com.apple.foundationdb.record.query.expressions.Query) KeyExpression(com.apple.foundationdb.record.metadata.expressions.KeyExpression) TestRecords1Proto(com.apple.foundationdb.record.TestRecords1Proto) Matchers.allOf(org.hamcrest.Matchers.allOf) Set(java.util.Set) Collectors(java.util.stream.Collectors) TupleRange(com.apple.foundationdb.record.TupleRange) ByteString(com.google.protobuf.ByteString) Test(org.junit.jupiter.api.Test) Matchers.instanceOf(org.hamcrest.Matchers.instanceOf) KeySpacePath(com.apple.foundationdb.record.provider.foundationdb.keyspace.KeySpacePath) PlanMatchers.hasTupleString(com.apple.foundationdb.record.query.plan.match.PlanMatchers.hasTupleString) List(java.util.List) PlanMatchers.indexName(com.apple.foundationdb.record.query.plan.match.PlanMatchers.indexName) RecordMetaDataProvider(com.apple.foundationdb.record.RecordMetaDataProvider) EvaluationContext(com.apple.foundationdb.record.EvaluationContext) Matchers.equalTo(org.hamcrest.Matchers.equalTo) Assertions.assertTrue(org.junit.jupiter.api.Assertions.assertTrue) IndexTypes(com.apple.foundationdb.record.metadata.IndexTypes) TestRecordsWithHeaderProto(com.apple.foundationdb.record.TestRecordsWithHeaderProto) Matchers.greaterThan(org.hamcrest.Matchers.greaterThan) Matchers.containsString(org.hamcrest.Matchers.containsString) Assertions.assertThrows(org.junit.jupiter.api.Assertions.assertThrows) Assertions.fail(org.junit.jupiter.api.Assertions.fail) RecordMetaData(com.apple.foundationdb.record.RecordMetaData) TestNoUnionProto(com.apple.foundationdb.record.TestNoUnionProto) Assertions.assertNotNull(org.junit.jupiter.api.Assertions.assertNotNull) TestRecordsDuplicateUnionFields(com.apple.foundationdb.record.TestRecordsDuplicateUnionFields) DynamicMessage(com.google.protobuf.DynamicMessage) Descriptors(com.google.protobuf.Descriptors) PlanMatchers.indexScan(com.apple.foundationdb.record.query.plan.match.PlanMatchers.indexScan) Assertions.assertNull(org.junit.jupiter.api.Assertions.assertNull) CompletableFuture(java.util.concurrent.CompletableFuture) RecordQuery(com.apple.foundationdb.record.query.RecordQuery) RecordQueryPlan(com.apple.foundationdb.record.query.plan.plans.RecordQueryPlan) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) Strings(com.google.common.base.Strings) ExecuteProperties(com.apple.foundationdb.record.ExecuteProperties) ScanProperties(com.apple.foundationdb.record.ScanProperties) RecordCursorIterator(com.apple.foundationdb.record.RecordCursorIterator) Matchers.lessThan(org.hamcrest.Matchers.lessThan) Assumptions.assumeTrue(org.junit.jupiter.api.Assumptions.assumeTrue) Matchers.hasSize(org.hamcrest.Matchers.hasSize) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) Nonnull(javax.annotation.Nonnull) Expressions.field(com.apple.foundationdb.record.metadata.Key.Expressions.field) Nullable(javax.annotation.Nullable) Charsets(com.google.common.base.Charsets) IsolationLevel(com.apple.foundationdb.record.IsolationLevel) RecordMetaDataBuilder(com.apple.foundationdb.record.RecordMetaDataBuilder) Tags(com.apple.test.Tags) ScanComparisons(com.apple.foundationdb.record.query.plan.ScanComparisons) Assertions.assertSame(org.junit.jupiter.api.Assertions.assertSame) ExecutionException(java.util.concurrent.ExecutionException) Consumer(java.util.function.Consumer) Comparisons(com.apple.foundationdb.record.query.expressions.Comparisons) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) TestRecordsImportProto(com.apple.foundationdb.record.TestRecordsImportProto) Index(com.apple.foundationdb.record.metadata.Index) FDBException(com.apple.foundationdb.FDBException) Message(com.google.protobuf.Message) RecordCursor(com.apple.foundationdb.record.RecordCursor) Collections(java.util.Collections) SAVE_INDEX_ENTRY(com.apple.foundationdb.record.provider.foundationdb.FDBStoreTimer.Events.SAVE_INDEX_ENTRY) RecordMetaData(com.apple.foundationdb.record.RecordMetaData) ArrayList(java.util.ArrayList) IndexEntry(com.apple.foundationdb.record.IndexEntry) Index(com.apple.foundationdb.record.metadata.Index) ScanComparisons(com.apple.foundationdb.record.query.plan.ScanComparisons) RecordMetaDataBuilder(com.apple.foundationdb.record.RecordMetaDataBuilder) ScanComparisons(com.apple.foundationdb.record.query.plan.ScanComparisons) Comparisons(com.apple.foundationdb.record.query.expressions.Comparisons) List(java.util.List) ArrayList(java.util.ArrayList) TupleRange(com.apple.foundationdb.record.TupleRange) Test(org.junit.jupiter.api.Test) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 28 with Range

use of com.apple.foundationdb.Range in project fdb-record-layer by FoundationDB.

the class RangeSetTest method concurrentWithInsert.

// Test out a few of the wild and wacky things that can happen when there are insertions going concurrently.
@Test
@Tag(Tags.Slow)
public void concurrentWithInsert() {
    final List<byte[]> keys = createKeys();
    final List<Range> ranges = new ArrayList<>();
    // Two disjoint ranges -- both should succeed.
    Transaction tr1 = db.createTransaction();
    Transaction tr2 = db.createTransaction();
    Range r1 = new Range(new byte[] { 0x01 }, new byte[] { 0x02 });
    Range r2 = new Range(new byte[] { 0x02 }, new byte[] { 0x03 });
    CompletableFuture<Boolean> future1 = rs.insertRange(tr1, r1);
    CompletableFuture<Boolean> future2 = rs.insertRange(tr2, r2);
    assertTrue(future1.join(), "Range 1 did not do insertion.");
    assertTrue(future2.join(), "Range 2 did not do insertion.");
    tr1.commit().join();
    tr2.commit().join();
    ranges.add(r1);
    ranges.add(r2);
    checkConsistent(ranges, keys);
    checkIncreasing();
    // Two non-disjoint ranges. The second should fail.
    tr1 = db.createTransaction();
    tr2 = db.createTransaction();
    r1 = new Range(new byte[] { 0x03 }, new byte[] { 0x05 });
    r2 = new Range(new byte[] { 0x04 }, new byte[] { 0x06 });
    future1 = rs.insertRange(tr1, r1);
    future2 = rs.insertRange(tr2, r2);
    assertTrue(future1.join(), "Range 1 did not do insertion");
    assertTrue(future2.join(), "Range 2 did not do insertion");
    tr1.commit().join();
    tr2.commit().handle((vignore, e) -> {
        assertNotNull(e, "No error thrown from commit");
        assertTrue(e instanceof FDBException, "Non-FDBException " + e.toString());
        FDBException fdbe = (FDBException) e;
        assertEquals(FDBError.NOT_COMMITTED.code(), fdbe.getCode(), "Did not get not-committed error.");
        return vignore;
    }).join();
    ranges.add(r1);
    checkConsistent(ranges, keys);
    checkIncreasing();
    // Read during write - the reads in the range should fail.
    r1 = new Range(new byte[] { 0x07 }, new byte[] { 0x08 });
    List<byte[]> specificKeys = Arrays.asList(new byte[] { 0x06, (byte) 0xff }, new byte[] { 0x07 }, new byte[] { 0x07, 0x00 }, new byte[] { 0x07, 0x10 }, new byte[] { 0x08 }, new byte[] { 0x08, 0x00 }, new byte[] { 0x08, 0x10 }, new byte[] { 0x09 });
    tr1 = db.createTransaction();
    List<Transaction> transactions = specificKeys.stream().map(ignore -> db.createTransaction()).collect(Collectors.toList());
    // Add write conflict ranges to each key so that they are not read only.
    transactions.forEach(tr -> tr.addWriteConflictKey(DEADC0DE));
    future1 = rs.insertRange(tr1, r1);
    List<CompletableFuture<Boolean>> futures = new ArrayList<>();
    for (int i = 0; i < specificKeys.size(); i++) {
        futures.add(rs.contains(transactions.get(i), specificKeys.get(i)));
    }
    assertTrue(future1.join(), "Range 1 did not do insertion");
    futures.forEach(future -> assertFalse(future.join(), "Key should not be present"));
    tr1.commit().join();
    Range range = r1;
    for (int i = 0; i < transactions.size(); i++) {
        final int index = i;
        transactions.get(i).commit().handle((vignore, e) -> {
            byte[] key = specificKeys.get(index);
            String repr = ByteArrayUtil.printable(key);
            if (ByteArrayUtil.compareUnsigned(range.begin, key) <= 0 && ByteArrayUtil.compareUnsigned(key, range.end) < 0) {
                assertNotNull(e, "No error from commit when key is " + repr);
                assertTrue(e instanceof FDBException, "Non-FDBException " + e.toString() + " with key " + repr);
                FDBException fdbe = (FDBException) e;
                assertEquals(FDBError.NOT_COMMITTED.code(), fdbe.getCode(), "Did not get non-committed error when key is " + repr);
            } else {
                assertNull(e, "Error when key is " + repr);
            }
            return vignore;
        }).join();
    }
    ranges.add(r1);
    checkConsistent(ranges, keys);
    checkIncreasing();
}
Also used : Assertions.assertThrows(org.junit.jupiter.api.Assertions.assertThrows) BeforeEach(org.junit.jupiter.api.BeforeEach) Assertions.assertNotNull(org.junit.jupiter.api.Assertions.assertNotNull) FDB(com.apple.foundationdb.FDB) Arrays(java.util.Arrays) FDBTestBase(com.apple.foundationdb.FDBTestBase) Assertions.assertNull(org.junit.jupiter.api.Assertions.assertNull) CompletableFuture(java.util.concurrent.CompletableFuture) Database(com.apple.foundationdb.Database) Subspace(com.apple.foundationdb.subspace.Subspace) ArrayList(java.util.ArrayList) Transaction(com.apple.foundationdb.Transaction) Range(com.apple.foundationdb.Range) Assertions.assertFalse(org.junit.jupiter.api.Assertions.assertFalse) FDBError(com.apple.foundationdb.FDBError) ByteArrayUtil.compareUnsigned(com.apple.foundationdb.tuple.ByteArrayUtil.compareUnsigned) Tag(org.junit.jupiter.api.Tag) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) Nonnull(javax.annotation.Nonnull) ByteArrayUtil.printable(com.apple.foundationdb.tuple.ByteArrayUtil.printable) ByteArrayUtil(com.apple.foundationdb.tuple.ByteArrayUtil) KeyValue(com.apple.foundationdb.KeyValue) Tags(com.apple.test.Tags) DirectoryLayer(com.apple.foundationdb.directory.DirectoryLayer) Collectors(java.util.stream.Collectors) Test(org.junit.jupiter.api.Test) Assertions.assertArrayEquals(org.junit.jupiter.api.Assertions.assertArrayEquals) List(java.util.List) PathUtil(com.apple.foundationdb.directory.PathUtil) FDBException(com.apple.foundationdb.FDBException) Assertions.assertTrue(org.junit.jupiter.api.Assertions.assertTrue) ArrayList(java.util.ArrayList) FDBException(com.apple.foundationdb.FDBException) Range(com.apple.foundationdb.Range) CompletableFuture(java.util.concurrent.CompletableFuture) Transaction(com.apple.foundationdb.Transaction) Test(org.junit.jupiter.api.Test) Tag(org.junit.jupiter.api.Tag)

Example 29 with Range

use of com.apple.foundationdb.Range in project fdb-record-layer by FoundationDB.

the class FDBRecordStore method addRecordReadConflict.

@Override
public void addRecordReadConflict(@Nonnull Tuple primaryKey) {
    final Range recordRange = getRangeForRecord(primaryKey);
    ensureContextActive().addReadConflictRange(recordRange.begin, recordRange.end);
}
Also used : TupleRange(com.apple.foundationdb.record.TupleRange) Range(com.apple.foundationdb.Range)

Example 30 with Range

use of com.apple.foundationdb.Range in project fdb-record-layer by FoundationDB.

the class FDBRecordStore method addRecordWriteConflict.

@Override
public void addRecordWriteConflict(@Nonnull Tuple primaryKey) {
    final Range recordRange = getRangeForRecord(primaryKey);
    ensureContextActive().addWriteConflictRange(recordRange.begin, recordRange.end);
}
Also used : TupleRange(com.apple.foundationdb.record.TupleRange) Range(com.apple.foundationdb.Range)

Aggregations

Range (com.apple.foundationdb.Range)38 Nonnull (javax.annotation.Nonnull)28 TupleRange (com.apple.foundationdb.record.TupleRange)26 Tuple (com.apple.foundationdb.tuple.Tuple)24 List (java.util.List)23 RangeSet (com.apple.foundationdb.async.RangeSet)22 CompletableFuture (java.util.concurrent.CompletableFuture)22 Index (com.apple.foundationdb.record.metadata.Index)21 Subspace (com.apple.foundationdb.subspace.Subspace)19 AsyncUtil (com.apple.foundationdb.async.AsyncUtil)18 LogMessageKeys (com.apple.foundationdb.record.logging.LogMessageKeys)18 IsolationLevel (com.apple.foundationdb.record.IsolationLevel)17 RecordCoreException (com.apple.foundationdb.record.RecordCoreException)17 Message (com.google.protobuf.Message)17 Collectors (java.util.stream.Collectors)17 Nullable (javax.annotation.Nullable)17 Transaction (com.apple.foundationdb.Transaction)16 AsyncIterator (com.apple.foundationdb.async.AsyncIterator)16 RecordCursor (com.apple.foundationdb.record.RecordCursor)16 ExecuteProperties (com.apple.foundationdb.record.ExecuteProperties)15