Search in sources :

Example 31 with Subspace

use of com.apple.foundationdb.subspace.Subspace in project fdb-record-layer by FoundationDB.

the class HighContentionAllocatorTest method validateAllocation.

private void validateAllocation(FDBRecordContext context, HighContentionAllocator hca, Map<Long, String> allocations) {
    Subspace allocationSubspace = hca.getAllocationSubspace();
    Transaction transaction = context.ensureActive();
    List<KeyValue> keyValueList = transaction.getRange(allocationSubspace.range()).asList().join();
    Map<Long, String> storedAllocations = keyValueList.stream().collect(Collectors.toMap(kv -> extractKey(allocationSubspace, kv), this::extractValue));
    assertThat("we see the allocated keys in the subspace", allocations.entrySet(), containsInAnyOrder(storedAllocations.entrySet().toArray()));
}
Also used : IntStream(java.util.stream.IntStream) Assertions.fail(org.junit.jupiter.api.Assertions.fail) BeforeEach(org.junit.jupiter.api.BeforeEach) KeySpace(com.apple.foundationdb.record.provider.foundationdb.keyspace.KeySpace) Matchers.not(org.hamcrest.Matchers.not) FDBRecordContext(com.apple.foundationdb.record.provider.foundationdb.FDBRecordContext) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) AsyncUtil(com.apple.foundationdb.async.AsyncUtil) Matchers.hasKey(org.hamcrest.Matchers.hasKey) FDBTestBase(com.apple.foundationdb.record.provider.foundationdb.FDBTestBase) Subspace(com.apple.foundationdb.subspace.Subspace) ArrayList(java.util.ArrayList) Transaction(com.apple.foundationdb.Transaction) KeyType(com.apple.foundationdb.record.provider.foundationdb.keyspace.KeySpaceDirectory.KeyType) Tuple(com.apple.foundationdb.tuple.Tuple) Range(com.apple.foundationdb.Range) Pair(org.apache.commons.lang3.tuple.Pair) Matchers.everyItem(org.hamcrest.Matchers.everyItem) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Map(java.util.Map) Matchers.lessThan(org.hamcrest.Matchers.lessThan) Is.is(org.hamcrest.core.Is.is) Matchers.hasSize(org.hamcrest.Matchers.hasSize) Tag(org.junit.jupiter.api.Tag) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) Matchers.greaterThanOrEqualTo(org.hamcrest.Matchers.greaterThanOrEqualTo) KeyValue(com.apple.foundationdb.KeyValue) KeySpaceDirectory(com.apple.foundationdb.record.provider.foundationdb.keyspace.KeySpaceDirectory) Tags(com.apple.test.Tags) Matchers.allOf(org.hamcrest.Matchers.allOf) FDBDatabase(com.apple.foundationdb.record.provider.foundationdb.FDBDatabase) Collectors(java.util.stream.Collectors) Test(org.junit.jupiter.api.Test) Assertions.assertArrayEquals(org.junit.jupiter.api.Assertions.assertArrayEquals) List(java.util.List) Matchers.containsInAnyOrder(org.hamcrest.Matchers.containsInAnyOrder) AllocationWindow(com.apple.foundationdb.record.provider.foundationdb.layers.interning.HighContentionAllocator.AllocationWindow) FDBDatabaseFactory(com.apple.foundationdb.record.provider.foundationdb.FDBDatabaseFactory) KeyValue(com.apple.foundationdb.KeyValue) Transaction(com.apple.foundationdb.Transaction) Subspace(com.apple.foundationdb.subspace.Subspace)

Example 32 with Subspace

use of com.apple.foundationdb.subspace.Subspace in project fdb-record-layer by FoundationDB.

the class HighContentionAllocatorTest method testCheckForRootConflicts.

@Test
@Tag(Tags.WipesFDB)
void testCheckForRootConflicts() {
    Range everything = new Range(new byte[] { (byte) 0x00 }, new byte[] { (byte) 0xFF });
    database.run(context -> {
        context.ensureActive().clear(everything);
        return null;
    });
    try (FDBRecordContext context = database.openContext()) {
        HighContentionAllocator hca = HighContentionAllocator.forRoot(context, keySpace.path("test-path"));
        // so write something for all of those keys
        for (int i = 0; i < 64; i++) {
            byte[] key = Tuple.from(i, "string-" + i).pack();
            byte[] value = new byte[0];
            context.ensureActive().set(key, value);
        }
        try {
            hca.allocate("some-string").join();
            fail("allocate should fail in the same transaction");
        } catch (Exception e) {
            assertThat("a", e.getCause().getMessage(), is("database already has keys in allocation range"));
        }
        // check that the hca marks these keys as invalid
        // the thing here is that when the post allocation hook fails the allocator still writes a key,
        // that's actually a good thing since it will prevent us from trying to allocate that key again
        // but we need to make sure we have a way to exclude from the reverse lookup
        Subspace allocationSubspace = hca.getAllocationSubspace();
        Range initialwindow = new Range(allocationSubspace.getKey(), allocationSubspace.pack(64));
        List<KeyValue> allocatedValues = context.ensureActive().getRange(initialwindow).asList().join();
        byte[] valueBytes = allocatedValues.get(0).getValue();
        assertThat("there's exactly one allocation key", allocatedValues, hasSize(1));
        assertArrayEquals(valueBytes, new byte[] { (byte) 0xFD }, "the value is set to the magic byte");
        context.commit();
    }
    try (FDBRecordContext context = database.openContext()) {
        HighContentionAllocator hca = HighContentionAllocator.forRoot(context, keySpace.path("test-path"));
        try {
            hca.allocate("some-string").join();
            fail("allocate should fail in new transaction");
        } catch (Exception e) {
            assertThat("a", e.getCause().getMessage(), is("database already has keys in allocation range"));
        }
    }
    database.run(context -> {
        context.ensureActive().clear(everything);
        return null;
    });
}
Also used : KeyValue(com.apple.foundationdb.KeyValue) FDBRecordContext(com.apple.foundationdb.record.provider.foundationdb.FDBRecordContext) Subspace(com.apple.foundationdb.subspace.Subspace) Range(com.apple.foundationdb.Range) Test(org.junit.jupiter.api.Test) Tag(org.junit.jupiter.api.Tag)

Example 33 with Subspace

use of com.apple.foundationdb.subspace.Subspace in project fdb-record-layer by FoundationDB.

the class ScopedDirectoryLayerTest method validate.

private void validate(FDBRecordContext context, LocatableResolver resolver, DirectoryLayer directoryLayer, String key, Long value) {
    List<String> directories = directoryLayer.list(context.ensureActive()).join();
    assertThat("entry was added to the appropriate directory layer", directories, hasItem(key));
    Subspace resultSubpsace = directoryLayer.open(context.ensureActive(), ImmutableList.of(key)).join();
    Long directoryValue = resolver.deserializeValue(resultSubpsace.getKey()).getValue();
    assertThat("resolver returned the value of the subspace prefix", directoryValue, is(value));
    Long newValue = resolver.resolve(context.getTimer(), key).join();
    assertThat("repeated calls to resolve return the same value", newValue, is(value));
}
Also used : Subspace(com.apple.foundationdb.subspace.Subspace)

Example 34 with Subspace

use of com.apple.foundationdb.subspace.Subspace in project fdb-record-layer by FoundationDB.

the class ScopedDirectoryLayerTest method testDefaultResolverSeesPreviousDefaultDirectoryLayerEntries.

@Test
public void testDefaultResolverSeesPreviousDefaultDirectoryLayerEntries() {
    final DirectoryLayer directoryLayer = DirectoryLayer.getDefault();
    final List<String> names = IntStream.range(0, 5).mapToObj(number -> String.format("name-%d", number)).collect(Collectors.toList());
    Map<String, Long> values = new HashMap<>();
    try (FDBRecordContext context = database.openContext()) {
        for (String name : names) {
            values.put(name, directoryLayer.createOrOpen(context.ensureActive(), ImmutableList.of(name)).thenApply(subspace -> Tuple.fromBytes(subspace.getKey()).getLong(0)).join());
        }
        context.commit();
    }
    try (FDBRecordContext context = database.openContext()) {
        for (String name : names) {
            Long resolvedValue = globalScope.resolve(context.getTimer(), name).join();
            assertThat("resolver sees all mappings in directory layer", values.get(name), is(resolvedValue));
        }
    }
}
Also used : IntStream(java.util.stream.IntStream) Assertions.assertThrows(org.junit.jupiter.api.Assertions.assertThrows) MetadataHook(com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook) Tags(com.apple.test.Tags) Matchers.not(org.hamcrest.Matchers.not) DirectoryLayer(com.apple.foundationdb.directory.DirectoryLayer) FDBRecordContext(com.apple.foundationdb.record.provider.foundationdb.FDBRecordContext) HashMap(java.util.HashMap) CompletionException(java.util.concurrent.CompletionException) Disabled(org.junit.jupiter.api.Disabled) Collectors(java.util.stream.Collectors) Bytes(com.google.common.primitives.Bytes) Subspace(com.apple.foundationdb.subspace.Subspace) Test(org.junit.jupiter.api.Test) KeyType(com.apple.foundationdb.record.provider.foundationdb.keyspace.KeySpaceDirectory.KeyType) Tuple(com.apple.foundationdb.tuple.Tuple) List(java.util.List) Matchers.hasItem(org.hamcrest.Matchers.hasItem) ImmutableList(com.google.common.collect.ImmutableList) Map(java.util.Map) Matchers.nullValue(org.hamcrest.Matchers.nullValue) Is.is(org.hamcrest.core.Is.is) Tag(org.junit.jupiter.api.Tag) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) DirectoryLayer(com.apple.foundationdb.directory.DirectoryLayer) HashMap(java.util.HashMap) FDBRecordContext(com.apple.foundationdb.record.provider.foundationdb.FDBRecordContext) Test(org.junit.jupiter.api.Test)

Example 35 with Subspace

use of com.apple.foundationdb.subspace.Subspace in project fdb-record-layer by FoundationDB.

the class FDBDirectoryBaseTest method setUp.

@BeforeEach
public void setUp() {
    if (fdb == null) {
        fdb = FDBDatabaseFactory.instance().getDatabase();
    }
    if (subspace == null) {
        subspace = fdb.run(context -> TestKeySpace.getKeyspacePath("record-test", "unit", "indexTest", "version").toSubspace(context));
    }
    fdb.run(context -> {
        context.ensureActive().clear(subspace.range());
        return null;
    });
    FDBRecordContext context = fdb.openContext(getContextConfig());
    directory = new FDBDirectory(subspace, context);
}
Also used : BeforeEach(org.junit.jupiter.api.BeforeEach) FDBStoreTimer(com.apple.foundationdb.record.provider.foundationdb.FDBStoreTimer) RecordLayerPropertyStorage(com.apple.foundationdb.record.provider.foundationdb.properties.RecordLayerPropertyStorage) FDBDatabaseFactory(com.apple.foundationdb.record.provider.foundationdb.FDBDatabaseFactory) FDBRecordContext(com.apple.foundationdb.record.provider.foundationdb.FDBRecordContext) FDBTransactionPriority(com.apple.foundationdb.record.provider.foundationdb.FDBTransactionPriority) Random(java.util.Random) LuceneRecordContextProperties(com.apple.foundationdb.record.lucene.LuceneRecordContextProperties) FDBDatabase(com.apple.foundationdb.record.provider.foundationdb.FDBDatabase) TestKeySpace(com.apple.foundationdb.record.provider.foundationdb.TestKeySpace) FDBRecordContextConfig(com.apple.foundationdb.record.provider.foundationdb.FDBRecordContextConfig) Subspace(com.apple.foundationdb.subspace.Subspace) FDBRecordContext(com.apple.foundationdb.record.provider.foundationdb.FDBRecordContext) BeforeEach(org.junit.jupiter.api.BeforeEach)

Aggregations

Subspace (com.apple.foundationdb.subspace.Subspace)61 Nonnull (javax.annotation.Nonnull)33 Tuple (com.apple.foundationdb.tuple.Tuple)28 List (java.util.List)23 ArrayList (java.util.ArrayList)21 CompletableFuture (java.util.concurrent.CompletableFuture)21 Nullable (javax.annotation.Nullable)20 RecordCoreException (com.apple.foundationdb.record.RecordCoreException)19 KeyValue (com.apple.foundationdb.KeyValue)18 Map (java.util.Map)18 AsyncUtil (com.apple.foundationdb.async.AsyncUtil)17 API (com.apple.foundationdb.annotation.API)16 ScanProperties (com.apple.foundationdb.record.ScanProperties)15 Collections (java.util.Collections)15 Test (org.junit.jupiter.api.Test)15 Transaction (com.apple.foundationdb.Transaction)14 RecordCursor (com.apple.foundationdb.record.RecordCursor)14 TupleRange (com.apple.foundationdb.record.TupleRange)14 LogMessageKeys (com.apple.foundationdb.record.logging.LogMessageKeys)14 Message (com.google.protobuf.Message)14