Search in sources :

Example 1 with MetadataHook

use of com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook in project fdb-record-layer by FoundationDB.

the class ResolverMappingReplicatorTest method seedDirectoryLayer.

protected Map<String, ResolverResult> seedDirectoryLayer(FDBRecordContext context, LocatableResolver scope, int entries) {
    Map<String, ResolverResult> mappings = new HashMap<>();
    for (int i = 0; i < entries; i++) {
        String key = "entry-" + random.nextLong();
        ResolverResult result;
        if (seedWithMetadata) {
            // include metadata in ~1/2 the entries
            MetadataHook hook = random.nextBoolean() ? name -> Tuple.from("metadata-for-key-" + name, random.nextLong()).pack() : ignore -> null;
            result = scope.resolveWithMetadata(context.getTimer(), key, new ResolverCreateHooks(ResolverCreateHooks.DEFAULT_CHECK, hook)).join();
        } else {
            result = scope.resolveWithMetadata(context.getTimer(), key, ResolverCreateHooks.getDefault()).join();
        }
        mappings.put(key, result);
    }
    context.commit();
    return mappings;
}
Also used : HashMap(java.util.HashMap) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) MetadataHook(com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook)

Example 2 with MetadataHook

use of com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook in project fdb-record-layer by FoundationDB.

the class ScopedDirectoryLayerTest method testResolveWithMetadata.

@Test
@Override
public void testResolveWithMetadata() {
    ResolverCreateHooks noMetadata = ResolverCreateHooks.getDefault();
    String key1 = "key1";
    assertThat(noMetadata.getMetadataHook().apply(key1), is(nullValue()));
    // works as long as the metadatahook returns null
    globalScope.resolveWithMetadata(key1, noMetadata).join();
    String key2 = "key2";
    MetadataHook hook = name -> Tuple.from(name).pack();
    ResolverCreateHooks withMetadata = new ResolverCreateHooks(ResolverCreateHooks.DEFAULT_CHECK, hook);
    assertThat(withMetadata.getMetadataHook().apply(key2), is(not(nullValue())));
    assertThrows(CompletionException.class, () -> globalScope.resolveWithMetadata(key2, withMetadata).join());
}
Also used : IntStream(java.util.stream.IntStream) Assertions.assertThrows(org.junit.jupiter.api.Assertions.assertThrows) MetadataHook(com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook) Tags(com.apple.test.Tags) Matchers.not(org.hamcrest.Matchers.not) DirectoryLayer(com.apple.foundationdb.directory.DirectoryLayer) FDBRecordContext(com.apple.foundationdb.record.provider.foundationdb.FDBRecordContext) HashMap(java.util.HashMap) CompletionException(java.util.concurrent.CompletionException) Disabled(org.junit.jupiter.api.Disabled) Collectors(java.util.stream.Collectors) Bytes(com.google.common.primitives.Bytes) Subspace(com.apple.foundationdb.subspace.Subspace) Test(org.junit.jupiter.api.Test) KeyType(com.apple.foundationdb.record.provider.foundationdb.keyspace.KeySpaceDirectory.KeyType) Tuple(com.apple.foundationdb.tuple.Tuple) List(java.util.List) Matchers.hasItem(org.hamcrest.Matchers.hasItem) ImmutableList(com.google.common.collect.ImmutableList) Map(java.util.Map) Matchers.nullValue(org.hamcrest.Matchers.nullValue) Is.is(org.hamcrest.core.Is.is) Tag(org.junit.jupiter.api.Tag) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) MetadataHook(com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook) Test(org.junit.jupiter.api.Test)

Example 3 with MetadataHook

use of com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook in project fdb-record-layer by FoundationDB.

the class ExtendedDirectoryLayerTest method testScopedDirectoryLayerResolvesWithoutMetadata.

@Test
public void testScopedDirectoryLayerResolvesWithoutMetadata() {
    MetadataHook hook = name -> Tuple.from("metadata-for-" + name).pack();
    ResolverCreateHooks createHooks = new ResolverCreateHooks(ResolverCreateHooks.DEFAULT_CHECK, hook);
    ResolverResult result = globalScope.resolveWithMetadata("some-key", createHooks).join();
    assertArrayEquals(Tuple.from("metadata-for-some-key").pack(), result.getMetadata(), "metadata was added");
    ResolverResult resultFromScoped = ScopedDirectoryLayer.global(database).resolveWithMetadata("some-key", /* no hooks */
    ResolverCreateHooks.getDefault()).join();
    assertEquals(resultFromScoped.getValue(), result.getValue());
    assertThat(resultFromScoped.getMetadata(), is(nullValue()));
    try (FDBRecordContext context = database.openContext()) {
        assertArrayEquals(globalScope.mustResolveWithMetadata(context, "some-key").join().getMetadata(), Tuple.from("metadata-for-some-key").pack(), "we can still read the metadata with " + globalScope.getClass().getName());
    }
}
Also used : Assertions.assertThrows(org.junit.jupiter.api.Assertions.assertThrows) MetadataHook(com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook) FDBRecordContext(com.apple.foundationdb.record.provider.foundationdb.FDBRecordContext) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) AsyncUtil(com.apple.foundationdb.async.AsyncUtil) Function(java.util.function.Function) ArrayList(java.util.ArrayList) ImmutableBiMap(com.google.common.collect.ImmutableBiMap) Tuple(com.apple.foundationdb.tuple.Tuple) STRING(com.apple.foundationdb.record.provider.foundationdb.keyspace.KeySpaceDirectory.KeyType.STRING) Map(java.util.Map) Matchers.nullValue(org.hamcrest.Matchers.nullValue) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) BiMap(com.google.common.collect.BiMap) FDBStoreTimer(com.apple.foundationdb.record.provider.foundationdb.FDBStoreTimer) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) DirectoryLayer(com.apple.foundationdb.directory.DirectoryLayer) FDBDatabase(com.apple.foundationdb.record.provider.foundationdb.FDBDatabase) Test(org.junit.jupiter.api.Test) Assertions.assertArrayEquals(org.junit.jupiter.api.Assertions.assertArrayEquals) AtomicLong(java.util.concurrent.atomic.AtomicLong) List(java.util.List) Matchers.is(org.hamcrest.Matchers.is) Collections(java.util.Collections) FDBRecordContext(com.apple.foundationdb.record.provider.foundationdb.FDBRecordContext) MetadataHook(com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook) Test(org.junit.jupiter.api.Test)

Example 4 with MetadataHook

use of com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook in project fdb-record-layer by FoundationDB.

the class ResolverMappingDigestTest method testComputeDigest.

private void testComputeDigest(LocatableResolver primary, LocatableResolver replica, boolean allowMetadata) throws Exception {
    SortedMap<String, ResolverResult> mappings = new TreeMap<>();
    ResolverResult result;
    for (int i = 0; i < 10; i++) {
        String key = "some-key-" + i;
        // if we allow metadata, set the metadata ~1/2 the time
        boolean metadataForThisKey = allowMetadata && random.nextBoolean();
        byte[] metadata = metadataForThisKey ? Tuple.from("some metadata for key: " + key).pack() : null;
        MetadataHook hook = ignore -> metadata;
        result = primary.resolveWithMetadata(key, new ResolverCreateHooks(DEFAULT_CHECK, hook)).join();
        mappings.put(key, result);
    }
    ResolverMappingReplicator replicator = new ResolverMappingReplicator(primary);
    replicator.copyTo(replica);
    final byte[] expectedDigest = expectedDigest(mappings);
    final byte[] wrongKeyDigest = wrongKeyDigest(mappings);
    final byte[] wrongValueDigest = wrongValueDigest(mappings);
    final byte[] wrongMetadataDigest = wrongMetadata(mappings);
    final byte[] extraEntryDigest = extraEntry(mappings);
    ResolverMappingDigest primaryResolverMappingDigest = new ResolverMappingDigest(primary);
    ResolverMappingDigest replicaResolverMappingDigest = new ResolverMappingDigest(replica);
    assertThat("digests match only if key, value and metadata are the same (independent of scope)", primaryResolverMappingDigest.computeDigest().join(), allOf(is(expectedDigest), is(replicaResolverMappingDigest.computeDigest().join()), is(not(wrongKeyDigest)), is(not(wrongValueDigest)), is(not(wrongMetadataDigest)), is(not(extraEntryDigest))));
}
Also used : MetadataHook(com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook) BeforeEach(org.junit.jupiter.api.BeforeEach) Tags(com.apple.test.Tags) MessageDigest(java.security.MessageDigest) Matchers.allOf(org.hamcrest.Matchers.allOf) Matchers.not(org.hamcrest.Matchers.not) FDBRecordContext(com.apple.foundationdb.record.provider.foundationdb.FDBRecordContext) Random(java.util.Random) FDBDatabase(com.apple.foundationdb.record.provider.foundationdb.FDBDatabase) ScopedInterningLayer(com.apple.foundationdb.record.provider.foundationdb.layers.interning.ScopedInterningLayer) FDBTestBase(com.apple.foundationdb.record.provider.foundationdb.FDBTestBase) Test(org.junit.jupiter.api.Test) KeyType(com.apple.foundationdb.record.provider.foundationdb.keyspace.KeySpaceDirectory.KeyType) Tuple(com.apple.foundationdb.tuple.Tuple) TreeMap(java.util.TreeMap) Map(java.util.Map) FDBDatabaseFactory(com.apple.foundationdb.record.provider.foundationdb.FDBDatabaseFactory) BiConsumer(java.util.function.BiConsumer) Matchers.is(org.hamcrest.Matchers.is) Tag(org.junit.jupiter.api.Tag) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) ImmutableSortedMap(com.google.common.collect.ImmutableSortedMap) SortedMap(java.util.SortedMap) DEFAULT_CHECK(com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.DEFAULT_CHECK) TreeMap(java.util.TreeMap) MetadataHook(com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook)

Example 5 with MetadataHook

use of com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook in project fdb-record-layer by FoundationDB.

the class LocatableResolverTest method testResolveWithMetadata.

@Test
public void testResolveWithMetadata() {
    byte[] metadata = Tuple.from("some-metadata").pack();
    MetadataHook hook = ignore -> metadata;
    final ResolverResult result;
    final ResolverCreateHooks hooks = new ResolverCreateHooks(DEFAULT_CHECK, hook);
    result = globalScope.resolveWithMetadata("a-key", hooks).join();
    assertArrayEquals(metadata, result.getMetadata());
    // check that the result with metadata is persisted to the database
    ResolverResult expected = new ResolverResult(result.getValue(), metadata);
    try (FDBRecordContext context = database.openContext()) {
        ResolverResult resultFromDB = globalScope.mustResolveWithMetadata(context, "a-key").join();
        assertEquals(expected.getValue(), resultFromDB.getValue());
        assertArrayEquals(expected.getMetadata(), resultFromDB.getMetadata());
    }
    assertEquals(expected, globalScope.resolveWithMetadata("a-key", hooks).join());
    byte[] newMetadata = Tuple.from("some-different-metadata").pack();
    MetadataHook newHook = ignore -> newMetadata;
    final ResolverCreateHooks newHooks = new ResolverCreateHooks(DEFAULT_CHECK, newHook);
    // make sure we don't just read the cached value
    database.clearCaches();
    assertArrayEquals(metadata, globalScope.resolveWithMetadata("a-key", newHooks).join().getMetadata(), "hook is only run on create, does not update metadata");
}
Also used : BeforeEach(org.junit.jupiter.api.BeforeEach) CoreMatchers.hasItem(org.hamcrest.CoreMatchers.hasItem) BiFunction(java.util.function.BiFunction) Matchers.not(org.hamcrest.Matchers.not) FDBRecordContext(com.apple.foundationdb.record.provider.foundationdb.FDBRecordContext) Random(java.util.Random) LocatableResolverLockedException(com.apple.foundationdb.record.provider.foundationdb.keyspace.LocatableResolver.LocatableResolverLockedException) TestHelpers.consistently(com.apple.foundationdb.record.TestHelpers.consistently) Matchers.hasKey(org.hamcrest.Matchers.hasKey) ExceptionMessageMatcher.hasMessageContaining(com.apple.foundationdb.record.TestHelpers.ExceptionMessageMatcher.hasMessageContaining) CoreMatchers.instanceOf(org.hamcrest.CoreMatchers.instanceOf) Tuple(com.apple.foundationdb.tuple.Tuple) Pair(org.apache.commons.lang3.tuple.Pair) RecordCoreException(com.apple.foundationdb.record.RecordCoreException) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Map(java.util.Map) Is.is(org.hamcrest.core.Is.is) Assertions.assertAll(org.junit.jupiter.api.Assertions.assertAll) Tag(org.junit.jupiter.api.Tag) FDBStoreTimer(com.apple.foundationdb.record.provider.foundationdb.FDBStoreTimer) Matchers.allOf(org.hamcrest.Matchers.allOf) Matchers.lessThanOrEqualTo(org.hamcrest.Matchers.lessThanOrEqualTo) Set(java.util.Set) CompletionException(java.util.concurrent.CompletionException) FDBDatabase(com.apple.foundationdb.record.provider.foundationdb.FDBDatabase) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) FDBRecordContextConfig(com.apple.foundationdb.record.provider.foundationdb.FDBRecordContextConfig) Test(org.junit.jupiter.api.Test) Objects(java.util.Objects) List(java.util.List) Matchers.contains(org.hamcrest.Matchers.contains) Matchers.equalTo(org.hamcrest.Matchers.equalTo) Optional(java.util.Optional) Matchers.greaterThan(org.hamcrest.Matchers.greaterThan) CacheBuilder(com.google.common.cache.CacheBuilder) DEFAULT_CHECK(com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.DEFAULT_CHECK) CacheStats(com.google.common.cache.CacheStats) FDBDatabaseFactoryImpl(com.apple.foundationdb.record.provider.foundationdb.FDBDatabaseFactoryImpl) IntStream(java.util.stream.IntStream) Assertions.assertThrows(org.junit.jupiter.api.Assertions.assertThrows) Assertions.fail(org.junit.jupiter.api.Assertions.fail) MetadataHook(com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) AsyncUtil(com.apple.foundationdb.async.AsyncUtil) Supplier(java.util.function.Supplier) FDBTestBase(com.apple.foundationdb.record.provider.foundationdb.FDBTestBase) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) ExecuteProperties(com.apple.foundationdb.record.ExecuteProperties) TestHelpers.eventually(com.apple.foundationdb.record.TestHelpers.eventually) KeyType(com.apple.foundationdb.record.provider.foundationdb.keyspace.KeySpaceDirectory.KeyType) ImmutableList(com.google.common.collect.ImmutableList) RegisterExtension(org.junit.jupiter.api.extension.RegisterExtension) ScanProperties(com.apple.foundationdb.record.ScanProperties) BooleanSource(com.apple.test.BooleanSource) Matchers.hasSize(org.hamcrest.Matchers.hasSize) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) NoSuchElementException(java.util.NoSuchElementException) Nonnull(javax.annotation.Nonnull) CoreMatchers.nullValue(org.hamcrest.CoreMatchers.nullValue) MoreAsyncUtil(com.apple.foundationdb.async.MoreAsyncUtil) Matchers.greaterThanOrEqualTo(org.hamcrest.Matchers.greaterThanOrEqualTo) Tags(com.apple.test.Tags) TimeUnit(java.util.concurrent.TimeUnit) DEFAULT_HOOK(com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.DEFAULT_HOOK) Assertions.assertArrayEquals(org.junit.jupiter.api.Assertions.assertArrayEquals) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) PreWriteCheck(com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.PreWriteCheck) FDBDatabaseFactory(com.apple.foundationdb.record.provider.foundationdb.FDBDatabaseFactory) Cache(com.google.common.cache.Cache) Collections(java.util.Collections) FDBRecordContext(com.apple.foundationdb.record.provider.foundationdb.FDBRecordContext) MetadataHook(com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook) Test(org.junit.jupiter.api.Test) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Aggregations

MetadataHook (com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.MetadataHook)5 FDBRecordContext (com.apple.foundationdb.record.provider.foundationdb.FDBRecordContext)4 Tuple (com.apple.foundationdb.tuple.Tuple)4 Map (java.util.Map)4 MatcherAssert.assertThat (org.hamcrest.MatcherAssert.assertThat)4 Test (org.junit.jupiter.api.Test)4 FDBDatabase (com.apple.foundationdb.record.provider.foundationdb.FDBDatabase)3 KeyType (com.apple.foundationdb.record.provider.foundationdb.keyspace.KeySpaceDirectory.KeyType)3 Tags (com.apple.test.Tags)3 HashMap (java.util.HashMap)3 Matchers.not (org.hamcrest.Matchers.not)3 Tag (org.junit.jupiter.api.Tag)3 AsyncUtil (com.apple.foundationdb.async.AsyncUtil)2 DirectoryLayer (com.apple.foundationdb.directory.DirectoryLayer)2 FDBDatabaseFactory (com.apple.foundationdb.record.provider.foundationdb.FDBDatabaseFactory)2 FDBStoreTimer (com.apple.foundationdb.record.provider.foundationdb.FDBStoreTimer)2 FDBTestBase (com.apple.foundationdb.record.provider.foundationdb.FDBTestBase)2 DEFAULT_CHECK (com.apple.foundationdb.record.provider.foundationdb.keyspace.ResolverCreateHooks.DEFAULT_CHECK)2 ImmutableList (com.google.common.collect.ImmutableList)2 ArrayList (java.util.ArrayList)2