Examples with StoredSchema - org.apache.pulsar.common.protocol.schema.StoredSchema

Example 1 with StoredSchema

use of org.apache.pulsar.common.protocol.schema.StoredSchema in project pulsar by apache.

the class BookkeeperSchemaStorage method getSchema.

@NotNull
private CompletableFuture<StoredSchema> getSchema(String schemaId) {
    // There's already a schema read operation in progress. Just piggyback on that
    return readSchemaOperations.computeIfAbsent(schemaId, key -> {
        if (log.isDebugEnabled()) {
            log.debug("[{}] Fetching schema from store", schemaId);
        }
        CompletableFuture<StoredSchema> future = new CompletableFuture<>();
        getSchemaLocator(getSchemaPath(schemaId)).thenCompose(locator -> {
            if (log.isDebugEnabled()) {
                log.debug("[{}] Got schema locator {}", schemaId, locator);
            }
            if (!locator.isPresent()) {
                return completedFuture(null);
            }
            SchemaStorageFormat.SchemaLocator schemaLocator = locator.get().locator;
            return readSchemaEntry(schemaLocator.getInfo().getPosition()).thenApply(entry -> new StoredSchema(entry.getSchemaData().toByteArray(), new LongSchemaVersion(schemaLocator.getInfo().getVersion())));
        }).handleAsync((res, ex) -> {
            if (log.isDebugEnabled()) {
                log.debug("[{}] Get operation completed. res={} -- ex={}", schemaId, res, ex);
            }
            // Cleanup the pending ops from the map
            readSchemaOperations.remove(schemaId, future);
            if (ex != null) {
                future.completeExceptionally(ex);
            } else {
                future.complete(res);
            }
            return null;
        });
        return future;
    });
}

Also used : SchemaException(org.apache.pulsar.broker.service.schema.exceptions.SchemaException) StoredSchema(org.apache.pulsar.common.protocol.schema.StoredSchema) CompletableFuture.completedFuture(java.util.concurrent.CompletableFuture.completedFuture) LoggerFactory(org.slf4j.LoggerFactory) CompletableFuture(java.util.concurrent.CompletableFuture) MetadataStoreExtended(org.apache.pulsar.metadata.api.extended.MetadataStoreExtended) ByteBuffer(java.nio.ByteBuffer) ArrayList(java.util.ArrayList) ConcurrentMap(java.util.concurrent.ConcurrentMap) HashSet(java.util.HashSet) MetadataCache(org.apache.pulsar.metadata.api.MetadataCache) SchemaStorage(org.apache.pulsar.common.protocol.schema.SchemaStorage) Iterables.concat(com.google.common.collect.Iterables.concat) MetadataSerde(org.apache.pulsar.metadata.api.MetadataSerde) Map(java.util.Map) Objects.isNull(java.util.Objects.isNull) LedgerHandle(org.apache.bookkeeper.client.LedgerHandle) LedgerEntry(org.apache.bookkeeper.client.LedgerEntry) MetadataStoreException(org.apache.pulsar.metadata.api.MetadataStoreException) Stat(org.apache.pulsar.metadata.api.Stat) Logger(org.slf4j.Logger) AlreadyExistsException(org.apache.pulsar.metadata.api.MetadataStoreException.AlreadyExistsException) ServiceConfiguration(org.apache.pulsar.broker.ServiceConfiguration) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Set(java.util.Set) IOException(java.io.IOException) CompletionException(java.util.concurrent.CompletionException) NotNull(javax.validation.constraints.NotNull) BookKeeper(org.apache.bookkeeper.client.BookKeeper) PulsarService(org.apache.pulsar.broker.PulsarService) SchemaVersion(org.apache.pulsar.common.protocol.schema.SchemaVersion) Collectors(java.util.stream.Collectors) BKException(org.apache.bookkeeper.client.BKException) ExecutionException(java.util.concurrent.ExecutionException) List(java.util.List) FutureUtil(org.apache.pulsar.common.util.FutureUtil) Lists.newArrayList(com.google.common.collect.Lists.newArrayList) LongSchemaVersion(org.apache.pulsar.common.schema.LongSchemaVersion) BadVersionException(org.apache.pulsar.metadata.api.MetadataStoreException.BadVersionException) Optional(java.util.Optional) VisibleForTesting(com.google.common.annotations.VisibleForTesting) Functions.newSchemaEntry(org.apache.pulsar.broker.service.schema.BookkeeperSchemaStorage.Functions.newSchemaEntry) LedgerMetadataUtils(org.apache.bookkeeper.mledger.impl.LedgerMetadataUtils) Collections(java.util.Collections) ByteString.copyFrom(com.google.protobuf.ByteString.copyFrom) CompletableFuture(java.util.concurrent.CompletableFuture) StoredSchema(org.apache.pulsar.common.protocol.schema.StoredSchema) LongSchemaVersion(org.apache.pulsar.common.schema.LongSchemaVersion) NotNull(javax.validation.constraints.NotNull)

Example 2 with StoredSchema

use of org.apache.pulsar.common.protocol.schema.StoredSchema in project pulsar by yahoo.

the class BookkeeperSchemaStorage method getAll.

@Override
public CompletableFuture<List<CompletableFuture<StoredSchema>>> getAll(String key) {
    CompletableFuture<List<CompletableFuture<StoredSchema>>> result = new CompletableFuture<>();
    getLocator(key).thenAccept(locator -> {
        if (log.isDebugEnabled()) {
            log.debug("[{}] Get all schemas - locator: {}", key, locator);
        }
        if (!locator.isPresent()) {
            result.complete(Collections.emptyList());
            return;
        }
        SchemaStorageFormat.SchemaLocator schemaLocator = locator.get().locator;
        List<CompletableFuture<StoredSchema>> list = new ArrayList<>();
        schemaLocator.getIndexList().forEach(indexEntry -> list.add(readSchemaEntry(indexEntry.getPosition()).thenApply(entry -> new StoredSchema(entry.getSchemaData().toByteArray(), new LongSchemaVersion(indexEntry.getVersion())))));
        result.complete(list);
    });
    return result;
}

Also used : CompletableFuture(java.util.concurrent.CompletableFuture) StoredSchema(org.apache.pulsar.common.protocol.schema.StoredSchema) ArrayList(java.util.ArrayList) Lists.newArrayList(com.google.common.collect.Lists.newArrayList) ArrayList(java.util.ArrayList) List(java.util.List) Lists.newArrayList(com.google.common.collect.Lists.newArrayList) LongSchemaVersion(org.apache.pulsar.common.schema.LongSchemaVersion)

Example 3 with StoredSchema

use of org.apache.pulsar.common.protocol.schema.StoredSchema in project incubator-pulsar by apache.

the class BookkeeperSchemaStorage method getSchema.

@NotNull
private CompletableFuture<StoredSchema> getSchema(String schemaId) {
    // There's already a schema read operation in progress. Just piggyback on that
    return readSchemaOperations.computeIfAbsent(schemaId, key -> {
        if (log.isDebugEnabled()) {
            log.debug("[{}] Fetching schema from store", schemaId);
        }
        CompletableFuture<StoredSchema> future = new CompletableFuture<>();
        getSchemaLocator(getSchemaPath(schemaId)).thenCompose(locator -> {
            if (log.isDebugEnabled()) {
                log.debug("[{}] Got schema locator {}", schemaId, locator);
            }
            if (!locator.isPresent()) {
                return completedFuture(null);
            }
            SchemaStorageFormat.SchemaLocator schemaLocator = locator.get().locator;
            return readSchemaEntry(schemaLocator.getInfo().getPosition()).thenApply(entry -> new StoredSchema(entry.getSchemaData().toByteArray(), new LongSchemaVersion(schemaLocator.getInfo().getVersion())));
        }).handleAsync((res, ex) -> {
            if (log.isDebugEnabled()) {
                log.debug("[{}] Get operation completed. res={} -- ex={}", schemaId, res, ex);
            }
            // Cleanup the pending ops from the map
            readSchemaOperations.remove(schemaId, future);
            if (ex != null) {
                future.completeExceptionally(ex);
            } else {
                future.complete(res);
            }
            return null;
        });
        return future;
    });
}

Example 4 with StoredSchema

use of org.apache.pulsar.common.protocol.schema.StoredSchema in project incubator-pulsar by apache.

the class BookkeeperSchemaStorage method getAll.

@Override
public CompletableFuture<List<CompletableFuture<StoredSchema>>> getAll(String key) {
    CompletableFuture<List<CompletableFuture<StoredSchema>>> result = new CompletableFuture<>();
    getLocator(key).thenAccept(locator -> {
        if (log.isDebugEnabled()) {
            log.debug("[{}] Get all schemas - locator: {}", key, locator);
        }
        if (!locator.isPresent()) {
            result.complete(Collections.emptyList());
            return;
        }
        SchemaStorageFormat.SchemaLocator schemaLocator = locator.get().locator;
        List<CompletableFuture<StoredSchema>> list = new ArrayList<>();
        schemaLocator.getIndexList().forEach(indexEntry -> list.add(readSchemaEntry(indexEntry.getPosition()).thenApply(entry -> new StoredSchema(entry.getSchemaData().toByteArray(), new LongSchemaVersion(indexEntry.getVersion())))));
        result.complete(list);
    });
    return result;
}

Example 5 with StoredSchema

use of org.apache.pulsar.common.protocol.schema.StoredSchema in project pulsar by apache.

the class BookkeeperSchemaStorage method getAll.

@Override
public CompletableFuture<List<CompletableFuture<StoredSchema>>> getAll(String key) {
    CompletableFuture<List<CompletableFuture<StoredSchema>>> result = new CompletableFuture<>();
    getLocator(key).thenAccept(locator -> {
        if (log.isDebugEnabled()) {
            log.debug("[{}] Get all schemas - locator: {}", key, locator);
        }
        if (!locator.isPresent()) {
            result.complete(Collections.emptyList());
            return;
        }
        SchemaStorageFormat.SchemaLocator schemaLocator = locator.get().locator;
        List<CompletableFuture<StoredSchema>> list = new ArrayList<>();
        schemaLocator.getIndexList().forEach(indexEntry -> list.add(readSchemaEntry(indexEntry.getPosition()).thenApply(entry -> new StoredSchema(entry.getSchemaData().toByteArray(), new LongSchemaVersion(indexEntry.getVersion())))));
        result.complete(list);
    });
    return result;
}

Aggregations

Lists.newArrayList (com.google.common.collect.Lists.newArrayList)6 ArrayList (java.util.ArrayList)6 List (java.util.List)6 CompletableFuture (java.util.concurrent.CompletableFuture)6 StoredSchema (org.apache.pulsar.common.protocol.schema.StoredSchema)6 LongSchemaVersion (org.apache.pulsar.common.schema.LongSchemaVersion)6 VisibleForTesting (com.google.common.annotations.VisibleForTesting)3 Iterables.concat (com.google.common.collect.Iterables.concat)3 ByteString.copyFrom (com.google.protobuf.ByteString.copyFrom)3 IOException (java.io.IOException)3 ByteBuffer (java.nio.ByteBuffer)3 Collections (java.util.Collections)3 HashSet (java.util.HashSet)3 Map (java.util.Map)3 Objects.isNull (java.util.Objects.isNull)3 Optional (java.util.Optional)3 Set (java.util.Set)3 CompletableFuture.completedFuture (java.util.concurrent.CompletableFuture.completedFuture)3 CompletionException (java.util.concurrent.CompletionException)3 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)3