Search in sources :

Example 16 with Notification

use of org.apache.pulsar.metadata.api.Notification in project incubator-pulsar by apache.

the class EtcdMetadataStore method handleWatchResponse.

private void handleWatchResponse(WatchResponse watchResponse) {
    watchResponse.getEvents().forEach(we -> {
        String path = we.getKeyValue().getKey().toString(StandardCharsets.UTF_8);
        if (we.getEventType() == WatchEvent.EventType.PUT) {
            if (we.getKeyValue().getVersion() == 1) {
                receivedNotification(new Notification(NotificationType.Created, path));
                notifyParentChildrenChanged(path);
            } else {
                receivedNotification(new Notification(NotificationType.Modified, path));
            }
        } else if (we.getEventType() == WatchEvent.EventType.DELETE) {
            receivedNotification(new Notification(NotificationType.Deleted, path));
            notifyParentChildrenChanged(path);
        }
    });
}
Also used : Notification(org.apache.pulsar.metadata.api.Notification)

Example 17 with Notification

use of org.apache.pulsar.metadata.api.Notification in project pulsar by yahoo.

the class RocksdbMetadataStore method storePut.

@Override
protected CompletableFuture<Stat> storePut(String path, byte[] data, Optional<Long> expectedVersion, EnumSet<CreateOption> options) {
    if (log.isDebugEnabled()) {
        log.debug("storePut.path={},instanceId={}", path, instanceId);
    }
    try {
        dbStateLock.readLock().lock();
        if (state == State.CLOSED) {
            throw new MetadataStoreException.AlreadyClosedException("");
        }
        try (Transaction transaction = db.beginTransaction(optionSync)) {
            byte[] pathBytes = toBytes(path);
            byte[] oldValueData = transaction.getForUpdate(optionDontCache, pathBytes, false);
            MetaValue metaValue = MetaValue.parse(oldValueData);
            if (expectedVersion.isPresent()) {
                if (metaValue == null && expectedVersion.get() != -1 || metaValue != null && !expectedVersion.get().equals(metaValue.getVersion())) {
                    throw new MetadataStoreException.BadVersionException(String.format("Version mismatch, actual=%s, expect=%s", metaValue == null ? null : metaValue.getVersion(), expectedVersion.get()));
                }
            }
            boolean created = false;
            long timestamp = System.currentTimeMillis();
            if (metaValue == null) {
                // create new node
                metaValue = new MetaValue();
                metaValue.version = 0;
                metaValue.createdTimestamp = timestamp;
                metaValue.ephemeral = options.contains(CreateOption.Ephemeral);
                if (options.contains(CreateOption.Sequential)) {
                    path += sequentialIdGenerator.getAndIncrement();
                    pathBytes = toBytes(path);
                    transaction.put(SEQUENTIAL_ID_KEY, toBytes(sequentialIdGenerator.get()));
                }
                created = true;
            } else {
                // update old node
                metaValue.version++;
            }
            metaValue.modifiedTimestamp = timestamp;
            metaValue.owner = instanceId;
            metaValue.data = data;
            // handle Sequential
            transaction.put(pathBytes, metaValue.serialize());
            transaction.commit();
            receivedNotification(new Notification(created ? NotificationType.Created : NotificationType.Modified, path));
            if (created) {
                notifyParentChildrenChanged(path);
            }
            return CompletableFuture.completedFuture(new Stat(path, metaValue.version, metaValue.createdTimestamp, metaValue.modifiedTimestamp, metaValue.ephemeral, true));
        }
    } catch (Throwable e) {
        return FutureUtil.failedFuture(MetadataStoreException.wrap(e));
    } finally {
        dbStateLock.readLock().unlock();
    }
}
Also used : Stat(org.apache.pulsar.metadata.api.Stat) Transaction(org.rocksdb.Transaction) Notification(org.apache.pulsar.metadata.api.Notification)

Example 18 with Notification

use of org.apache.pulsar.metadata.api.Notification in project pulsar by yahoo.

the class AbstractMetadataStore method notifyParentChildrenChanged.

protected void notifyParentChildrenChanged(String path) {
    String parent = parent(path);
    while (parent != null) {
        receivedNotification(new Notification(NotificationType.ChildrenChanged, parent));
        parent = parent(parent);
    }
}
Also used : Notification(org.apache.pulsar.metadata.api.Notification)

Example 19 with Notification

use of org.apache.pulsar.metadata.api.Notification in project pulsar by apache.

the class RocksdbMetadataStore method storePut.

@Override
protected CompletableFuture<Stat> storePut(String path, byte[] data, Optional<Long> expectedVersion, EnumSet<CreateOption> options) {
    if (log.isDebugEnabled()) {
        log.debug("storePut.path={},instanceId={}", path, instanceId);
    }
    try {
        dbStateLock.readLock().lock();
        if (state == State.CLOSED) {
            throw new MetadataStoreException.AlreadyClosedException("");
        }
        try (Transaction transaction = db.beginTransaction(optionSync)) {
            byte[] pathBytes = toBytes(path);
            byte[] oldValueData = transaction.getForUpdate(optionDontCache, pathBytes, false);
            MetaValue metaValue = MetaValue.parse(oldValueData);
            if (expectedVersion.isPresent()) {
                if (metaValue == null && expectedVersion.get() != -1 || metaValue != null && !expectedVersion.get().equals(metaValue.getVersion())) {
                    throw new MetadataStoreException.BadVersionException(String.format("Version mismatch, actual=%s, expect=%s", metaValue == null ? null : metaValue.getVersion(), expectedVersion.get()));
                }
            }
            boolean created = false;
            long timestamp = System.currentTimeMillis();
            if (metaValue == null) {
                // create new node
                metaValue = new MetaValue();
                metaValue.version = 0;
                metaValue.createdTimestamp = timestamp;
                metaValue.ephemeral = options.contains(CreateOption.Ephemeral);
                if (options.contains(CreateOption.Sequential)) {
                    path += sequentialIdGenerator.getAndIncrement();
                    pathBytes = toBytes(path);
                    transaction.put(SEQUENTIAL_ID_KEY, toBytes(sequentialIdGenerator.get()));
                }
                created = true;
            } else {
                // update old node
                metaValue.version++;
            }
            metaValue.modifiedTimestamp = timestamp;
            metaValue.owner = instanceId;
            metaValue.data = data;
            // handle Sequential
            transaction.put(pathBytes, metaValue.serialize());
            transaction.commit();
            receivedNotification(new Notification(created ? NotificationType.Created : NotificationType.Modified, path));
            if (created) {
                notifyParentChildrenChanged(path);
            }
            return CompletableFuture.completedFuture(new Stat(path, metaValue.version, metaValue.createdTimestamp, metaValue.modifiedTimestamp, metaValue.ephemeral, true));
        }
    } catch (Throwable e) {
        return FutureUtil.failedFuture(MetadataStoreException.wrap(e));
    } finally {
        dbStateLock.readLock().unlock();
    }
}
Also used : Stat(org.apache.pulsar.metadata.api.Stat) Transaction(org.rocksdb.Transaction) Notification(org.apache.pulsar.metadata.api.Notification)

Example 20 with Notification

use of org.apache.pulsar.metadata.api.Notification in project pulsar by apache.

the class RocksdbMetadataStore method storeDelete.

@Override
protected CompletableFuture<Void> storeDelete(String path, Optional<Long> expectedVersion) {
    if (log.isDebugEnabled()) {
        log.debug("storeDelete.path={},instanceId={}", path, instanceId);
    }
    try {
        dbStateLock.readLock().lock();
        if (state == State.CLOSED) {
            throw new MetadataStoreException.AlreadyClosedException("");
        }
        try (Transaction transaction = db.beginTransaction(optionSync)) {
            byte[] pathBytes = toBytes(path);
            byte[] oldValueData = transaction.getForUpdate(optionDontCache, pathBytes, false);
            MetaValue metaValue = MetaValue.parse(oldValueData);
            if (metaValue == null) {
                throw new MetadataStoreException.NotFoundException(String.format("path %s not found.", path));
            }
            if (expectedVersion.isPresent() && !expectedVersion.get().equals(metaValue.getVersion())) {
                throw new MetadataStoreException.BadVersionException(String.format("Version mismatch, actual=%s, expect=%s", metaValue.getVersion(), expectedVersion.get()));
            }
            transaction.delete(pathBytes);
            transaction.commit();
            receivedNotification(new Notification(NotificationType.Deleted, path));
            notifyParentChildrenChanged(path);
            return CompletableFuture.completedFuture(null);
        }
    } catch (Throwable e) {
        return FutureUtil.failedFuture(MetadataStoreException.wrap(e));
    } finally {
        dbStateLock.readLock().unlock();
    }
}
Also used : Transaction(org.rocksdb.Transaction) Notification(org.apache.pulsar.metadata.api.Notification)

Aggregations

Notification (org.apache.pulsar.metadata.api.Notification)25 Stat (org.apache.pulsar.metadata.api.Stat)9 NotificationType (org.apache.pulsar.metadata.api.NotificationType)6 Transaction (org.rocksdb.Transaction)6 Test (org.testng.annotations.Test)6 MetadataStoreException (org.apache.pulsar.metadata.api.MetadataStoreException)4 BadVersionException (org.apache.pulsar.metadata.api.MetadataStoreException.BadVersionException)4 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)3 LinkedBlockingDeque (java.util.concurrent.LinkedBlockingDeque)3 AtomicReference (java.util.concurrent.atomic.AtomicReference)3 Cleanup (lombok.Cleanup)3 Namespaces (org.apache.pulsar.client.admin.Namespaces)3 MetadataStore (org.apache.pulsar.metadata.api.MetadataStore)3 LocalBrokerData (org.apache.pulsar.policies.data.loadbalancer.LocalBrokerData)3 NamespaceBundleStats (org.apache.pulsar.policies.data.loadbalancer.NamespaceBundleStats)3 ResourceUsage (org.apache.pulsar.policies.data.loadbalancer.ResourceUsage)3 SystemResourceUsage (org.apache.pulsar.policies.data.loadbalancer.SystemResourceUsage)3 CompletableFuture (java.util.concurrent.CompletableFuture)2 BrokerData (org.apache.pulsar.policies.data.loadbalancer.BrokerData)2 TimeAverageBrokerData (org.apache.pulsar.policies.data.loadbalancer.TimeAverageBrokerData)2