Search in sources :

Example 26 with StoredGroupEntry

use of org.onosproject.net.group.StoredGroupEntry in project onos by opennetworkinglab.

the class DistributedGroupStore method activate.

@Activate
public void activate(ComponentContext context) {
    cfgService.registerProperties(getClass());
    modified(context);
    KryoNamespace.Builder kryoBuilder = new KryoNamespace.Builder().register(KryoNamespaces.API).nextId(KryoNamespaces.BEGIN_USER_CUSTOM_ID).register(GroupStoreMessage.class, GroupStoreMessage.Type.class, UpdateType.class, GroupStoreMessageSubjects.class, MultiValuedTimestamp.class, GroupStoreKeyMapKey.class, GroupStoreIdMapKey.class, GroupStoreMapKey.class);
    clusterMsgSerializer = kryoBuilder.build("GroupStore");
    Serializer serializer = Serializer.using(clusterMsgSerializer);
    messageHandlingExecutor = Executors.newFixedThreadPool(MESSAGE_HANDLER_THREAD_POOL_SIZE, groupedThreads("onos/store/group", "message-handlers", log));
    clusterCommunicator.addSubscriber(GroupStoreMessageSubjects.REMOTE_GROUP_OP_REQUEST, clusterMsgSerializer::deserialize, this::process, messageHandlingExecutor);
    log.debug("Creating Consistent map onos-group-store-keymap");
    groupStoreEntriesByKey = storageService.<GroupStoreKeyMapKey, StoredGroupEntry>consistentMapBuilder().withName("onos-group-store-keymap").withSerializer(serializer).build();
    groupStoreEntriesByKey.addListener(mapListener);
    log.debug("Current size of groupstorekeymap:{}", groupStoreEntriesByKey.size());
    synchronizeGroupStoreEntries();
    log.debug("Creating GroupStoreId Map From GroupStoreKey Map");
    matchGroupEntries();
    executor = newSingleThreadScheduledExecutor(groupedThreads("onos/group", "store", log));
    statusChangeListener = status -> {
        if (status == Status.ACTIVE) {
            executor.execute(this::matchGroupEntries);
        }
    };
    groupStoreEntriesByKey.addStatusChangeListener(statusChangeListener);
    log.debug("Creating Consistent map pendinggroupkeymap");
    auditPendingReqQueue = storageService.<GroupStoreKeyMapKey, StoredGroupEntry>consistentMapBuilder().withName("onos-pending-group-keymap").withSerializer(serializer).build();
    log.debug("Current size of pendinggroupkeymap:{}", auditPendingReqQueue.size());
    groupTopic = getOrCreateGroupTopic(serializer);
    groupTopic.subscribe(this::processGroupMessage);
    local = clusterService.getLocalNode().id();
    log.info("Started");
}
Also used : KryoNamespace(org.onlab.util.KryoNamespace) StoredGroupEntry(org.onosproject.net.group.StoredGroupEntry) Serializer(org.onosproject.store.service.Serializer) Activate(org.osgi.service.component.annotations.Activate)

Example 27 with StoredGroupEntry

use of org.onosproject.net.group.StoredGroupEntry in project onos by opennetworkinglab.

the class DistributedGroupStore method getStoredGroups.

private Iterable<StoredGroupEntry> getStoredGroups(DeviceId deviceId) {
    NodeId master = mastershipService.getMasterFor(deviceId);
    if (master == null && deviceService.isAvailable(deviceId)) {
        log.debug("Failed to getGroups: No master for {}", deviceId);
        return Collections.emptySet();
    }
    Set<StoredGroupEntry> storedGroups = getGroupStoreKeyMap().values().stream().filter(input -> input.deviceId().equals(deviceId)).collect(Collectors.toSet());
    return ImmutableSet.copyOf(storedGroups);
}
Also used : ConsistentMap(org.onosproject.store.service.ConsistentMap) DeviceService(org.onosproject.net.device.DeviceService) Tools.groupedThreads(org.onlab.util.Tools.groupedThreads) GroupOperation(org.onosproject.net.group.GroupOperation) DriverService(org.onosproject.net.driver.DriverService) ALLOW_EXTRANEOUS_GROUPS(org.onosproject.store.OsgiPropertyConstants.ALLOW_EXTRANEOUS_GROUPS) Executors.newSingleThreadScheduledExecutor(java.util.concurrent.Executors.newSingleThreadScheduledExecutor) StorageService(org.onosproject.store.service.StorageService) FluentIterable(com.google.common.collect.FluentIterable) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Map(java.util.Map) ApplicationId(org.onosproject.core.ApplicationId) KryoNamespaces(org.onosproject.store.serializers.KryoNamespaces) MastershipService(org.onosproject.mastership.MastershipService) NodeId(org.onosproject.cluster.NodeId) Serializer(org.onosproject.store.service.Serializer) Tools.get(org.onlab.util.Tools.get) ImmutableSet(com.google.common.collect.ImmutableSet) StoredGroupBucketEntry(org.onosproject.net.group.StoredGroupBucketEntry) Deactivate(org.osgi.service.component.annotations.Deactivate) Collection(java.util.Collection) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Set(java.util.Set) GroupEvent(org.onosproject.net.group.GroupEvent) Collectors(java.util.stream.Collectors) Sets(com.google.common.collect.Sets) Executors(java.util.concurrent.Executors) Objects(java.util.Objects) Versioned(org.onosproject.store.service.Versioned) List(java.util.List) GroupStoreDelegate(org.onosproject.net.group.GroupStoreDelegate) GARBAGE_COLLECT(org.onosproject.store.OsgiPropertyConstants.GARBAGE_COLLECT) GroupBuckets(org.onosproject.net.group.GroupBuckets) Type(org.onosproject.net.group.GroupEvent.Type) Entry(java.util.Map.Entry) ClusterCommunicationService(org.onosproject.store.cluster.messaging.ClusterCommunicationService) Optional(java.util.Optional) ClusterService(org.onosproject.cluster.ClusterService) DeviceId(org.onosproject.net.DeviceId) GARBAGE_COLLECT_THRESH_DEFAULT(org.onosproject.store.OsgiPropertyConstants.GARBAGE_COLLECT_THRESH_DEFAULT) DefaultGroupDescription(org.onosproject.net.group.DefaultGroupDescription) GroupDescription(org.onosproject.net.group.GroupDescription) Dictionary(java.util.Dictionary) Iterables(com.google.common.collect.Iterables) ComponentContext(org.osgi.service.component.ComponentContext) Strings.isNullOrEmpty(com.google.common.base.Strings.isNullOrEmpty) HashMap(java.util.HashMap) DefaultGroup(org.onosproject.net.group.DefaultGroup) GroupBucket(org.onosproject.net.group.GroupBucket) KryoNamespace(org.onlab.util.KryoNamespace) GroupKey(org.onosproject.net.group.GroupKey) Group(org.onosproject.net.group.Group) MapEventListener(org.onosproject.store.service.MapEventListener) ArrayList(java.util.ArrayList) ConcurrentMap(java.util.concurrent.ConcurrentMap) HashSet(java.util.HashSet) GroupStore(org.onosproject.net.group.GroupStore) Component(org.osgi.service.component.annotations.Component) StoredGroupEntry(org.onosproject.net.group.StoredGroupEntry) GroupState(org.onosproject.net.group.Group.GroupState) ALLOW_EXTRANEOUS_GROUPS_DEFAULT(org.onosproject.store.OsgiPropertyConstants.ALLOW_EXTRANEOUS_GROUPS_DEFAULT) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) Activate(org.osgi.service.component.annotations.Activate) ExecutorService(java.util.concurrent.ExecutorService) ComponentConfigService(org.onosproject.cfg.ComponentConfigService) Logger(org.slf4j.Logger) Properties(java.util.Properties) Iterator(java.util.Iterator) GARBAGE_COLLECT_DEFAULT(org.onosproject.store.OsgiPropertyConstants.GARBAGE_COLLECT_DEFAULT) Topic(org.onosproject.store.service.Topic) MastershipRole(org.onosproject.net.MastershipRole) GARBAGE_COLLECT_THRESH(org.onosproject.store.OsgiPropertyConstants.GARBAGE_COLLECT_THRESH) Status(org.onosproject.store.service.DistributedPrimitive.Status) ReferenceCardinality(org.osgi.service.component.annotations.ReferenceCardinality) Consumer(java.util.function.Consumer) MultiValuedTimestamp(org.onosproject.store.service.MultiValuedTimestamp) GroupId(org.onosproject.core.GroupId) MapEvent(org.onosproject.store.service.MapEvent) AbstractStore(org.onosproject.store.AbstractStore) Modified(org.osgi.service.component.annotations.Modified) LoggerFactory.getLogger(org.slf4j.LoggerFactory.getLogger) Reference(org.osgi.service.component.annotations.Reference) Collections(java.util.Collections) NodeId(org.onosproject.cluster.NodeId) StoredGroupEntry(org.onosproject.net.group.StoredGroupEntry)

Example 28 with StoredGroupEntry

use of org.onosproject.net.group.StoredGroupEntry in project onos by opennetworkinglab.

the class DistributedGroupStore method garbageCollect.

private void garbageCollect(DeviceId deviceId, Set<Group> southboundGroupEntries, Set<StoredGroupEntry> storedGroupEntries) {
    if (!garbageCollect) {
        return;
    }
    NodeId master;
    Iterator<StoredGroupEntry> it = storedGroupEntries.iterator();
    while (it.hasNext()) {
        // Mastership change can occur during this iteration
        master = mastershipService.getMasterFor(deviceId);
        if (!Objects.equals(local, master)) {
            log.warn("Tried to run garbage collector while the node was not the master");
            return;
        }
        StoredGroupEntry group = it.next();
        if (group.state() != GroupState.PENDING_DELETE && checkGroupRefCount(group)) {
            log.debug("Garbage collecting group {} on {}", group, deviceId);
            deleteGroupDescription(deviceId, group.appCookie());
            southboundGroupEntries.remove(group);
            it.remove();
        }
    }
}
Also used : NodeId(org.onosproject.cluster.NodeId) StoredGroupEntry(org.onosproject.net.group.StoredGroupEntry)

Example 29 with StoredGroupEntry

use of org.onosproject.net.group.StoredGroupEntry in project onos by opennetworkinglab.

the class DistributedGroupStore method deviceInitialAuditCompleted.

@Override
public void deviceInitialAuditCompleted(DeviceId deviceId, boolean completed) {
    synchronized (deviceAuditStatus) {
        if (completed) {
            log.debug("AUDIT completed for device {}", deviceId);
            deviceAuditStatus.put(deviceId, true);
            // Execute all pending group requests
            List<StoredGroupEntry> pendingGroupRequests = getPendingGroupKeyTable().values().stream().filter(g -> g.deviceId().equals(deviceId)).collect(Collectors.toList());
            if (log.isDebugEnabled()) {
                List<String> pendingIds = pendingGroupRequests.stream().map(GroupDescription::givenGroupId).map(id -> id != null ? "0x" + Integer.toHexString(id) : "N/A").collect(Collectors.toList());
                log.debug("processing pending group add requests for device {}: {}", deviceId, pendingIds);
            }
            NodeId master;
            for (Group group : pendingGroupRequests) {
                // Mastership change can occur during this iteration
                if (!shouldHandle(deviceId)) {
                    log.warn("Tried to process pending groups while the node was not the master" + " or the device {} was not available", deviceId);
                    return;
                }
                GroupDescription tmp = new DefaultGroupDescription(group.deviceId(), group.type(), group.buckets(), group.appCookie(), group.givenGroupId(), group.appId());
                storeGroupDescriptionInternal(tmp);
                getPendingGroupKeyTable().remove(new GroupStoreKeyMapKey(deviceId, group.appCookie()));
            }
        } else {
            Boolean audited = deviceAuditStatus.get(deviceId);
            if (audited != null && audited) {
                log.debug("Clearing AUDIT status for device {}", deviceId);
                deviceAuditStatus.put(deviceId, false);
            }
        }
    }
}
Also used : ConsistentMap(org.onosproject.store.service.ConsistentMap) DeviceService(org.onosproject.net.device.DeviceService) Tools.groupedThreads(org.onlab.util.Tools.groupedThreads) GroupOperation(org.onosproject.net.group.GroupOperation) DriverService(org.onosproject.net.driver.DriverService) ALLOW_EXTRANEOUS_GROUPS(org.onosproject.store.OsgiPropertyConstants.ALLOW_EXTRANEOUS_GROUPS) Executors.newSingleThreadScheduledExecutor(java.util.concurrent.Executors.newSingleThreadScheduledExecutor) StorageService(org.onosproject.store.service.StorageService) FluentIterable(com.google.common.collect.FluentIterable) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Map(java.util.Map) ApplicationId(org.onosproject.core.ApplicationId) KryoNamespaces(org.onosproject.store.serializers.KryoNamespaces) MastershipService(org.onosproject.mastership.MastershipService) NodeId(org.onosproject.cluster.NodeId) Serializer(org.onosproject.store.service.Serializer) Tools.get(org.onlab.util.Tools.get) ImmutableSet(com.google.common.collect.ImmutableSet) StoredGroupBucketEntry(org.onosproject.net.group.StoredGroupBucketEntry) Deactivate(org.osgi.service.component.annotations.Deactivate) Collection(java.util.Collection) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Set(java.util.Set) GroupEvent(org.onosproject.net.group.GroupEvent) Collectors(java.util.stream.Collectors) Sets(com.google.common.collect.Sets) Executors(java.util.concurrent.Executors) Objects(java.util.Objects) Versioned(org.onosproject.store.service.Versioned) List(java.util.List) GroupStoreDelegate(org.onosproject.net.group.GroupStoreDelegate) GARBAGE_COLLECT(org.onosproject.store.OsgiPropertyConstants.GARBAGE_COLLECT) GroupBuckets(org.onosproject.net.group.GroupBuckets) Type(org.onosproject.net.group.GroupEvent.Type) Entry(java.util.Map.Entry) ClusterCommunicationService(org.onosproject.store.cluster.messaging.ClusterCommunicationService) Optional(java.util.Optional) ClusterService(org.onosproject.cluster.ClusterService) DeviceId(org.onosproject.net.DeviceId) GARBAGE_COLLECT_THRESH_DEFAULT(org.onosproject.store.OsgiPropertyConstants.GARBAGE_COLLECT_THRESH_DEFAULT) DefaultGroupDescription(org.onosproject.net.group.DefaultGroupDescription) GroupDescription(org.onosproject.net.group.GroupDescription) Dictionary(java.util.Dictionary) Iterables(com.google.common.collect.Iterables) ComponentContext(org.osgi.service.component.ComponentContext) Strings.isNullOrEmpty(com.google.common.base.Strings.isNullOrEmpty) HashMap(java.util.HashMap) DefaultGroup(org.onosproject.net.group.DefaultGroup) GroupBucket(org.onosproject.net.group.GroupBucket) KryoNamespace(org.onlab.util.KryoNamespace) GroupKey(org.onosproject.net.group.GroupKey) Group(org.onosproject.net.group.Group) MapEventListener(org.onosproject.store.service.MapEventListener) ArrayList(java.util.ArrayList) ConcurrentMap(java.util.concurrent.ConcurrentMap) HashSet(java.util.HashSet) GroupStore(org.onosproject.net.group.GroupStore) Component(org.osgi.service.component.annotations.Component) StoredGroupEntry(org.onosproject.net.group.StoredGroupEntry) GroupState(org.onosproject.net.group.Group.GroupState) ALLOW_EXTRANEOUS_GROUPS_DEFAULT(org.onosproject.store.OsgiPropertyConstants.ALLOW_EXTRANEOUS_GROUPS_DEFAULT) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) Activate(org.osgi.service.component.annotations.Activate) ExecutorService(java.util.concurrent.ExecutorService) ComponentConfigService(org.onosproject.cfg.ComponentConfigService) Logger(org.slf4j.Logger) Properties(java.util.Properties) Iterator(java.util.Iterator) GARBAGE_COLLECT_DEFAULT(org.onosproject.store.OsgiPropertyConstants.GARBAGE_COLLECT_DEFAULT) Topic(org.onosproject.store.service.Topic) MastershipRole(org.onosproject.net.MastershipRole) GARBAGE_COLLECT_THRESH(org.onosproject.store.OsgiPropertyConstants.GARBAGE_COLLECT_THRESH) Status(org.onosproject.store.service.DistributedPrimitive.Status) ReferenceCardinality(org.osgi.service.component.annotations.ReferenceCardinality) Consumer(java.util.function.Consumer) MultiValuedTimestamp(org.onosproject.store.service.MultiValuedTimestamp) GroupId(org.onosproject.core.GroupId) MapEvent(org.onosproject.store.service.MapEvent) AbstractStore(org.onosproject.store.AbstractStore) Modified(org.osgi.service.component.annotations.Modified) LoggerFactory.getLogger(org.slf4j.LoggerFactory.getLogger) Reference(org.osgi.service.component.annotations.Reference) Collections(java.util.Collections) DefaultGroupDescription(org.onosproject.net.group.DefaultGroupDescription) GroupDescription(org.onosproject.net.group.GroupDescription) DefaultGroup(org.onosproject.net.group.DefaultGroup) Group(org.onosproject.net.group.Group) NodeId(org.onosproject.cluster.NodeId) DefaultGroupDescription(org.onosproject.net.group.DefaultGroupDescription) StoredGroupEntry(org.onosproject.net.group.StoredGroupEntry)

Example 30 with StoredGroupEntry

use of org.onosproject.net.group.StoredGroupEntry in project onos by opennetworkinglab.

the class DistributedGroupStore method groupOperationFailed.

@Override
public void groupOperationFailed(DeviceId deviceId, GroupOperation operation) {
    StoredGroupEntry existing = getStoredGroupEntry(deviceId, operation.groupId());
    if (existing == null) {
        log.warn("No group entry with ID {} found ", operation.groupId());
        return;
    }
    log.warn("groupOperationFailed: group operation {} failed in state {} " + "for group {} in device {} with code {}", operation.opType(), existing.state(), existing.id(), existing.deviceId(), operation.failureCode());
    if (operation.failureCode() == GroupOperation.GroupMsgErrorCode.GROUP_EXISTS) {
        if (operation.buckets().equals(existing.buckets())) {
            if (existing.state() == GroupState.PENDING_ADD || existing.state() == GroupState.PENDING_ADD_RETRY) {
                log.info("GROUP_EXISTS: GroupID and Buckets match for group in pending " + "add state - moving to ADDED for group {} in device {}", existing.id(), deviceId);
                addOrUpdateGroupEntry(existing);
                return;
            } else {
                log.warn("GROUP_EXISTS: GroupId and Buckets match but existing" + "group in state: {}", existing.state());
            }
        } else {
            log.warn("GROUP EXISTS: Group ID matched but buckets did not. " + "Operation: {} Existing: {}", operation.buckets(), existing.buckets());
        }
    }
    if (operation.failureCode() == GroupOperation.GroupMsgErrorCode.INVALID_GROUP) {
        existing.incrFailedRetryCount();
        if (existing.failedRetryCount() < MAX_FAILED_ATTEMPTS) {
            log.warn("Group {} programming failed {} of {} times in dev {}, " + "retrying ..", existing.id(), existing.failedRetryCount(), MAX_FAILED_ATTEMPTS, deviceId);
            return;
        }
        log.warn("Group {} programming failed {} of {} times in dev {}, " + "removing group from store", existing.id(), existing.failedRetryCount(), MAX_FAILED_ATTEMPTS, deviceId);
    // fall through to case
    }
    switch(operation.opType()) {
        case ADD:
            if (existing.state() == GroupState.PENDING_ADD || existing.state() == GroupState.PENDING_ADD_RETRY) {
                notifyDelegate(new GroupEvent(Type.GROUP_ADD_FAILED, existing));
                log.warn("groupOperationFailed: cleaning up " + "group {} from store in device {}....", existing.id(), existing.deviceId());
                // Removal from groupid based map will happen in the
                // map update listener
                getGroupStoreKeyMap().remove(new GroupStoreKeyMapKey(existing.deviceId(), existing.appCookie()));
            }
            break;
        case MODIFY:
            notifyDelegate(new GroupEvent(Type.GROUP_UPDATE_FAILED, existing));
            break;
        case DELETE:
            notifyDelegate(new GroupEvent(Type.GROUP_REMOVE_FAILED, existing));
            break;
        default:
            log.warn("Unknown group operation type {}", operation.opType());
    }
}
Also used : StoredGroupEntry(org.onosproject.net.group.StoredGroupEntry) GroupEvent(org.onosproject.net.group.GroupEvent)

Aggregations

StoredGroupEntry (org.onosproject.net.group.StoredGroupEntry)38 GroupEvent (org.onosproject.net.group.GroupEvent)26 DefaultGroup (org.onosproject.net.group.DefaultGroup)19 GroupKey (org.onosproject.net.group.GroupKey)18 GroupId (org.onosproject.core.GroupId)15 Group (org.onosproject.net.group.Group)13 GroupBucket (org.onosproject.net.group.GroupBucket)11 GroupBuckets (org.onosproject.net.group.GroupBuckets)11 DefaultGroupDescription (org.onosproject.net.group.DefaultGroupDescription)10 GroupDescription (org.onosproject.net.group.GroupDescription)10 ArrayList (java.util.ArrayList)8 StoredGroupBucketEntry (org.onosproject.net.group.StoredGroupBucketEntry)8 DeviceId (org.onosproject.net.DeviceId)6 Activate (org.osgi.service.component.annotations.Activate)6 FluentIterable (com.google.common.collect.FluentIterable)5 Sets (com.google.common.collect.Sets)5 Collection (java.util.Collection)5 HashMap (java.util.HashMap)5 Iterator (java.util.Iterator)5 List (java.util.List)5