Search in sources :

Example 11 with ConfiguredComponent

use of org.apache.nifi.controller.ConfiguredComponent in project nifi by apache.

the class StandardProcessGroup method getComponentsAffectedByVariable.

@Override
public Set<ConfiguredComponent> getComponentsAffectedByVariable(final String variableName) {
    final Set<ConfiguredComponent> affected = new HashSet<>();
    // Determine any Processors that references the variable
    for (final ProcessorNode processor : getProcessors()) {
        for (final VariableImpact impact : getVariableImpact(processor)) {
            if (impact.isImpacted(variableName)) {
                affected.add(processor);
            }
        }
    }
    // find any references to that service and add it.
    for (final ControllerServiceNode service : getControllerServices(false)) {
        for (final VariableImpact impact : getVariableImpact(service)) {
            if (impact.isImpacted(variableName)) {
                affected.add(service);
                final ControllerServiceReference reference = service.getReferences();
                affected.addAll(reference.findRecursiveReferences(ConfiguredComponent.class));
            }
        }
    }
    // is overriding the variable and its components are actually referencing a different variable.
    for (final ProcessGroup childGroup : getProcessGroups()) {
        final ComponentVariableRegistry childRegistry = childGroup.getVariableRegistry();
        final VariableDescriptor descriptor = childRegistry.getVariableKey(variableName);
        final boolean overridden = childRegistry.getVariableMap().containsKey(descriptor);
        if (!overridden) {
            affected.addAll(childGroup.getComponentsAffectedByVariable(variableName));
        }
    }
    return affected;
}
Also used : VariableImpact(org.apache.nifi.attribute.expression.language.VariableImpact) ProcessorNode(org.apache.nifi.controller.ProcessorNode) ControllerServiceNode(org.apache.nifi.controller.service.ControllerServiceNode) ConfiguredComponent(org.apache.nifi.controller.ConfiguredComponent) ComponentVariableRegistry(org.apache.nifi.registry.ComponentVariableRegistry) ControllerServiceReference(org.apache.nifi.controller.service.ControllerServiceReference) VersionedProcessGroup(org.apache.nifi.registry.flow.VersionedProcessGroup) VersionedRemoteProcessGroup(org.apache.nifi.registry.flow.VersionedRemoteProcessGroup) VariableDescriptor(org.apache.nifi.registry.VariableDescriptor) LinkedHashSet(java.util.LinkedHashSet) HashSet(java.util.HashSet)

Example 12 with ConfiguredComponent

use of org.apache.nifi.controller.ConfiguredComponent in project nifi by apache.

the class StandardProcessGroup method removeControllerService.

@Override
public void removeControllerService(final ControllerServiceNode service) {
    boolean removed = false;
    writeLock.lock();
    try {
        final ControllerServiceNode existing = controllerServices.get(requireNonNull(service).getIdentifier());
        if (existing == null) {
            throw new IllegalStateException("ControllerService " + service.getIdentifier() + " is not a member of this Process Group");
        }
        service.verifyCanDelete();
        try (final NarCloseable x = NarCloseable.withComponentNarLoader(service.getControllerServiceImplementation().getClass(), service.getIdentifier())) {
            final ConfigurationContext configurationContext = new StandardConfigurationContext(service, controllerServiceProvider, null, variableRegistry);
            ReflectionUtils.quietlyInvokeMethodsWithAnnotation(OnRemoved.class, service.getControllerServiceImplementation(), configurationContext);
        }
        for (final Map.Entry<PropertyDescriptor, String> entry : service.getProperties().entrySet()) {
            final PropertyDescriptor descriptor = entry.getKey();
            if (descriptor.getControllerServiceDefinition() != null) {
                final String value = entry.getValue() == null ? descriptor.getDefaultValue() : entry.getValue();
                if (value != null) {
                    final ControllerServiceNode referencedNode = getControllerService(value);
                    if (referencedNode != null) {
                        referencedNode.removeReference(service);
                    }
                }
            }
        }
        controllerServices.remove(service.getIdentifier());
        onComponentModified();
        // For any component that references this Controller Service, find the component's Process Group
        // and notify the Process Group that a component has been modified. This way, we know to re-calculate
        // whether or not the Process Group has local modifications.
        service.getReferences().getReferencingComponents().stream().map(ConfiguredComponent::getProcessGroupIdentifier).filter(id -> !id.equals(getIdentifier())).forEach(groupId -> {
            final ProcessGroup descendant = findProcessGroup(groupId);
            if (descendant != null) {
                descendant.onComponentModified();
            }
        });
        flowController.getStateManagerProvider().onComponentRemoved(service.getIdentifier());
        removed = true;
        LOG.info("{} removed from {}", service, this);
    } finally {
        if (removed) {
            try {
                ExtensionManager.removeInstanceClassLoader(service.getIdentifier());
            } catch (Throwable t) {
            }
        }
        writeLock.unlock();
    }
}
Also used : OnRemoved(org.apache.nifi.annotation.lifecycle.OnRemoved) ConfigurationContext(org.apache.nifi.controller.ConfigurationContext) Size(org.apache.nifi.connectable.Size) FlowComparison(org.apache.nifi.registry.flow.diff.FlowComparison) StringUtils(org.apache.commons.lang3.StringUtils) ReflectionUtils(org.apache.nifi.util.ReflectionUtils) PropertyDescriptor(org.apache.nifi.components.PropertyDescriptor) SecureRandom(java.security.SecureRandom) NiFiRegistryException(org.apache.nifi.registry.client.NiFiRegistryException) ComponentType(org.apache.nifi.registry.flow.ComponentType) VersionedProcessGroup(org.apache.nifi.registry.flow.VersionedProcessGroup) SnippetUtils(org.apache.nifi.util.SnippetUtils) Map(java.util.Map) HashCodeBuilder(org.apache.commons.lang3.builder.HashCodeBuilder) RootGroupPort(org.apache.nifi.remote.RootGroupPort) Connectable(org.apache.nifi.connectable.Connectable) Connection(org.apache.nifi.connectable.Connection) Bundle(org.apache.nifi.registry.flow.Bundle) FlowFilePrioritizer(org.apache.nifi.flowfile.FlowFilePrioritizer) FlowDifferenceFilters(org.apache.nifi.util.FlowDifferenceFilters) VersionedFlowStatus(org.apache.nifi.registry.flow.VersionedFlowStatus) Set(java.util.Set) VersionedFlowCoordinates(org.apache.nifi.registry.flow.VersionedFlowCoordinates) VersionedRemoteGroupPort(org.apache.nifi.registry.flow.VersionedRemoteGroupPort) FlowController(org.apache.nifi.controller.FlowController) StandardCharsets(java.nio.charset.StandardCharsets) StateManagerProvider(org.apache.nifi.components.state.StateManagerProvider) Position(org.apache.nifi.connectable.Position) ScheduledState(org.apache.nifi.controller.ScheduledState) ControllerService(org.apache.nifi.controller.ControllerService) ExtensionManager(org.apache.nifi.nar.ExtensionManager) StandardVersionControlInformation(org.apache.nifi.registry.flow.StandardVersionControlInformation) Resource(org.apache.nifi.authorization.Resource) FlowComparator(org.apache.nifi.registry.flow.diff.FlowComparator) StaticDifferenceDescriptor(org.apache.nifi.registry.flow.diff.StaticDifferenceDescriptor) StandardComparableDataFlow(org.apache.nifi.registry.flow.diff.StandardComparableDataFlow) SiteToSiteTransportProtocol(org.apache.nifi.remote.protocol.SiteToSiteTransportProtocol) ReentrantReadWriteLock(java.util.concurrent.locks.ReentrantReadWriteLock) ArrayList(java.util.ArrayList) Relationship(org.apache.nifi.processor.Relationship) ControllerServiceReference(org.apache.nifi.controller.service.ControllerServiceReference) ControllerServiceProvider(org.apache.nifi.controller.service.ControllerServiceProvider) VersionedLabel(org.apache.nifi.registry.flow.VersionedLabel) LinkedHashSet(java.util.LinkedHashSet) VersionedFlowState(org.apache.nifi.registry.flow.VersionedFlowState) EvolvingDifferenceDescriptor(org.apache.nifi.registry.flow.diff.EvolvingDifferenceDescriptor) ConfiguredComponent(org.apache.nifi.controller.ConfiguredComponent) Positionable(org.apache.nifi.connectable.Positionable) ExecutionNode(org.apache.nifi.scheduling.ExecutionNode) IOException(java.io.IOException) VersionedFlowSnapshot(org.apache.nifi.registry.flow.VersionedFlowSnapshot) NiFiRegistryFlowMapper(org.apache.nifi.registry.flow.mapping.NiFiRegistryFlowMapper) Lock(java.util.concurrent.locks.Lock) NiFiProperties(org.apache.nifi.util.NiFiProperties) VariableImpact(org.apache.nifi.attribute.expression.language.VariableImpact) FlowFileQueue(org.apache.nifi.controller.queue.FlowFileQueue) ProcessorInstantiationException(org.apache.nifi.controller.exception.ProcessorInstantiationException) BundleCoordinate(org.apache.nifi.bundle.BundleCoordinate) URL(java.net.URL) ConnectableType(org.apache.nifi.connectable.ConnectableType) ConnectableComponent(org.apache.nifi.registry.flow.ConnectableComponent) VariableDescriptor(org.apache.nifi.registry.VariableDescriptor) LoggerFactory(org.slf4j.LoggerFactory) Port(org.apache.nifi.connectable.Port) StandardFlowComparator(org.apache.nifi.registry.flow.diff.StandardFlowComparator) Query(org.apache.nifi.attribute.expression.language.Query) ResourceType(org.apache.nifi.authorization.resource.ResourceType) TemplateDTO(org.apache.nifi.web.api.dto.TemplateDTO) SchedulingStrategy(org.apache.nifi.scheduling.SchedulingStrategy) VersionedPort(org.apache.nifi.registry.flow.VersionedPort) VersionedRemoteProcessGroup(org.apache.nifi.registry.flow.VersionedRemoteProcessGroup) StandardProcessScheduler(org.apache.nifi.controller.scheduling.StandardProcessScheduler) VersionedComponent(org.apache.nifi.registry.flow.VersionedComponent) DifferenceType(org.apache.nifi.registry.flow.diff.DifferenceType) VersionedConnection(org.apache.nifi.registry.flow.VersionedConnection) Template(org.apache.nifi.controller.Template) Label(org.apache.nifi.controller.label.Label) FlowRegistryClient(org.apache.nifi.registry.flow.FlowRegistryClient) OnShutdown(org.apache.nifi.annotation.lifecycle.OnShutdown) MutableVariableRegistry(org.apache.nifi.registry.variable.MutableVariableRegistry) Authorizable(org.apache.nifi.authorization.resource.Authorizable) UUID(java.util.UUID) ComponentLifeCycleException(org.apache.nifi.controller.exception.ComponentLifeCycleException) Snippet(org.apache.nifi.controller.Snippet) Collectors(java.util.stream.Collectors) ResourceFactory(org.apache.nifi.authorization.resource.ResourceFactory) Objects(java.util.Objects) List(java.util.List) BatchSize(org.apache.nifi.registry.flow.BatchSize) VersionedFunnel(org.apache.nifi.registry.flow.VersionedFunnel) ToStringBuilder(org.apache.commons.lang3.builder.ToStringBuilder) VersionControlInformation(org.apache.nifi.registry.flow.VersionControlInformation) Optional(java.util.Optional) LocalPort(org.apache.nifi.connectable.LocalPort) StandardProcessContext(org.apache.nifi.processor.StandardProcessContext) ProcessorNode(org.apache.nifi.controller.ProcessorNode) Revision(org.apache.nifi.web.Revision) Funnel(org.apache.nifi.connectable.Funnel) ControllerServiceNode(org.apache.nifi.controller.service.ControllerServiceNode) ToStringStyle(org.apache.commons.lang3.builder.ToStringStyle) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) AtomicReference(java.util.concurrent.atomic.AtomicReference) Function(java.util.function.Function) FlowRegistry(org.apache.nifi.registry.flow.FlowRegistry) HashSet(java.util.HashSet) StringEncryptor(org.apache.nifi.encrypt.StringEncryptor) ComparableDataFlow(org.apache.nifi.registry.flow.diff.ComparableDataFlow) Objects.requireNonNull(java.util.Objects.requireNonNull) StandardConfigurationContext(org.apache.nifi.controller.service.StandardConfigurationContext) NarCloseable(org.apache.nifi.nar.NarCloseable) LogLevel(org.apache.nifi.logging.LogLevel) VersionedProcessor(org.apache.nifi.registry.flow.VersionedProcessor) Logger(org.slf4j.Logger) StateManager(org.apache.nifi.components.state.StateManager) RemoteGroupPort(org.apache.nifi.remote.RemoteGroupPort) StandardRemoteProcessGroupPortDescriptor(org.apache.nifi.remote.StandardRemoteProcessGroupPortDescriptor) VersionedFlow(org.apache.nifi.registry.flow.VersionedFlow) VersionedControllerService(org.apache.nifi.registry.flow.VersionedControllerService) TimeUnit(java.util.concurrent.TimeUnit) ComponentVariableRegistry(org.apache.nifi.registry.ComponentVariableRegistry) FlowDifference(org.apache.nifi.registry.flow.diff.FlowDifference) VersionedPropertyDescriptor(org.apache.nifi.registry.flow.VersionedPropertyDescriptor) Collections(java.util.Collections) LogRepositoryFactory(org.apache.nifi.logging.LogRepositoryFactory) NarCloseable(org.apache.nifi.nar.NarCloseable) ConfigurationContext(org.apache.nifi.controller.ConfigurationContext) StandardConfigurationContext(org.apache.nifi.controller.service.StandardConfigurationContext) PropertyDescriptor(org.apache.nifi.components.PropertyDescriptor) VersionedPropertyDescriptor(org.apache.nifi.registry.flow.VersionedPropertyDescriptor) ConfiguredComponent(org.apache.nifi.controller.ConfiguredComponent) StandardConfigurationContext(org.apache.nifi.controller.service.StandardConfigurationContext) ControllerServiceNode(org.apache.nifi.controller.service.ControllerServiceNode) VersionedProcessGroup(org.apache.nifi.registry.flow.VersionedProcessGroup) VersionedRemoteProcessGroup(org.apache.nifi.registry.flow.VersionedRemoteProcessGroup) Map(java.util.Map) HashMap(java.util.HashMap)

Example 13 with ConfiguredComponent

use of org.apache.nifi.controller.ConfiguredComponent in project nifi by apache.

the class StandardControllerServiceNode method verifyCanDisable.

@Override
public void verifyCanDisable(final Set<ControllerServiceNode> ignoreReferences) {
    if (!this.isActive()) {
        throw new IllegalStateException("Cannot disable " + getControllerServiceImplementation().getIdentifier() + " because it is not enabled");
    }
    final ControllerServiceReference references = getReferences();
    final Set<String> activeReferencesIdentifiers = new HashSet<>();
    for (final ConfiguredComponent activeReference : references.getActiveReferences()) {
        if (!ignoreReferences.contains(activeReference)) {
            activeReferencesIdentifiers.add(activeReference.getIdentifier());
        }
    }
    if (!activeReferencesIdentifiers.isEmpty()) {
        throw new IllegalStateException(getControllerServiceImplementation().getIdentifier() + " cannot be disabled because it is referenced by " + activeReferencesIdentifiers.size() + " components that are currently running: [" + StringUtils.join(activeReferencesIdentifiers, ", ") + "]");
    }
}
Also used : ConfiguredComponent(org.apache.nifi.controller.ConfiguredComponent) AbstractConfiguredComponent(org.apache.nifi.controller.AbstractConfiguredComponent) HashSet(java.util.HashSet)

Example 14 with ConfiguredComponent

use of org.apache.nifi.controller.ConfiguredComponent in project nifi by apache.

the class ControllerServiceAuditor method getUpdateActionsForReferencingComponents.

/**
 * Gets the update actions for all specified referencing components.
 *
 * @param user user
 * @param actions actions
 * @param visitedServices services
 * @param referencingComponents components
 */
private void getUpdateActionsForReferencingComponents(final NiFiUser user, final Collection<Action> actions, final Collection<String> visitedServices, final Set<ConfiguredComponent> referencingComponents) {
    // consider each component updates
    for (final ConfiguredComponent component : referencingComponents) {
        if (component instanceof ProcessorNode) {
            final ProcessorNode processor = ((ProcessorNode) component);
            // create the processor details
            FlowChangeExtensionDetails processorDetails = new FlowChangeExtensionDetails();
            processorDetails.setType(processor.getComponentType());
            // create a processor action
            FlowChangeAction processorAction = new FlowChangeAction();
            processorAction.setUserIdentity(user.getIdentity());
            processorAction.setTimestamp(new Date());
            processorAction.setSourceId(processor.getIdentifier());
            processorAction.setSourceName(processor.getName());
            processorAction.setSourceType(Component.Processor);
            processorAction.setComponentDetails(processorDetails);
            processorAction.setOperation(ScheduledState.RUNNING.equals(processor.getScheduledState()) ? Operation.Start : Operation.Stop);
            actions.add(processorAction);
        } else if (component instanceof ReportingTask) {
            final ReportingTaskNode reportingTask = ((ReportingTaskNode) component);
            // create the reporting task details
            FlowChangeExtensionDetails taskDetails = new FlowChangeExtensionDetails();
            taskDetails.setType(reportingTask.getComponentType());
            // create a reporting task action
            FlowChangeAction reportingTaskAction = new FlowChangeAction();
            reportingTaskAction.setUserIdentity(user.getIdentity());
            reportingTaskAction.setTimestamp(new Date());
            reportingTaskAction.setSourceId(reportingTask.getIdentifier());
            reportingTaskAction.setSourceName(reportingTask.getName());
            reportingTaskAction.setSourceType(Component.ReportingTask);
            reportingTaskAction.setComponentDetails(taskDetails);
            reportingTaskAction.setOperation(ScheduledState.RUNNING.equals(reportingTask.getScheduledState()) ? Operation.Start : Operation.Stop);
            actions.add(reportingTaskAction);
        } else if (component instanceof ControllerServiceNode) {
            final ControllerServiceNode controllerService = ((ControllerServiceNode) component);
            // create the controller service details
            FlowChangeExtensionDetails serviceDetails = new FlowChangeExtensionDetails();
            serviceDetails.setType(controllerService.getComponentType());
            // create a controller service action
            FlowChangeAction serviceAction = new FlowChangeAction();
            serviceAction.setUserIdentity(user.getIdentity());
            serviceAction.setTimestamp(new Date());
            serviceAction.setSourceId(controllerService.getIdentifier());
            serviceAction.setSourceName(controllerService.getName());
            serviceAction.setSourceType(Component.ControllerService);
            serviceAction.setComponentDetails(serviceDetails);
            serviceAction.setOperation(isDisabled(controllerService) ? Operation.Disable : Operation.Enable);
            actions.add(serviceAction);
            // need to consider components referencing this controller service (transitive)
            if (!visitedServices.contains(controllerService.getIdentifier())) {
                getUpdateActionsForReferencingComponents(user, actions, visitedServices, controllerService.getReferences().getReferencingComponents());
            }
        }
    }
}
Also used : ProcessorNode(org.apache.nifi.controller.ProcessorNode) ReportingTaskNode(org.apache.nifi.controller.ReportingTaskNode) ControllerServiceNode(org.apache.nifi.controller.service.ControllerServiceNode) ConfiguredComponent(org.apache.nifi.controller.ConfiguredComponent) FlowChangeExtensionDetails(org.apache.nifi.action.component.details.FlowChangeExtensionDetails) Date(java.util.Date) FlowChangeAction(org.apache.nifi.action.FlowChangeAction) ReportingTask(org.apache.nifi.reporting.ReportingTask)

Example 15 with ConfiguredComponent

use of org.apache.nifi.controller.ConfiguredComponent in project nifi by apache.

the class StandardAuthorizableLookup method getControllerServiceReferencingComponent.

@Override
public Authorizable getControllerServiceReferencingComponent(String controllerServiceId, String id) {
    final ControllerServiceNode controllerService = controllerServiceDAO.getControllerService(controllerServiceId);
    final ControllerServiceReference referencingComponents = controllerService.getReferences();
    final ConfiguredComponent reference = findControllerServiceReferencingComponent(referencingComponents, id);
    if (reference == null) {
        throw new ResourceNotFoundException("Unable to find referencing component with id " + id);
    }
    return reference;
}
Also used : ControllerServiceNode(org.apache.nifi.controller.service.ControllerServiceNode) ConfiguredComponent(org.apache.nifi.controller.ConfiguredComponent) ControllerServiceReference(org.apache.nifi.controller.service.ControllerServiceReference) ResourceNotFoundException(org.apache.nifi.web.ResourceNotFoundException)

Aggregations

ConfiguredComponent (org.apache.nifi.controller.ConfiguredComponent)19 HashSet (java.util.HashSet)13 ControllerServiceNode (org.apache.nifi.controller.service.ControllerServiceNode)11 LinkedHashSet (java.util.LinkedHashSet)8 HashMap (java.util.HashMap)7 ProcessorNode (org.apache.nifi.controller.ProcessorNode)7 LinkedHashMap (java.util.LinkedHashMap)5 ControllerServiceReference (org.apache.nifi.controller.service.ControllerServiceReference)5 VersionedProcessGroup (org.apache.nifi.registry.flow.VersionedProcessGroup)5 ArrayList (java.util.ArrayList)4 Map (java.util.Map)4 Set (java.util.Set)4 List (java.util.List)3 ReportingTaskNode (org.apache.nifi.controller.ReportingTaskNode)3 ProcessGroup (org.apache.nifi.groups.ProcessGroup)3 URL (java.net.URL)2 StandardCharsets (java.nio.charset.StandardCharsets)2 Collections (java.util.Collections)2 Optional (java.util.Optional)2 UUID (java.util.UUID)2