Search in sources :

Example 21 with AffectedComponentEntity

use of org.apache.nifi.web.api.entity.AffectedComponentEntity in project nifi by apache.

the class ProcessGroupResource method createVariableRegistryUpdateRequest.

private VariableRegistryUpdateRequest createVariableRegistryUpdateRequest(final String groupId, final Set<AffectedComponentEntity> affectedComponents, final NiFiUser user) {
    final VariableRegistryUpdateRequest updateRequest = new VariableRegistryUpdateRequest(UUID.randomUUID().toString(), groupId, affectedComponents, user);
    // before adding to the request map, purge any old requests. Must do this by creating a List of ID's
    // and then removing those ID's one-at-a-time in order to avoid ConcurrentModificationException.
    final Date oneMinuteAgo = new Date(System.currentTimeMillis() - VARIABLE_REGISTRY_UPDATE_REQUEST_EXPIRATION);
    final List<String> completedRequestIds = varRegistryUpdateRequests.entrySet().stream().filter(entry -> entry.getValue().isComplete()).filter(entry -> entry.getValue().getLastUpdated().before(oneMinuteAgo)).map(Map.Entry::getKey).collect(Collectors.toList());
    completedRequestIds.stream().forEach(id -> varRegistryUpdateRequests.remove(id));
    final int requestCount = varRegistryUpdateRequests.size();
    if (requestCount > MAX_VARIABLE_REGISTRY_UPDATE_REQUESTS) {
        throw new IllegalStateException("There are already " + requestCount + " update requests for variable registries. " + "Cannot issue any more requests until the older ones are deleted or expire");
    }
    this.varRegistryUpdateRequests.put(updateRequest.getRequestId(), updateRequest);
    return updateRequest;
}
Also used : FunnelsEntity(org.apache.nifi.web.api.entity.FunnelsEntity) Produces(javax.ws.rs.Produces) InstantiateTemplateRequestEntity(org.apache.nifi.web.api.entity.InstantiateTemplateRequestEntity) ApiParam(io.swagger.annotations.ApiParam) SiteToSiteRestApiClient(org.apache.nifi.remote.util.SiteToSiteRestApiClient) ConnectionDTO(org.apache.nifi.web.api.dto.ConnectionDTO) ComponentAuthorizable(org.apache.nifi.authorization.ComponentAuthorizable) StringUtils(org.apache.commons.lang3.StringUtils) ClientIdParameter(org.apache.nifi.web.api.request.ClientIdParameter) ProcessorEntity(org.apache.nifi.web.api.entity.ProcessorEntity) AuthorizeAccess(org.apache.nifi.authorization.AuthorizeAccess) VariableRegistryUpdateStep(org.apache.nifi.registry.variable.VariableRegistryUpdateStep) PositionDTO(org.apache.nifi.web.api.dto.PositionDTO) MediaType(javax.ws.rs.core.MediaType) ProcessGroupDTO(org.apache.nifi.web.api.dto.ProcessGroupDTO) NiFiRegistryException(org.apache.nifi.registry.client.NiFiRegistryException) Map(java.util.Map) ResourceNotFoundException(org.apache.nifi.web.ResourceNotFoundException) UriBuilder(javax.ws.rs.core.UriBuilder) SecurityContextHolder(org.springframework.security.core.context.SecurityContextHolder) ConnectionsEntity(org.apache.nifi.web.api.entity.ConnectionsEntity) FunnelEntity(org.apache.nifi.web.api.entity.FunnelEntity) VariableRegistryUpdateRequest(org.apache.nifi.registry.variable.VariableRegistryUpdateRequest) ControllerServicesEntity(org.apache.nifi.web.api.entity.ControllerServicesEntity) Set(java.util.Set) InputPortsEntity(org.apache.nifi.web.api.entity.InputPortsEntity) Executors(java.util.concurrent.Executors) ArrayBlockingQueue(java.util.concurrent.ArrayBlockingQueue) FormDataParam(org.glassfish.jersey.media.multipart.FormDataParam) ProcessGroupsEntity(org.apache.nifi.web.api.entity.ProcessGroupsEntity) FlowComparisonEntity(org.apache.nifi.web.api.entity.FlowComparisonEntity) ScheduledState(org.apache.nifi.controller.ScheduledState) LabelsEntity(org.apache.nifi.web.api.entity.LabelsEntity) UriInfo(javax.ws.rs.core.UriInfo) ApiImplicitParams(io.swagger.annotations.ApiImplicitParams) DtoFactory(org.apache.nifi.web.api.dto.DtoFactory) Entity(org.apache.nifi.web.api.entity.Entity) GET(javax.ws.rs.GET) ControllerServiceEntity(org.apache.nifi.web.api.entity.ControllerServiceEntity) ConfigurableComponent(org.apache.nifi.components.ConfigurableComponent) TemplateEntity(org.apache.nifi.web.api.entity.TemplateEntity) RevisionDTO(org.apache.nifi.web.api.dto.RevisionDTO) HttpMethod(javax.ws.rs.HttpMethod) HttpServletRequest(javax.servlet.http.HttpServletRequest) NiFiUser(org.apache.nifi.authorization.user.NiFiUser) NiFiUserDetails(org.apache.nifi.authorization.user.NiFiUserDetails) Api(io.swagger.annotations.Api) VariableRegistryDTO(org.apache.nifi.web.api.dto.VariableRegistryDTO) FlowDTO(org.apache.nifi.web.api.dto.flow.FlowDTO) VersionedFlowState(org.apache.nifi.registry.flow.VersionedFlowState) NiFiServiceFacade(org.apache.nifi.web.NiFiServiceFacade) AuthorizableLookup(org.apache.nifi.authorization.AuthorizableLookup) RequestAction(org.apache.nifi.authorization.RequestAction) FlowEncodingVersion(org.apache.nifi.controller.serialization.FlowEncodingVersion) JAXBElement(javax.xml.bind.JAXBElement) RemoteProcessGroupsEntity(org.apache.nifi.web.api.entity.RemoteProcessGroupsEntity) IOException(java.io.IOException) VersionedFlowSnapshot(org.apache.nifi.registry.flow.VersionedFlowSnapshot) Authorizer(org.apache.nifi.authorization.Authorizer) ApiResponse(io.swagger.annotations.ApiResponse) FlowEntity(org.apache.nifi.web.api.entity.FlowEntity) AffectedComponentEntity(org.apache.nifi.web.api.entity.AffectedComponentEntity) OutputPortsEntity(org.apache.nifi.web.api.entity.OutputPortsEntity) ScheduleComponentsEntity(org.apache.nifi.web.api.entity.ScheduleComponentsEntity) XmlUtils(org.apache.nifi.security.xml.XmlUtils) BundleCoordinate(org.apache.nifi.bundle.BundleCoordinate) ProcessorConfigDTO(org.apache.nifi.web.api.dto.ProcessorConfigDTO) Date(java.util.Date) ConnectableType(org.apache.nifi.connectable.ConnectableType) ProcessorStatusDTO(org.apache.nifi.web.api.dto.status.ProcessorStatusDTO) URISyntaxException(java.net.URISyntaxException) LoggerFactory(org.slf4j.LoggerFactory) Path(javax.ws.rs.Path) BundleDTO(org.apache.nifi.web.api.dto.BundleDTO) ApiOperation(io.swagger.annotations.ApiOperation) AuthorizeControllerServiceReference(org.apache.nifi.authorization.AuthorizeControllerServiceReference) QueryParam(javax.ws.rs.QueryParam) Consumes(javax.ws.rs.Consumes) TemplateDTO(org.apache.nifi.web.api.dto.TemplateDTO) ActivateControllerServicesEntity(org.apache.nifi.web.api.entity.ActivateControllerServicesEntity) XMLStreamReader(javax.xml.stream.XMLStreamReader) DefaultValue(javax.ws.rs.DefaultValue) URI(java.net.URI) ThreadFactory(java.util.concurrent.ThreadFactory) NodeResponse(org.apache.nifi.cluster.manager.NodeResponse) DELETE(javax.ws.rs.DELETE) Context(javax.ws.rs.core.Context) Authorizable(org.apache.nifi.authorization.resource.Authorizable) ControllerServiceDTO(org.apache.nifi.web.api.dto.ControllerServiceDTO) ApiImplicitParam(io.swagger.annotations.ApiImplicitParam) Collection(java.util.Collection) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) SnippetAuthorizable(org.apache.nifi.authorization.SnippetAuthorizable) UUID(java.util.UUID) BundleUtils(org.apache.nifi.util.BundleUtils) PortEntity(org.apache.nifi.web.api.entity.PortEntity) LongParameter(org.apache.nifi.web.api.request.LongParameter) JAXBException(javax.xml.bind.JAXBException) Collectors(java.util.stream.Collectors) List(java.util.List) Response(javax.ws.rs.core.Response) ProcessGroupEntity(org.apache.nifi.web.api.entity.ProcessGroupEntity) ProcessorDTO(org.apache.nifi.web.api.dto.ProcessorDTO) ControllerServiceState(org.apache.nifi.controller.service.ControllerServiceState) CopySnippetRequestEntity(org.apache.nifi.web.api.entity.CopySnippetRequestEntity) Authentication(org.springframework.security.core.Authentication) Pause(org.apache.nifi.web.util.Pause) FlowSnippetDTO(org.apache.nifi.web.api.dto.FlowSnippetDTO) RemoteProcessGroupDTO(org.apache.nifi.web.api.dto.RemoteProcessGroupDTO) PathParam(javax.ws.rs.PathParam) Bucket(org.apache.nifi.registry.bucket.Bucket) Revision(org.apache.nifi.web.Revision) ThreadPoolExecutor(java.util.concurrent.ThreadPoolExecutor) HashMap(java.util.HashMap) ApiResponses(io.swagger.annotations.ApiResponses) Function(java.util.function.Function) AffectedComponentDTO(org.apache.nifi.web.api.dto.AffectedComponentDTO) ConcurrentMap(java.util.concurrent.ConcurrentMap) FlowRegistryUtils(org.apache.nifi.registry.flow.FlowRegistryUtils) CreateTemplateRequestEntity(org.apache.nifi.web.api.entity.CreateTemplateRequestEntity) VersionControlInformationDTO(org.apache.nifi.web.api.dto.VersionControlInformationDTO) VariableRegistryUpdateRequestEntity(org.apache.nifi.web.api.entity.VariableRegistryUpdateRequestEntity) NiFiAuthenticationToken(org.apache.nifi.web.security.token.NiFiAuthenticationToken) Status(javax.ws.rs.core.Response.Status) JAXBContext(javax.xml.bind.JAXBContext) ExecutorService(java.util.concurrent.ExecutorService) Unmarshaller(javax.xml.bind.Unmarshaller) TemplateContentsAuthorizable(org.apache.nifi.authorization.TemplateContentsAuthorizable) Logger(org.slf4j.Logger) POST(javax.ws.rs.POST) ProcessorsEntity(org.apache.nifi.web.api.entity.ProcessorsEntity) VariableRegistryEntity(org.apache.nifi.web.api.entity.VariableRegistryEntity) VersionedFlow(org.apache.nifi.registry.flow.VersionedFlow) MultivaluedHashMap(javax.ws.rs.core.MultivaluedHashMap) MultivaluedMap(javax.ws.rs.core.MultivaluedMap) TimeUnit(java.util.concurrent.TimeUnit) Consumer(java.util.function.Consumer) LabelEntity(org.apache.nifi.web.api.entity.LabelEntity) ConnectionEntity(org.apache.nifi.web.api.entity.ConnectionEntity) ProcessGroupAuthorizable(org.apache.nifi.authorization.ProcessGroupAuthorizable) RemoteProcessGroupEntity(org.apache.nifi.web.api.entity.RemoteProcessGroupEntity) NiFiUserUtils(org.apache.nifi.authorization.user.NiFiUserUtils) PUT(javax.ws.rs.PUT) Authorization(io.swagger.annotations.Authorization) Collections(java.util.Collections) InputStream(java.io.InputStream) VariableRegistryUpdateRequest(org.apache.nifi.registry.variable.VariableRegistryUpdateRequest) Map(java.util.Map) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) HashMap(java.util.HashMap) ConcurrentMap(java.util.concurrent.ConcurrentMap) MultivaluedHashMap(javax.ws.rs.core.MultivaluedHashMap) MultivaluedMap(javax.ws.rs.core.MultivaluedMap) Date(java.util.Date)

Example 22 with AffectedComponentEntity

use of org.apache.nifi.web.api.entity.AffectedComponentEntity in project nifi by apache.

the class ProcessGroupResource method submitUpdateVariableRegistryRequest.

/**
 * Updates the variable registry for the specified process group.
 *
 * @param httpServletRequest request
 * @param groupId The id of the process group.
 * @param requestVariableRegistryEntity the Variable Registry Entity
 * @return A Variable Registry Entry.
 */
@POST
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
@Path("{id}/variable-registry/update-requests")
@ApiOperation(value = "Submits a request to update a process group's variable registry", response = VariableRegistryUpdateRequestEntity.class, notes = NON_GUARANTEED_ENDPOINT, authorizations = { @Authorization(value = "Write - /process-groups/{uuid}") })
@ApiResponses(value = { @ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."), @ApiResponse(code = 401, message = "Client could not be authenticated."), @ApiResponse(code = 403, message = "Client is not authorized to make this request."), @ApiResponse(code = 404, message = "The specified resource could not be found."), @ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.") })
public Response submitUpdateVariableRegistryRequest(@Context final HttpServletRequest httpServletRequest, @ApiParam(value = "The process group id.", required = true) @PathParam("id") final String groupId, @ApiParam(value = "The variable registry configuration details.", required = true) final VariableRegistryEntity requestVariableRegistryEntity) {
    if (requestVariableRegistryEntity == null || requestVariableRegistryEntity.getVariableRegistry() == null) {
        throw new IllegalArgumentException("Variable Registry details must be specified.");
    }
    if (requestVariableRegistryEntity.getProcessGroupRevision() == null) {
        throw new IllegalArgumentException("Process Group Revision must be specified.");
    }
    // In order to update variables in a variable registry, we have to perform the following steps:
    // 1. Determine Affected Components (this includes any Processors and Controller Services and any components that reference an affected Controller Service).
    // 1a. Determine ID's of components
    // 1b. Determine Revision's of associated components
    // 2. Stop All Active Affected Processors
    // 3. Disable All Active Affected Controller Services
    // 4. Update the Variables
    // 5. Re-Enable all previously Active Affected Controller Services (services only, not dependent components)
    // 6. Re-Enable all previously Active Processors that Depended on the Controller Services
    // Determine the affected components (and their associated revisions)
    final VariableRegistryEntity computedEntity = serviceFacade.populateAffectedComponents(requestVariableRegistryEntity.getVariableRegistry());
    final VariableRegistryDTO computedRegistryDto = computedEntity.getVariableRegistry();
    if (computedRegistryDto == null) {
        throw new ResourceNotFoundException(String.format("Unable to locate group with id '%s'.", groupId));
    }
    final Set<AffectedComponentEntity> allAffectedComponents = serviceFacade.getComponentsAffectedByVariableRegistryUpdate(requestVariableRegistryEntity.getVariableRegistry());
    final Set<AffectedComponentDTO> activeAffectedComponents = serviceFacade.getActiveComponentsAffectedByVariableRegistryUpdate(requestVariableRegistryEntity.getVariableRegistry());
    final Map<String, List<AffectedComponentDTO>> activeAffectedComponentsByType = activeAffectedComponents.stream().collect(Collectors.groupingBy(comp -> comp.getReferenceType()));
    final List<AffectedComponentDTO> activeAffectedProcessors = activeAffectedComponentsByType.get(AffectedComponentDTO.COMPONENT_TYPE_PROCESSOR);
    final List<AffectedComponentDTO> activeAffectedServices = activeAffectedComponentsByType.get(AffectedComponentDTO.COMPONENT_TYPE_CONTROLLER_SERVICE);
    final NiFiUser user = NiFiUserUtils.getNiFiUser();
    // define access authorize for execution below
    final AuthorizeAccess authorizeAccess = lookup -> {
        final Authorizable groupAuthorizable = lookup.getProcessGroup(groupId).getAuthorizable();
        groupAuthorizable.authorize(authorizer, RequestAction.WRITE, user);
        // (because this action requires stopping the component).
        if (activeAffectedProcessors != null) {
            for (final AffectedComponentDTO activeAffectedComponent : activeAffectedProcessors) {
                final Authorizable authorizable = lookup.getProcessor(activeAffectedComponent.getId()).getAuthorizable();
                authorizable.authorize(authorizer, RequestAction.READ, user);
                authorizable.authorize(authorizer, RequestAction.WRITE, user);
            }
        }
        if (activeAffectedServices != null) {
            for (final AffectedComponentDTO activeAffectedComponent : activeAffectedServices) {
                final Authorizable authorizable = lookup.getControllerService(activeAffectedComponent.getId()).getAuthorizable();
                authorizable.authorize(authorizer, RequestAction.READ, user);
                authorizable.authorize(authorizer, RequestAction.WRITE, user);
            }
        }
    };
    if (isReplicateRequest()) {
        // authorize access
        serviceFacade.authorizeAccess(authorizeAccess);
        // update the variable registry
        final VariableRegistryUpdateRequest updateRequest = createVariableRegistryUpdateRequest(groupId, allAffectedComponents, user);
        updateRequest.getIdentifyRelevantComponentsStep().setComplete(true);
        final URI originalUri = getAbsolutePath();
        // Submit the task to be run in the background
        final Runnable taskWrapper = () -> {
            try {
                // set the user authentication token
                final Authentication authentication = new NiFiAuthenticationToken(new NiFiUserDetails(user));
                SecurityContextHolder.getContext().setAuthentication(authentication);
                updateVariableRegistryReplicated(groupId, originalUri, activeAffectedProcessors, activeAffectedServices, updateRequest, requestVariableRegistryEntity);
                // ensure the request is marked complete
                updateRequest.setComplete(true);
            } catch (final Exception e) {
                logger.error("Failed to update variable registry", e);
                updateRequest.setComplete(true);
                updateRequest.setFailureReason("An unexpected error has occurred: " + e);
            } finally {
                // clear the authentication token
                SecurityContextHolder.getContext().setAuthentication(null);
            }
        };
        variableRegistryThreadPool.submit(taskWrapper);
        final VariableRegistryUpdateRequestEntity responseEntity = new VariableRegistryUpdateRequestEntity();
        responseEntity.setRequest(dtoFactory.createVariableRegistryUpdateRequestDto(updateRequest));
        responseEntity.setProcessGroupRevision(updateRequest.getProcessGroupRevision());
        responseEntity.getRequest().setUri(generateResourceUri("process-groups", groupId, "variable-registry", "update-requests", updateRequest.getRequestId()));
        final URI location = URI.create(responseEntity.getRequest().getUri());
        return Response.status(Status.ACCEPTED).location(location).entity(responseEntity).build();
    }
    final UpdateVariableRegistryRequestWrapper requestWrapper = new UpdateVariableRegistryRequestWrapper(allAffectedComponents, activeAffectedProcessors, activeAffectedServices, requestVariableRegistryEntity);
    final Revision requestRevision = getRevision(requestVariableRegistryEntity.getProcessGroupRevision(), groupId);
    return withWriteLock(serviceFacade, requestWrapper, requestRevision, authorizeAccess, null, (revision, wrapper) -> updateVariableRegistryLocal(groupId, wrapper.getAllAffectedComponents(), wrapper.getActiveAffectedProcessors(), wrapper.getActiveAffectedServices(), user, revision, wrapper.getVariableRegistryEntity()));
}
Also used : FunnelsEntity(org.apache.nifi.web.api.entity.FunnelsEntity) Produces(javax.ws.rs.Produces) InstantiateTemplateRequestEntity(org.apache.nifi.web.api.entity.InstantiateTemplateRequestEntity) ApiParam(io.swagger.annotations.ApiParam) SiteToSiteRestApiClient(org.apache.nifi.remote.util.SiteToSiteRestApiClient) ConnectionDTO(org.apache.nifi.web.api.dto.ConnectionDTO) ComponentAuthorizable(org.apache.nifi.authorization.ComponentAuthorizable) StringUtils(org.apache.commons.lang3.StringUtils) ClientIdParameter(org.apache.nifi.web.api.request.ClientIdParameter) ProcessorEntity(org.apache.nifi.web.api.entity.ProcessorEntity) AuthorizeAccess(org.apache.nifi.authorization.AuthorizeAccess) VariableRegistryUpdateStep(org.apache.nifi.registry.variable.VariableRegistryUpdateStep) PositionDTO(org.apache.nifi.web.api.dto.PositionDTO) MediaType(javax.ws.rs.core.MediaType) ProcessGroupDTO(org.apache.nifi.web.api.dto.ProcessGroupDTO) NiFiRegistryException(org.apache.nifi.registry.client.NiFiRegistryException) Map(java.util.Map) ResourceNotFoundException(org.apache.nifi.web.ResourceNotFoundException) UriBuilder(javax.ws.rs.core.UriBuilder) SecurityContextHolder(org.springframework.security.core.context.SecurityContextHolder) ConnectionsEntity(org.apache.nifi.web.api.entity.ConnectionsEntity) FunnelEntity(org.apache.nifi.web.api.entity.FunnelEntity) VariableRegistryUpdateRequest(org.apache.nifi.registry.variable.VariableRegistryUpdateRequest) ControllerServicesEntity(org.apache.nifi.web.api.entity.ControllerServicesEntity) Set(java.util.Set) InputPortsEntity(org.apache.nifi.web.api.entity.InputPortsEntity) Executors(java.util.concurrent.Executors) ArrayBlockingQueue(java.util.concurrent.ArrayBlockingQueue) FormDataParam(org.glassfish.jersey.media.multipart.FormDataParam) ProcessGroupsEntity(org.apache.nifi.web.api.entity.ProcessGroupsEntity) FlowComparisonEntity(org.apache.nifi.web.api.entity.FlowComparisonEntity) ScheduledState(org.apache.nifi.controller.ScheduledState) LabelsEntity(org.apache.nifi.web.api.entity.LabelsEntity) UriInfo(javax.ws.rs.core.UriInfo) ApiImplicitParams(io.swagger.annotations.ApiImplicitParams) DtoFactory(org.apache.nifi.web.api.dto.DtoFactory) Entity(org.apache.nifi.web.api.entity.Entity) GET(javax.ws.rs.GET) ControllerServiceEntity(org.apache.nifi.web.api.entity.ControllerServiceEntity) ConfigurableComponent(org.apache.nifi.components.ConfigurableComponent) TemplateEntity(org.apache.nifi.web.api.entity.TemplateEntity) RevisionDTO(org.apache.nifi.web.api.dto.RevisionDTO) HttpMethod(javax.ws.rs.HttpMethod) HttpServletRequest(javax.servlet.http.HttpServletRequest) NiFiUser(org.apache.nifi.authorization.user.NiFiUser) NiFiUserDetails(org.apache.nifi.authorization.user.NiFiUserDetails) Api(io.swagger.annotations.Api) VariableRegistryDTO(org.apache.nifi.web.api.dto.VariableRegistryDTO) FlowDTO(org.apache.nifi.web.api.dto.flow.FlowDTO) VersionedFlowState(org.apache.nifi.registry.flow.VersionedFlowState) NiFiServiceFacade(org.apache.nifi.web.NiFiServiceFacade) AuthorizableLookup(org.apache.nifi.authorization.AuthorizableLookup) RequestAction(org.apache.nifi.authorization.RequestAction) FlowEncodingVersion(org.apache.nifi.controller.serialization.FlowEncodingVersion) JAXBElement(javax.xml.bind.JAXBElement) RemoteProcessGroupsEntity(org.apache.nifi.web.api.entity.RemoteProcessGroupsEntity) IOException(java.io.IOException) VersionedFlowSnapshot(org.apache.nifi.registry.flow.VersionedFlowSnapshot) Authorizer(org.apache.nifi.authorization.Authorizer) ApiResponse(io.swagger.annotations.ApiResponse) FlowEntity(org.apache.nifi.web.api.entity.FlowEntity) AffectedComponentEntity(org.apache.nifi.web.api.entity.AffectedComponentEntity) OutputPortsEntity(org.apache.nifi.web.api.entity.OutputPortsEntity) ScheduleComponentsEntity(org.apache.nifi.web.api.entity.ScheduleComponentsEntity) XmlUtils(org.apache.nifi.security.xml.XmlUtils) BundleCoordinate(org.apache.nifi.bundle.BundleCoordinate) ProcessorConfigDTO(org.apache.nifi.web.api.dto.ProcessorConfigDTO) Date(java.util.Date) ConnectableType(org.apache.nifi.connectable.ConnectableType) ProcessorStatusDTO(org.apache.nifi.web.api.dto.status.ProcessorStatusDTO) URISyntaxException(java.net.URISyntaxException) LoggerFactory(org.slf4j.LoggerFactory) Path(javax.ws.rs.Path) BundleDTO(org.apache.nifi.web.api.dto.BundleDTO) ApiOperation(io.swagger.annotations.ApiOperation) AuthorizeControllerServiceReference(org.apache.nifi.authorization.AuthorizeControllerServiceReference) QueryParam(javax.ws.rs.QueryParam) Consumes(javax.ws.rs.Consumes) TemplateDTO(org.apache.nifi.web.api.dto.TemplateDTO) ActivateControllerServicesEntity(org.apache.nifi.web.api.entity.ActivateControllerServicesEntity) XMLStreamReader(javax.xml.stream.XMLStreamReader) DefaultValue(javax.ws.rs.DefaultValue) URI(java.net.URI) ThreadFactory(java.util.concurrent.ThreadFactory) NodeResponse(org.apache.nifi.cluster.manager.NodeResponse) DELETE(javax.ws.rs.DELETE) Context(javax.ws.rs.core.Context) Authorizable(org.apache.nifi.authorization.resource.Authorizable) ControllerServiceDTO(org.apache.nifi.web.api.dto.ControllerServiceDTO) ApiImplicitParam(io.swagger.annotations.ApiImplicitParam) Collection(java.util.Collection) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) SnippetAuthorizable(org.apache.nifi.authorization.SnippetAuthorizable) UUID(java.util.UUID) BundleUtils(org.apache.nifi.util.BundleUtils) PortEntity(org.apache.nifi.web.api.entity.PortEntity) LongParameter(org.apache.nifi.web.api.request.LongParameter) JAXBException(javax.xml.bind.JAXBException) Collectors(java.util.stream.Collectors) List(java.util.List) Response(javax.ws.rs.core.Response) ProcessGroupEntity(org.apache.nifi.web.api.entity.ProcessGroupEntity) ProcessorDTO(org.apache.nifi.web.api.dto.ProcessorDTO) ControllerServiceState(org.apache.nifi.controller.service.ControllerServiceState) CopySnippetRequestEntity(org.apache.nifi.web.api.entity.CopySnippetRequestEntity) Authentication(org.springframework.security.core.Authentication) Pause(org.apache.nifi.web.util.Pause) FlowSnippetDTO(org.apache.nifi.web.api.dto.FlowSnippetDTO) RemoteProcessGroupDTO(org.apache.nifi.web.api.dto.RemoteProcessGroupDTO) PathParam(javax.ws.rs.PathParam) Bucket(org.apache.nifi.registry.bucket.Bucket) Revision(org.apache.nifi.web.Revision) ThreadPoolExecutor(java.util.concurrent.ThreadPoolExecutor) HashMap(java.util.HashMap) ApiResponses(io.swagger.annotations.ApiResponses) Function(java.util.function.Function) AffectedComponentDTO(org.apache.nifi.web.api.dto.AffectedComponentDTO) ConcurrentMap(java.util.concurrent.ConcurrentMap) FlowRegistryUtils(org.apache.nifi.registry.flow.FlowRegistryUtils) CreateTemplateRequestEntity(org.apache.nifi.web.api.entity.CreateTemplateRequestEntity) VersionControlInformationDTO(org.apache.nifi.web.api.dto.VersionControlInformationDTO) VariableRegistryUpdateRequestEntity(org.apache.nifi.web.api.entity.VariableRegistryUpdateRequestEntity) NiFiAuthenticationToken(org.apache.nifi.web.security.token.NiFiAuthenticationToken) Status(javax.ws.rs.core.Response.Status) JAXBContext(javax.xml.bind.JAXBContext) ExecutorService(java.util.concurrent.ExecutorService) Unmarshaller(javax.xml.bind.Unmarshaller) TemplateContentsAuthorizable(org.apache.nifi.authorization.TemplateContentsAuthorizable) Logger(org.slf4j.Logger) POST(javax.ws.rs.POST) ProcessorsEntity(org.apache.nifi.web.api.entity.ProcessorsEntity) VariableRegistryEntity(org.apache.nifi.web.api.entity.VariableRegistryEntity) VersionedFlow(org.apache.nifi.registry.flow.VersionedFlow) MultivaluedHashMap(javax.ws.rs.core.MultivaluedHashMap) MultivaluedMap(javax.ws.rs.core.MultivaluedMap) TimeUnit(java.util.concurrent.TimeUnit) Consumer(java.util.function.Consumer) LabelEntity(org.apache.nifi.web.api.entity.LabelEntity) ConnectionEntity(org.apache.nifi.web.api.entity.ConnectionEntity) ProcessGroupAuthorizable(org.apache.nifi.authorization.ProcessGroupAuthorizable) RemoteProcessGroupEntity(org.apache.nifi.web.api.entity.RemoteProcessGroupEntity) NiFiUserUtils(org.apache.nifi.authorization.user.NiFiUserUtils) PUT(javax.ws.rs.PUT) Authorization(io.swagger.annotations.Authorization) Collections(java.util.Collections) InputStream(java.io.InputStream) VariableRegistryUpdateRequest(org.apache.nifi.registry.variable.VariableRegistryUpdateRequest) NiFiUser(org.apache.nifi.authorization.user.NiFiUser) VariableRegistryDTO(org.apache.nifi.web.api.dto.VariableRegistryDTO) URI(java.net.URI) ComponentAuthorizable(org.apache.nifi.authorization.ComponentAuthorizable) Authorizable(org.apache.nifi.authorization.resource.Authorizable) SnippetAuthorizable(org.apache.nifi.authorization.SnippetAuthorizable) TemplateContentsAuthorizable(org.apache.nifi.authorization.TemplateContentsAuthorizable) ProcessGroupAuthorizable(org.apache.nifi.authorization.ProcessGroupAuthorizable) List(java.util.List) ResourceNotFoundException(org.apache.nifi.web.ResourceNotFoundException) AffectedComponentEntity(org.apache.nifi.web.api.entity.AffectedComponentEntity) NiFiUserDetails(org.apache.nifi.authorization.user.NiFiUserDetails) VariableRegistryUpdateRequestEntity(org.apache.nifi.web.api.entity.VariableRegistryUpdateRequestEntity) VariableRegistryEntity(org.apache.nifi.web.api.entity.VariableRegistryEntity) NiFiRegistryException(org.apache.nifi.registry.client.NiFiRegistryException) ResourceNotFoundException(org.apache.nifi.web.ResourceNotFoundException) IOException(java.io.IOException) URISyntaxException(java.net.URISyntaxException) JAXBException(javax.xml.bind.JAXBException) NiFiAuthenticationToken(org.apache.nifi.web.security.token.NiFiAuthenticationToken) AuthorizeAccess(org.apache.nifi.authorization.AuthorizeAccess) Revision(org.apache.nifi.web.Revision) Authentication(org.springframework.security.core.Authentication) AffectedComponentDTO(org.apache.nifi.web.api.dto.AffectedComponentDTO) Path(javax.ws.rs.Path) POST(javax.ws.rs.POST) Consumes(javax.ws.rs.Consumes) Produces(javax.ws.rs.Produces) ApiOperation(io.swagger.annotations.ApiOperation) ApiResponses(io.swagger.annotations.ApiResponses)

Example 23 with AffectedComponentEntity

use of org.apache.nifi.web.api.entity.AffectedComponentEntity in project nifi by apache.

the class EntityFactory method createAffectedComponentEntity.

public AffectedComponentEntity createAffectedComponentEntity(final AffectedComponentDTO dto, final RevisionDTO revision, final PermissionsDTO permissions) {
    final AffectedComponentEntity entity = new AffectedComponentEntity();
    entity.setRevision(revision);
    if (dto != null) {
        entity.setPermissions(permissions);
        entity.setId(dto.getId());
        if (permissions != null && permissions.getCanRead()) {
            entity.setComponent(dto);
        }
    }
    return entity;
}
Also used : AffectedComponentEntity(org.apache.nifi.web.api.entity.AffectedComponentEntity)

Example 24 with AffectedComponentEntity

use of org.apache.nifi.web.api.entity.AffectedComponentEntity in project nifi by apache.

the class ClusterReplicationComponentLifecycle method updateAffectedControllerServices.

/**
 * Updates the affected controller services in the specified updateRequest with the serviceEntities.
 *
 * @param serviceEntities service entities
 * @param affectedServices affected services
 */
private void updateAffectedControllerServices(final Set<ControllerServiceEntity> serviceEntities, final Map<String, AffectedComponentEntity> affectedServices) {
    // update the affected components
    serviceEntities.stream().filter(entity -> affectedServices.containsKey(entity.getId())).forEach(entity -> {
        final AffectedComponentEntity affectedComponentEntity = affectedServices.get(entity.getId());
        affectedComponentEntity.setRevision(entity.getRevision());
        // only consider update this component if the user had permissions to it
        if (Boolean.TRUE.equals(affectedComponentEntity.getPermissions().getCanRead())) {
            final AffectedComponentDTO affectedComponent = affectedComponentEntity.getComponent();
            affectedComponent.setState(entity.getComponent().getState());
            if (Boolean.TRUE.equals(entity.getPermissions().getCanRead())) {
                affectedComponent.setValidationErrors(entity.getComponent().getValidationErrors());
            }
        }
    });
}
Also used : NodeIdentifier(org.apache.nifi.cluster.protocol.NodeIdentifier) Revision(org.apache.nifi.web.Revision) ProcessorStatusDTO(org.apache.nifi.web.api.dto.status.ProcessorStatusDTO) URISyntaxException(java.net.URISyntaxException) LoggerFactory(org.slf4j.LoggerFactory) ControllerServiceEntity(org.apache.nifi.web.api.entity.ControllerServiceEntity) HashMap(java.util.HashMap) RevisionDTO(org.apache.nifi.web.api.dto.RevisionDTO) Function(java.util.function.Function) AffectedComponentDTO(org.apache.nifi.web.api.dto.AffectedComponentDTO) HttpMethod(javax.ws.rs.HttpMethod) ProcessorEntity(org.apache.nifi.web.api.entity.ProcessorEntity) MediaType(javax.ws.rs.core.MediaType) NiFiUser(org.apache.nifi.authorization.user.NiFiUser) ActivateControllerServicesEntity(org.apache.nifi.web.api.entity.ActivateControllerServicesEntity) Map(java.util.Map) ClusterCoordinator(org.apache.nifi.cluster.coordination.ClusterCoordinator) RequestReplicator(org.apache.nifi.cluster.coordination.http.replication.RequestReplicator) URI(java.net.URI) Status(javax.ws.rs.core.Response.Status) NodeResponse(org.apache.nifi.cluster.manager.NodeResponse) NiFiServiceFacade(org.apache.nifi.web.NiFiServiceFacade) NoClusterCoordinatorException(org.apache.nifi.cluster.exception.NoClusterCoordinatorException) Logger(org.slf4j.Logger) ControllerServicesEntity(org.apache.nifi.web.api.entity.ControllerServicesEntity) Set(java.util.Set) ProcessorsEntity(org.apache.nifi.web.api.entity.ProcessorsEntity) MultivaluedHashMap(javax.ws.rs.core.MultivaluedHashMap) Collectors(java.util.stream.Collectors) ReplicationTarget(org.apache.nifi.web.api.ApplicationResource.ReplicationTarget) MultivaluedMap(javax.ws.rs.core.MultivaluedMap) ScheduledState(org.apache.nifi.controller.ScheduledState) ControllerServiceState(org.apache.nifi.controller.service.ControllerServiceState) AffectedComponentEntity(org.apache.nifi.web.api.entity.AffectedComponentEntity) DtoFactory(org.apache.nifi.web.api.dto.DtoFactory) ScheduleComponentsEntity(org.apache.nifi.web.api.entity.ScheduleComponentsEntity) AffectedComponentDTO(org.apache.nifi.web.api.dto.AffectedComponentDTO) AffectedComponentEntity(org.apache.nifi.web.api.entity.AffectedComponentEntity)

Example 25 with AffectedComponentEntity

use of org.apache.nifi.web.api.entity.AffectedComponentEntity in project nifi by apache.

the class ClusterReplicationComponentLifecycle method scheduleComponents.

@Override
public Set<AffectedComponentEntity> scheduleComponents(final URI exampleUri, final NiFiUser user, final String groupId, final Set<AffectedComponentEntity> components, final ScheduledState desiredState, final Pause pause) throws LifecycleManagementException {
    final Set<String> componentIds = components.stream().map(component -> component.getId()).collect(Collectors.toSet());
    final Map<String, AffectedComponentEntity> componentMap = components.stream().collect(Collectors.toMap(AffectedComponentEntity::getId, Function.identity()));
    final Map<String, Revision> componentRevisionMap = getRevisions(groupId, componentIds);
    final Map<String, RevisionDTO> componentRevisionDtoMap = componentRevisionMap.entrySet().stream().collect(Collectors.toMap(Map.Entry::getKey, entry -> dtoFactory.createRevisionDTO(entry.getValue())));
    final ScheduleComponentsEntity scheduleProcessorsEntity = new ScheduleComponentsEntity();
    scheduleProcessorsEntity.setComponents(componentRevisionDtoMap);
    scheduleProcessorsEntity.setId(groupId);
    scheduleProcessorsEntity.setState(desiredState.name());
    URI scheduleGroupUri;
    try {
        scheduleGroupUri = new URI(exampleUri.getScheme(), exampleUri.getUserInfo(), exampleUri.getHost(), exampleUri.getPort(), "/nifi-api/flow/process-groups/" + groupId, null, exampleUri.getFragment());
    } catch (URISyntaxException e) {
        throw new RuntimeException(e);
    }
    final Map<String, String> headers = new HashMap<>();
    headers.put("content-type", MediaType.APPLICATION_JSON);
    // Determine whether we should replicate only to the cluster coordinator, or if we should replicate directly to the cluster nodes themselves.
    try {
        final NodeResponse clusterResponse;
        if (getReplicationTarget() == ReplicationTarget.CLUSTER_NODES) {
            clusterResponse = getRequestReplicator().replicate(user, HttpMethod.PUT, scheduleGroupUri, scheduleProcessorsEntity, headers).awaitMergedResponse();
        } else {
            clusterResponse = getRequestReplicator().forwardToCoordinator(getClusterCoordinatorNode(), user, HttpMethod.PUT, scheduleGroupUri, scheduleProcessorsEntity, headers).awaitMergedResponse();
        }
        final int scheduleComponentStatus = clusterResponse.getStatus();
        if (scheduleComponentStatus != Status.OK.getStatusCode()) {
            final String explanation = getResponseEntity(clusterResponse, String.class);
            throw new LifecycleManagementException("Failed to transition components to a state of " + desiredState + " due to " + explanation);
        }
        final boolean processorsTransitioned = waitForProcessorStatus(user, exampleUri, groupId, componentMap, desiredState, pause);
        if (!processorsTransitioned) {
            throw new LifecycleManagementException("Failed while waiting for components to transition to state of " + desiredState);
        }
    } catch (final InterruptedException ie) {
        Thread.currentThread().interrupt();
        throw new LifecycleManagementException("Interrupted while attempting to transition components to state of " + desiredState);
    }
    final Set<AffectedComponentEntity> updatedEntities = components.stream().map(component -> AffectedComponentUtils.updateEntity(component, serviceFacade, dtoFactory, user)).collect(Collectors.toSet());
    return updatedEntities;
}
Also used : NodeIdentifier(org.apache.nifi.cluster.protocol.NodeIdentifier) Revision(org.apache.nifi.web.Revision) ProcessorStatusDTO(org.apache.nifi.web.api.dto.status.ProcessorStatusDTO) URISyntaxException(java.net.URISyntaxException) LoggerFactory(org.slf4j.LoggerFactory) ControllerServiceEntity(org.apache.nifi.web.api.entity.ControllerServiceEntity) HashMap(java.util.HashMap) RevisionDTO(org.apache.nifi.web.api.dto.RevisionDTO) Function(java.util.function.Function) AffectedComponentDTO(org.apache.nifi.web.api.dto.AffectedComponentDTO) HttpMethod(javax.ws.rs.HttpMethod) ProcessorEntity(org.apache.nifi.web.api.entity.ProcessorEntity) MediaType(javax.ws.rs.core.MediaType) NiFiUser(org.apache.nifi.authorization.user.NiFiUser) ActivateControllerServicesEntity(org.apache.nifi.web.api.entity.ActivateControllerServicesEntity) Map(java.util.Map) ClusterCoordinator(org.apache.nifi.cluster.coordination.ClusterCoordinator) RequestReplicator(org.apache.nifi.cluster.coordination.http.replication.RequestReplicator) URI(java.net.URI) Status(javax.ws.rs.core.Response.Status) NodeResponse(org.apache.nifi.cluster.manager.NodeResponse) NiFiServiceFacade(org.apache.nifi.web.NiFiServiceFacade) NoClusterCoordinatorException(org.apache.nifi.cluster.exception.NoClusterCoordinatorException) Logger(org.slf4j.Logger) ControllerServicesEntity(org.apache.nifi.web.api.entity.ControllerServicesEntity) Set(java.util.Set) ProcessorsEntity(org.apache.nifi.web.api.entity.ProcessorsEntity) MultivaluedHashMap(javax.ws.rs.core.MultivaluedHashMap) Collectors(java.util.stream.Collectors) ReplicationTarget(org.apache.nifi.web.api.ApplicationResource.ReplicationTarget) MultivaluedMap(javax.ws.rs.core.MultivaluedMap) ScheduledState(org.apache.nifi.controller.ScheduledState) ControllerServiceState(org.apache.nifi.controller.service.ControllerServiceState) AffectedComponentEntity(org.apache.nifi.web.api.entity.AffectedComponentEntity) DtoFactory(org.apache.nifi.web.api.dto.DtoFactory) ScheduleComponentsEntity(org.apache.nifi.web.api.entity.ScheduleComponentsEntity) HashMap(java.util.HashMap) MultivaluedHashMap(javax.ws.rs.core.MultivaluedHashMap) NodeResponse(org.apache.nifi.cluster.manager.NodeResponse) ScheduleComponentsEntity(org.apache.nifi.web.api.entity.ScheduleComponentsEntity) URISyntaxException(java.net.URISyntaxException) URI(java.net.URI) RevisionDTO(org.apache.nifi.web.api.dto.RevisionDTO) Revision(org.apache.nifi.web.Revision) AffectedComponentEntity(org.apache.nifi.web.api.entity.AffectedComponentEntity) HashMap(java.util.HashMap) Map(java.util.Map) MultivaluedHashMap(javax.ws.rs.core.MultivaluedHashMap) MultivaluedMap(javax.ws.rs.core.MultivaluedMap)

Aggregations

AffectedComponentEntity (org.apache.nifi.web.api.entity.AffectedComponentEntity)28 AffectedComponentDTO (org.apache.nifi.web.api.dto.AffectedComponentDTO)19 Map (java.util.Map)18 Set (java.util.Set)18 HashMap (java.util.HashMap)17 Collectors (java.util.stream.Collectors)16 NiFiUser (org.apache.nifi.authorization.user.NiFiUser)16 ScheduledState (org.apache.nifi.controller.ScheduledState)15 ControllerServiceState (org.apache.nifi.controller.service.ControllerServiceState)15 Revision (org.apache.nifi.web.Revision)15 DtoFactory (org.apache.nifi.web.api.dto.DtoFactory)15 Logger (org.slf4j.Logger)15 LoggerFactory (org.slf4j.LoggerFactory)15 URI (java.net.URI)14 NiFiServiceFacade (org.apache.nifi.web.NiFiServiceFacade)14 Function (java.util.function.Function)13 RevisionDTO (org.apache.nifi.web.api.dto.RevisionDTO)13 ProcessorStatusDTO (org.apache.nifi.web.api.dto.status.ProcessorStatusDTO)13 ControllerServiceEntity (org.apache.nifi.web.api.entity.ControllerServiceEntity)13 ProcessorEntity (org.apache.nifi.web.api.entity.ProcessorEntity)13