Search in sources :

Example 1 with VersionControlComponentMappingEntity

use of org.apache.nifi.web.api.entity.VersionControlComponentMappingEntity in project nifi by apache.

the class StandardNiFiServiceFacade method registerFlowWithFlowRegistry.

@Override
public VersionControlComponentMappingEntity registerFlowWithFlowRegistry(final String groupId, final StartVersionControlRequestEntity requestEntity) {
    final ProcessGroup processGroup = processGroupDAO.getProcessGroup(groupId);
    final VersionControlInformation currentVci = processGroup.getVersionControlInformation();
    final int expectedVersion = currentVci == null ? 1 : currentVci.getVersion() + 1;
    // Create a VersionedProcessGroup snapshot of the flow as it is currently.
    final InstantiatedVersionedProcessGroup versionedProcessGroup = createFlowSnapshot(groupId);
    final VersionedFlowDTO versionedFlowDto = requestEntity.getVersionedFlow();
    final String flowId = versionedFlowDto.getFlowId() == null ? UUID.randomUUID().toString() : versionedFlowDto.getFlowId();
    final VersionedFlow versionedFlow = new VersionedFlow();
    versionedFlow.setBucketIdentifier(versionedFlowDto.getBucketId());
    versionedFlow.setCreatedTimestamp(System.currentTimeMillis());
    versionedFlow.setDescription(versionedFlowDto.getDescription());
    versionedFlow.setModifiedTimestamp(versionedFlow.getCreatedTimestamp());
    versionedFlow.setName(versionedFlowDto.getFlowName());
    versionedFlow.setIdentifier(flowId);
    // Add the Versioned Flow and first snapshot to the Flow Registry
    final String registryId = requestEntity.getVersionedFlow().getRegistryId();
    final VersionedFlowSnapshot registeredSnapshot;
    final VersionedFlow registeredFlow;
    String action = "create the flow";
    try {
        // first, create the flow in the registry, if necessary
        if (versionedFlowDto.getFlowId() == null) {
            registeredFlow = registerVersionedFlow(registryId, versionedFlow);
        } else {
            registeredFlow = getVersionedFlow(registryId, versionedFlowDto.getBucketId(), versionedFlowDto.getFlowId());
        }
        action = "add the local flow to the Flow Registry as the first Snapshot";
        // add first snapshot to the flow in the registry
        registeredSnapshot = registerVersionedFlowSnapshot(registryId, registeredFlow, versionedProcessGroup, versionedFlowDto.getComments(), expectedVersion);
    } catch (final NiFiRegistryException e) {
        throw new IllegalArgumentException(e.getLocalizedMessage());
    } catch (final IOException ioe) {
        throw new IllegalStateException("Failed to communicate with Flow Registry when attempting to " + action);
    }
    final Bucket bucket = registeredSnapshot.getBucket();
    final VersionedFlow flow = registeredSnapshot.getFlow();
    // Update the Process Group with the new VersionControlInformation. (Send this to all nodes).
    final VersionControlInformationDTO vci = new VersionControlInformationDTO();
    vci.setBucketId(bucket.getIdentifier());
    vci.setBucketName(bucket.getName());
    vci.setFlowId(flow.getIdentifier());
    vci.setFlowName(flow.getName());
    vci.setFlowDescription(flow.getDescription());
    vci.setGroupId(groupId);
    vci.setRegistryId(registryId);
    vci.setRegistryName(getFlowRegistryName(registryId));
    vci.setVersion(registeredSnapshot.getSnapshotMetadata().getVersion());
    vci.setState(VersionedFlowState.UP_TO_DATE.name());
    final Map<String, String> mapping = dtoFactory.createVersionControlComponentMappingDto(versionedProcessGroup);
    final Revision groupRevision = revisionManager.getRevision(groupId);
    final RevisionDTO groupRevisionDto = dtoFactory.createRevisionDTO(groupRevision);
    final VersionControlComponentMappingEntity entity = new VersionControlComponentMappingEntity();
    entity.setVersionControlInformation(vci);
    entity.setProcessGroupRevision(groupRevisionDto);
    entity.setVersionControlComponentMapping(mapping);
    return entity;
}
Also used : VersionedFlow(org.apache.nifi.registry.flow.VersionedFlow) InstantiatedVersionedProcessGroup(org.apache.nifi.registry.flow.mapping.InstantiatedVersionedProcessGroup) IOException(java.io.IOException) RevisionDTO(org.apache.nifi.web.api.dto.RevisionDTO) VersionControlInformationDTO(org.apache.nifi.web.api.dto.VersionControlInformationDTO) VersionControlInformation(org.apache.nifi.registry.flow.VersionControlInformation) VersionedFlowDTO(org.apache.nifi.web.api.dto.VersionedFlowDTO) Bucket(org.apache.nifi.registry.bucket.Bucket) VersionedProcessGroup(org.apache.nifi.registry.flow.VersionedProcessGroup) RemoteProcessGroup(org.apache.nifi.groups.RemoteProcessGroup) ProcessGroup(org.apache.nifi.groups.ProcessGroup) InstantiatedVersionedProcessGroup(org.apache.nifi.registry.flow.mapping.InstantiatedVersionedProcessGroup) VersionedFlowSnapshot(org.apache.nifi.registry.flow.VersionedFlowSnapshot) NiFiRegistryException(org.apache.nifi.registry.client.NiFiRegistryException) VersionControlComponentMappingEntity(org.apache.nifi.web.api.entity.VersionControlComponentMappingEntity)

Example 2 with VersionControlComponentMappingEntity

use of org.apache.nifi.web.api.entity.VersionControlComponentMappingEntity in project nifi by apache.

the class VersionsResource method saveToFlowRegistry.

@POST
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
@Path("process-groups/{id}")
@ApiOperation(value = "Save the Process Group with the given ID", response = VersionControlInformationEntity.class, notes = "Begins version controlling the Process Group with the given ID or commits changes to the Versioned Flow, " + "depending on if the provided VersionControlInformation includes a flowId. " + NON_GUARANTEED_ENDPOINT, authorizations = { @Authorization(value = "Read - /process-groups/{uuid}"), @Authorization(value = "Write - /process-groups/{uuid}"), @Authorization(value = "Read - /{component-type}/{uuid} - For all encapsulated components"), @Authorization(value = "Read - any referenced Controller Services by any encapsulated components - /controller-services/{uuid}") })
@ApiResponses(value = { @ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."), @ApiResponse(code = 401, message = "Client could not be authenticated."), @ApiResponse(code = 403, message = "Client is not authorized to make this request."), @ApiResponse(code = 404, message = "The specified resource could not be found."), @ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.") })
public Response saveToFlowRegistry(@ApiParam("The process group id.") @PathParam("id") final String groupId, @ApiParam(value = "The versioned flow details.", required = true) final StartVersionControlRequestEntity requestEntity) {
    // Verify the request
    final RevisionDTO revisionDto = requestEntity.getProcessGroupRevision();
    if (revisionDto == null) {
        throw new IllegalArgumentException("Process Group Revision must be specified");
    }
    final VersionedFlowDTO versionedFlowDto = requestEntity.getVersionedFlow();
    if (versionedFlowDto == null) {
        throw new IllegalArgumentException("Version Control Information must be supplied.");
    }
    if (StringUtils.isEmpty(versionedFlowDto.getBucketId())) {
        throw new IllegalArgumentException("The Bucket ID must be supplied.");
    }
    if (StringUtils.isEmpty(versionedFlowDto.getFlowName()) && StringUtils.isEmpty(versionedFlowDto.getFlowId())) {
        throw new IllegalArgumentException("The Flow Name or Flow ID must be supplied.");
    }
    if (versionedFlowDto.getFlowName() != null && versionedFlowDto.getFlowName().length() > 1000) {
        throw new IllegalArgumentException("The Flow Name cannot exceed 1,000 characters");
    }
    if (StringUtils.isEmpty(versionedFlowDto.getRegistryId())) {
        throw new IllegalArgumentException("The Registry ID must be supplied.");
    }
    if (versionedFlowDto.getDescription() != null && versionedFlowDto.getDescription().length() > 65535) {
        throw new IllegalArgumentException("Flow Description cannot exceed 65,535 characters");
    }
    if (versionedFlowDto.getComments() != null && versionedFlowDto.getComments().length() > 65535) {
        throw new IllegalArgumentException("Comments cannot exceed 65,535 characters");
    }
    // ensure we're not attempting to version the root group
    final ProcessGroupEntity root = serviceFacade.getProcessGroup(FlowController.ROOT_GROUP_ID_ALIAS);
    if (root.getId().equals(groupId)) {
        throw new IllegalArgumentException("The Root Process Group cannot be versioned.");
    }
    if (isReplicateRequest()) {
        // We first have to obtain a "lock" on all nodes in the cluster so that multiple Version Control requests
        // are not being made simultaneously. We do this by making a POST to /nifi-api/versions/active-requests.
        // The Response gives us back the Request ID.
        final URI requestUri;
        try {
            final URI originalUri = getAbsolutePath();
            final String requestId = lockVersionControl(originalUri, groupId);
            requestUri = new URI(originalUri.getScheme(), originalUri.getUserInfo(), originalUri.getHost(), originalUri.getPort(), "/nifi-api/versions/active-requests/" + requestId, null, originalUri.getFragment());
        } catch (final URISyntaxException e) {
            throw new RuntimeException(e);
        }
        // Finally, we can delete the Request.
        try {
            final VersionControlComponentMappingEntity mappingEntity = serviceFacade.registerFlowWithFlowRegistry(groupId, requestEntity);
            replicateVersionControlMapping(mappingEntity, requestEntity, requestUri, groupId);
            final VersionControlInformationEntity responseEntity = serviceFacade.getVersionControlInformation(groupId);
            return generateOkResponse(responseEntity).build();
        } finally {
            unlockVersionControl(requestUri, groupId);
        }
    }
    // Perform local task. If running in a cluster environment, we will never get to this point. This is because
    // in the above block, we check if (isReplicate()) and if true, we implement the 'cluster logic', but this
    // does not involve replicating the actual request, because we only want a single node to handle the logic of
    // creating the flow in the Registry.
    final Revision groupRevision = new Revision(revisionDto.getVersion(), revisionDto.getClientId(), groupId);
    return withWriteLock(serviceFacade, requestEntity, groupRevision, lookup -> {
        final ProcessGroupAuthorizable groupAuthorizable = lookup.getProcessGroup(groupId);
        final Authorizable processGroup = groupAuthorizable.getAuthorizable();
        // require write to this group
        processGroup.authorize(authorizer, RequestAction.WRITE, NiFiUserUtils.getNiFiUser());
        // require read to this group and all descendants
        authorizeProcessGroup(groupAuthorizable, authorizer, lookup, RequestAction.READ, true, false, true, true);
    }, () -> {
        final VersionedFlowDTO versionedFlow = requestEntity.getVersionedFlow();
        final String registryId = versionedFlow.getRegistryId();
        final String bucketId = versionedFlow.getBucketId();
        final String flowId = versionedFlow.getFlowId();
        serviceFacade.verifyCanSaveToFlowRegistry(groupId, registryId, bucketId, flowId);
    }, (rev, flowEntity) -> {
        // Register the current flow with the Flow Registry.
        final VersionControlComponentMappingEntity mappingEntity = serviceFacade.registerFlowWithFlowRegistry(groupId, flowEntity);
        // Update the Process Group's Version Control Information
        final VersionControlInformationEntity responseEntity = serviceFacade.setVersionControlInformation(rev, groupId, mappingEntity.getVersionControlInformation(), mappingEntity.getVersionControlComponentMapping());
        return generateOkResponse(responseEntity).build();
    });
}
Also used : ProcessGroupEntity(org.apache.nifi.web.api.entity.ProcessGroupEntity) URISyntaxException(java.net.URISyntaxException) URI(java.net.URI) RevisionDTO(org.apache.nifi.web.api.dto.RevisionDTO) VersionControlInformationEntity(org.apache.nifi.web.api.entity.VersionControlInformationEntity) ProcessGroupAuthorizable(org.apache.nifi.authorization.ProcessGroupAuthorizable) VersionedFlowDTO(org.apache.nifi.web.api.dto.VersionedFlowDTO) Revision(org.apache.nifi.web.Revision) ComponentAuthorizable(org.apache.nifi.authorization.ComponentAuthorizable) Authorizable(org.apache.nifi.authorization.resource.Authorizable) ProcessGroupAuthorizable(org.apache.nifi.authorization.ProcessGroupAuthorizable) VersionControlComponentMappingEntity(org.apache.nifi.web.api.entity.VersionControlComponentMappingEntity) Path(javax.ws.rs.Path) POST(javax.ws.rs.POST) Consumes(javax.ws.rs.Consumes) Produces(javax.ws.rs.Produces) ApiOperation(io.swagger.annotations.ApiOperation) ApiResponses(io.swagger.annotations.ApiResponses)

Aggregations

RevisionDTO (org.apache.nifi.web.api.dto.RevisionDTO)2 VersionedFlowDTO (org.apache.nifi.web.api.dto.VersionedFlowDTO)2 VersionControlComponentMappingEntity (org.apache.nifi.web.api.entity.VersionControlComponentMappingEntity)2 ApiOperation (io.swagger.annotations.ApiOperation)1 ApiResponses (io.swagger.annotations.ApiResponses)1 IOException (java.io.IOException)1 URI (java.net.URI)1 URISyntaxException (java.net.URISyntaxException)1 Consumes (javax.ws.rs.Consumes)1 POST (javax.ws.rs.POST)1 Path (javax.ws.rs.Path)1 Produces (javax.ws.rs.Produces)1 ComponentAuthorizable (org.apache.nifi.authorization.ComponentAuthorizable)1 ProcessGroupAuthorizable (org.apache.nifi.authorization.ProcessGroupAuthorizable)1 Authorizable (org.apache.nifi.authorization.resource.Authorizable)1 ProcessGroup (org.apache.nifi.groups.ProcessGroup)1 RemoteProcessGroup (org.apache.nifi.groups.RemoteProcessGroup)1 Bucket (org.apache.nifi.registry.bucket.Bucket)1 NiFiRegistryException (org.apache.nifi.registry.client.NiFiRegistryException)1 VersionControlInformation (org.apache.nifi.registry.flow.VersionControlInformation)1