Search in sources :

Example 86 with FeedMetadata

use of com.thinkbiganalytics.feedmgr.rest.model.FeedMetadata in project kylo by Teradata.

the class FeedManagerMetadataService method disableFeed.

public FeedSummary disableFeed(final String feedId) {
    return metadataAccess.commit(() -> {
        this.accessController.checkPermission(AccessController.SERVICES, FeedServicesAccessControl.EDIT_FEEDS);
        FeedMetadata feedMetadata = feedProvider.getFeedById(feedId);
        if (feedMetadata == null) {
            throw new NotFoundException("Feed not found for id " + feedId);
        }
        if (!feedMetadata.getState().equals(Feed.State.DISABLED.name())) {
            FeedSummary feedSummary = feedProvider.disableFeed(feedId);
            boolean updatedNifi = updateNifiFeedRunningStatus(feedSummary, Feed.State.DISABLED);
            if (!updatedNifi) {
                // rollback
                throw new RuntimeException("Unable to disable Feed " + feedId);
            }
            return feedSummary;
        }
        return new FeedSummary(feedMetadata);
    });
}
Also used : NifiClientRuntimeException(com.thinkbiganalytics.nifi.rest.client.NifiClientRuntimeException) FeedMetadata(com.thinkbiganalytics.feedmgr.rest.model.FeedMetadata) NotFoundException(javax.ws.rs.NotFoundException) FeedNotFoundException(com.thinkbiganalytics.metadata.api.feed.FeedNotFoundException) FeedSummary(com.thinkbiganalytics.feedmgr.rest.model.FeedSummary)

Example 87 with FeedMetadata

use of com.thinkbiganalytics.feedmgr.rest.model.FeedMetadata in project kylo by Teradata.

the class FeedRestController method profileStats.

@GET
@Path("/{feedId}/profile-stats")
@Produces(MediaType.APPLICATION_JSON)
@ApiOperation("Gets the profile statistics for the specified job.")
@ApiResponses({ @ApiResponse(code = 200, message = "Returns the profile statistics.", response = Map.class, responseContainer = "List"), @ApiResponse(code = 500, message = "The profile is unavailable.", response = RestResponseStatus.class) })
public Response profileStats(@PathParam("feedId") String feedId, @QueryParam("processingdttm") String processingdttm) {
    FeedMetadata feedMetadata = getMetadataService().getFeedById(feedId);
    String profileTable = feedMetadata.getProfileTableName();
    String query = "SELECT * from " + HiveUtils.quoteIdentifier(profileTable) + " where processing_dttm = " + HiveUtils.quoteString(processingdttm);
    QueryResult rows = hiveService.query(query);
    return Response.ok(rows.getRows()).build();
}
Also used : QueryResult(com.thinkbiganalytics.discovery.schema.QueryResult) FeedMetadata(com.thinkbiganalytics.feedmgr.rest.model.FeedMetadata) Path(javax.ws.rs.Path) Produces(javax.ws.rs.Produces) GET(javax.ws.rs.GET) ApiOperation(io.swagger.annotations.ApiOperation) ApiResponses(io.swagger.annotations.ApiResponses)

Example 88 with FeedMetadata

use of com.thinkbiganalytics.feedmgr.rest.model.FeedMetadata in project kylo by Teradata.

the class DefaultFeedManagerFeedService method disableFeed.

public FeedSummary disableFeed(final String feedId) {
    return metadataAccess.commit(() -> {
        this.accessController.checkPermission(AccessController.SERVICES, FeedServicesAccessControl.EDIT_FEEDS);
        if (StringUtils.isNotBlank(feedId)) {
            FeedMetadata feedMetadata = getFeedById(feedId);
            Feed.ID domainId = feedProvider.resolveFeed(feedId);
            boolean disabled = disableFeed(domainId);
            // re fetch it
            if (disabled) {
                feedMetadata.setState(Feed.State.DISABLED.name());
                serviceLevelAgreementService.disableServiceLevelAgreementSchedule(domainId);
            }
            FeedSummary feedSummary = new FeedSummary(feedMetadata);
            return feedSummary;
        }
        return null;
    });
}
Also used : FeedMetadata(com.thinkbiganalytics.feedmgr.rest.model.FeedMetadata) NifiFeed(com.thinkbiganalytics.feedmgr.rest.model.NifiFeed) Feed(com.thinkbiganalytics.metadata.api.feed.Feed) UIFeed(com.thinkbiganalytics.feedmgr.rest.model.UIFeed) FeedSummary(com.thinkbiganalytics.feedmgr.rest.model.FeedSummary)

Example 89 with FeedMetadata

use of com.thinkbiganalytics.feedmgr.rest.model.FeedMetadata in project kylo by Teradata.

the class FeedModelTransform method deserializeFeedMetadata.

public FeedMetadata deserializeFeedMetadata(Feed domain, boolean clearSensitiveProperties) {
    String json = domain.getJson();
    FeedMetadata feedMetadata = ObjectMapperSerializer.deserialize(json, FeedMetadata.class);
    populate(feedMetadata, domain);
    if (clearSensitiveProperties) {
        clearSensitivePropertyValues(feedMetadata);
    }
    return feedMetadata;
}
Also used : FeedMetadata(com.thinkbiganalytics.feedmgr.rest.model.FeedMetadata)

Example 90 with FeedMetadata

use of com.thinkbiganalytics.feedmgr.rest.model.FeedMetadata in project kylo by Teradata.

the class FeedModelTransform method stripMetadata.

/**
 * Clean out any excess or redundant data that should not be serialized and stored
 * with the feed domain entity as JSON.
 * @param source the source metadata
 * @return a new metadata instance without the excess data
 */
private FeedMetadata stripMetadata(FeedMetadata source) {
    FeedMetadata result = new FeedMetadata();
    result.setDataOwner(source.getDataOwner());
    result.setDataTransformation(source.getDataTransformation());
    result.setHadoopAuthorizationType(source.getHadoopAuthorizationType());
    result.setInputProcessorType(source.getInputProcessorType());
    result.setInputProcessorName(source.getInputProcessorName());
    result.setIsReusableFeed(source.isReusableFeed());
    result.setNifiProcessGroupId(source.getNifiProcessGroupId());
    result.setOptions(source.getOptions());
    result.setProperties(source.getProperties());
    result.setSchedule(source.getSchedule());
    result.setTable(source.getTable());
    result.setTableOption(source.getTableOption());
    if (source.getSourceDataSets() != null) {
        result.setSourceDataSets(source.getSourceDataSets());
    }
    if (source.getSampleDataSet() != null) {
        result.setSampleDataSet(source.getSampleDataSet());
    }
    result.setUiState(source.getUiState());
    result.setActive(source.isActive());
    return result;
}
Also used : FeedMetadata(com.thinkbiganalytics.feedmgr.rest.model.FeedMetadata)

Aggregations

FeedMetadata (com.thinkbiganalytics.feedmgr.rest.model.FeedMetadata)96 NifiProperty (com.thinkbiganalytics.nifi.rest.model.NifiProperty)36 ArrayList (java.util.ArrayList)35 NifiFeed (com.thinkbiganalytics.feedmgr.rest.model.NifiFeed)32 List (java.util.List)30 Map (java.util.Map)30 Collectors (java.util.stream.Collectors)30 Set (java.util.Set)28 StringUtils (org.apache.commons.lang3.StringUtils)28 FeedCategory (com.thinkbiganalytics.feedmgr.rest.model.FeedCategory)27 RegisteredTemplate (com.thinkbiganalytics.feedmgr.rest.model.RegisteredTemplate)27 Logger (org.slf4j.Logger)27 LoggerFactory (org.slf4j.LoggerFactory)27 HashSet (java.util.HashSet)26 Inject (javax.inject.Inject)25 Nonnull (javax.annotation.Nonnull)24 FeedSummary (com.thinkbiganalytics.feedmgr.rest.model.FeedSummary)23 UIFeed (com.thinkbiganalytics.feedmgr.rest.model.UIFeed)23 FeedServicesAccessControl (com.thinkbiganalytics.feedmgr.security.FeedServicesAccessControl)23 AccessController (com.thinkbiganalytics.security.AccessController)23