Search in sources :

Example 21 with MetadataProviderService

use of com.thinkbiganalytics.nifi.core.api.metadata.MetadataProviderService in project kylo by Teradata.

the class UpdateFeedHistoryReindex method onTrigger.

@Override
public void onTrigger(ProcessContext context, ProcessSession session) throws ProcessException {
    final ComponentLog logger = getLog();
    FlowFile flowFile = session.get();
    if (flowFile == null) {
        logger.warn("Update feed history reindex status processor called without an in-coming flow file. " + "Ensure that this is the intended usage.");
        flowFile = session.create();
    }
    final MetadataProviderService metadataProviderService = context.getProperty(METADATA_SERVICE).asControllerService(MetadataProviderService.class);
    final String feedId = context.getProperty(FEED_ID).evaluateAttributeExpressions(flowFile).getValue();
    final String historyReindexingStateAsString = context.getProperty(FEED_REINDEX_STATUS).evaluateAttributeExpressions(flowFile).getValue();
    logger.debug("Updating history reindex status for feed with id {}. New status for update is {}...", new Object[] { feedId, historyReindexingStateAsString });
    if (feedId != null && metadataProviderService != null && metadataProviderService.getRecorder() != null) {
        try {
            FeedDataHistoryReindexParams feedDataHistoryReindexParams = metadataProviderService.getRecorder().updateFeedHistoryReindexing(feedId, new HistoryReindexingStatus(HistoryReindexingState.valueOf(historyReindexingStateAsString)));
            if (feedDataHistoryReindexParams == null) {
                logger.error("Error updating history reindex status to {} for feed with id {}", new Object[] { feedId, historyReindexingStateAsString });
                session.transfer(flowFile, REL_FAILURE);
            } else {
                flowFile = session.putAttribute(flowFile, UPDATED_FEED_INFO_FOR_HISTORY_REINDEX_KEY, getFeedInfo(feedDataHistoryReindexParams));
                flowFile = session.putAttribute(flowFile, UPDATED_FEED_STATUS_FOR_HISTORY_REINDEX_KEY, feedDataHistoryReindexParams.getHistoryReindexingStatus().getHistoryReindexingState().toString());
                flowFile = session.putAttribute(flowFile, UPDATED_TIME_UTC_FOR_HISTORY_REINDEX_KEY, feedDataHistoryReindexParams.getHistoryReindexingStatus().getLastModifiedTimestamp().toString());
                flowFile = session.putAttribute(flowFile, UPDATED_INDEX_COLUMNS_STRING_FOR_HISTORY_REINDEX_KEY, feedDataHistoryReindexParams.getCommaSeparatedColumnsForIndexing());
                logger.info("Updated reindex history status to {} for feed with id {}", new Object[] { feedDataHistoryReindexParams.getHistoryReindexingStatus().getHistoryReindexingState().toString(), feedDataHistoryReindexParams.getFeedId() });
                session.transfer(flowFile, REL_SUCCESS);
            }
        } catch (Exception e) {
            logger.error("An exception was thrown during updating history reindex status to {} for feed id {}: {}", new Object[] { feedId, historyReindexingStateAsString, e });
            session.transfer(flowFile, REL_FAILURE);
        }
    } else {
        logger.error("One or more of {feed id, metadata service, metadata service recorder} is null for updating history reindex status operation");
        session.transfer(flowFile, REL_FAILURE);
    }
}
Also used : FlowFile(org.apache.nifi.flowfile.FlowFile) HistoryReindexingStatus(com.thinkbiganalytics.metadata.rest.model.feed.reindex.HistoryReindexingStatus) FeedDataHistoryReindexParams(com.thinkbiganalytics.metadata.rest.model.feed.reindex.FeedDataHistoryReindexParams) ComponentLog(org.apache.nifi.logging.ComponentLog) MetadataProviderService(com.thinkbiganalytics.nifi.core.api.metadata.MetadataProviderService) ProcessException(org.apache.nifi.processor.exception.ProcessException)

Aggregations

MetadataProviderService (com.thinkbiganalytics.nifi.core.api.metadata.MetadataProviderService)21 Test (org.junit.Test)14 TestRunner (org.apache.nifi.util.TestRunner)13 Matchers.anyString (org.mockito.Matchers.anyString)9 MockFlowFile (org.apache.nifi.util.MockFlowFile)5 FlowFile (org.apache.nifi.flowfile.FlowFile)4 ComponentLog (org.apache.nifi.logging.ComponentLog)4 ProcessException (org.apache.nifi.processor.exception.ProcessException)3 Before (org.junit.Before)3 IOException (java.io.IOException)2 DBCPService (org.apache.nifi.dbcp.DBCPService)2 JsonProcessingException (com.fasterxml.jackson.core.JsonProcessingException)1 GetTableDataSupport (com.thinkbiganalytics.ingest.GetTableDataSupport)1 Feed (com.thinkbiganalytics.metadata.rest.model.feed.Feed)1 FeedDataHistoryReindexParams (com.thinkbiganalytics.metadata.rest.model.feed.reindex.FeedDataHistoryReindexParams)1 FeedsForDataHistoryReindex (com.thinkbiganalytics.metadata.rest.model.feed.reindex.FeedsForDataHistoryReindex)1 HistoryReindexingStatus (com.thinkbiganalytics.metadata.rest.model.feed.reindex.HistoryReindexingStatus)1 CleanupEventService (com.thinkbiganalytics.nifi.core.api.cleanup.CleanupEventService)1 ApplySecurityPolicy (com.thinkbiganalytics.nifi.security.ApplySecurityPolicy)1 InputStreamReaderRunnable (com.thinkbiganalytics.nifi.util.InputStreamReaderRunnable)1