Search in sources :

Example 6 with ImportFeed

use of com.thinkbiganalytics.feedmgr.service.feed.importing.model.ImportFeed in project kylo by Teradata.

the class ImportUtil method applyImportPropertiesToFeed.

public static boolean applyImportPropertiesToFeed(FeedMetadata metadata, ImportFeed importFeed, ImportComponent component) {
    ImportComponentOption option = importFeed.getImportOptions().findImportComponentOption(component);
    if (!option.getProperties().isEmpty() && option.getProperties().stream().anyMatch(importProperty -> StringUtils.isBlank(importProperty.getPropertyValue()))) {
        importFeed.setSuccess(false);
        if (importFeed.getTemplate() == null) {
            ImportTemplate importTemplate = new ImportTemplate(importFeed.getFileName());
            importFeed.setTemplate(importTemplate);
        }
        String feedCategory = importFeed.getImportOptions().getCategorySystemName() != null ? importFeed.getImportOptions().getCategorySystemName() : metadata.getSystemCategoryName();
        String msg = "The feed " + FeedNameUtil.fullName(feedCategory, metadata.getSystemFeedName()) + " needs additional properties to be supplied before importing.";
        importFeed.addErrorMessage(metadata, msg);
        option.getErrorMessages().add(msg);
        return false;
    } else {
        metadata.getSensitiveProperties().forEach(nifiProperty -> {
            ImportProperty userSuppliedValue = importFeed.getImportOptions().getProperties(ImportComponent.FEED_DATA).stream().filter(importFeedProperty -> {
                return nifiProperty.getProcessorId().equalsIgnoreCase(importFeedProperty.getProcessorId()) && nifiProperty.getKey().equalsIgnoreCase(importFeedProperty.getPropertyKey());
            }).findFirst().orElse(null);
            // deal with nulls?
            if (userSuppliedValue != null) {
                nifiProperty.setValue(userSuppliedValue.getPropertyValue());
            }
        });
        return true;
    }
}
Also used : FeedDataTransformation(com.thinkbiganalytics.feedmgr.rest.model.FeedDataTransformation) ZipInputStream(java.util.zip.ZipInputStream) ByteArrayOutputStream(java.io.ByteArrayOutputStream) NifiError(com.thinkbiganalytics.nifi.rest.model.NifiError) ReusableTemplateConnectionInfo(com.thinkbiganalytics.feedmgr.rest.model.ReusableTemplateConnectionInfo) FeedMetadata(com.thinkbiganalytics.feedmgr.rest.model.FeedMetadata) StringUtils(org.apache.commons.lang3.StringUtils) ImportOptions(com.thinkbiganalytics.feedmgr.rest.model.ImportOptions) HashSet(java.util.HashSet) ObjectMapperSerializer(com.thinkbiganalytics.json.ObjectMapperSerializer) ByteArrayInputStream(java.io.ByteArrayInputStream) ImportProperty(com.thinkbiganalytics.feedmgr.rest.model.ImportProperty) Map(java.util.Map) TypeReference(com.fasterxml.jackson.core.type.TypeReference) ImportType(com.thinkbiganalytics.feedmgr.rest.ImportType) RegisteredTemplate(com.thinkbiganalytics.feedmgr.rest.model.RegisteredTemplate) ZipEntry(java.util.zip.ZipEntry) Nonnull(javax.annotation.Nonnull) RemoteProcessGroupInputPort(com.thinkbiganalytics.feedmgr.rest.model.RemoteProcessGroupInputPort) ImportTemplate(com.thinkbiganalytics.feedmgr.service.template.importing.model.ImportTemplate) NifiProperty(com.thinkbiganalytics.nifi.rest.model.NifiProperty) FeedNameUtil(com.thinkbiganalytics.support.FeedNameUtil) StringWriter(java.io.StringWriter) Set(java.util.Set) IOException(java.io.IOException) Collectors(java.util.stream.Collectors) ZipFileUtil(com.thinkbiganalytics.feedmgr.support.ZipFileUtil) IOUtils(org.apache.commons.io.IOUtils) List(java.util.List) ImportFeed(com.thinkbiganalytics.feedmgr.service.feed.importing.model.ImportFeed) Optional(java.util.Optional) ImportComponent(com.thinkbiganalytics.feedmgr.rest.ImportComponent) ImportComponentOption(com.thinkbiganalytics.feedmgr.rest.model.ImportComponentOption) NifiProcessGroup(com.thinkbiganalytics.nifi.rest.model.NifiProcessGroup) InputStream(java.io.InputStream) ImportProperty(com.thinkbiganalytics.feedmgr.rest.model.ImportProperty) ImportComponentOption(com.thinkbiganalytics.feedmgr.rest.model.ImportComponentOption) ImportTemplate(com.thinkbiganalytics.feedmgr.service.template.importing.model.ImportTemplate)

Example 7 with ImportFeed

use of com.thinkbiganalytics.feedmgr.service.feed.importing.model.ImportFeed in project kylo by Teradata.

the class FeedIT method importSystemFeeds.

protected void importSystemFeeds() {
    ImportFeed textIndex = importFeed(sampleFeedsPath + INDEX_TEXT_SERVICE_V2_FEED_ZIP);
    enableFeed(textIndex.getNifiFeed().getFeedMetadata().getFeedId());
}
Also used : ImportFeed(com.thinkbiganalytics.feedmgr.service.feed.importing.model.ImportFeed)

Aggregations

ImportFeed (com.thinkbiganalytics.feedmgr.service.feed.importing.model.ImportFeed)7 ImportComponentOption (com.thinkbiganalytics.feedmgr.rest.model.ImportComponentOption)4 ImportFeedOptions (com.thinkbiganalytics.feedmgr.rest.model.ImportFeedOptions)4 ImportProperty (com.thinkbiganalytics.feedmgr.rest.model.ImportProperty)4 List (java.util.List)4 ImportComponent (com.thinkbiganalytics.feedmgr.rest.ImportComponent)3 ImportType (com.thinkbiganalytics.feedmgr.rest.ImportType)3 FeedMetadata (com.thinkbiganalytics.feedmgr.rest.model.FeedMetadata)3 RegisteredTemplate (com.thinkbiganalytics.feedmgr.rest.model.RegisteredTemplate)3 ImportTemplate (com.thinkbiganalytics.feedmgr.service.template.importing.model.ImportTemplate)3 ZipFileUtil (com.thinkbiganalytics.feedmgr.support.ZipFileUtil)3 ByteArrayInputStream (java.io.ByteArrayInputStream)3 InputStream (java.io.InputStream)3 ZipEntry (java.util.zip.ZipEntry)3 ZipInputStream (java.util.zip.ZipInputStream)3 TypeReference (com.fasterxml.jackson.core.type.TypeReference)2 Sets (com.google.common.collect.Sets)2 MetadataFieldAnnotationFieldNameResolver (com.thinkbiganalytics.feedmgr.MetadataFieldAnnotationFieldNameResolver)2 ImportSection (com.thinkbiganalytics.feedmgr.rest.ImportSection)2 FeedCategory (com.thinkbiganalytics.feedmgr.rest.model.FeedCategory)2