Search in sources :

Example 6 with JobTemplate

use of org.apache.gobblin.runtime.api.JobTemplate in project incubator-gobblin by apache.

the class InheritingJobTemplate method getResolvedConfigHelper.

private Config getResolvedConfigHelper(Config userConfig, Set<JobTemplate> alreadyLoadedTemplates) throws SpecNotFoundException, TemplateException {
    Config config = getLocallyResolvedConfig(userConfig);
    for (JobTemplate template : Lists.reverse(this.superTemplates)) {
        if (!alreadyLoadedTemplates.contains(template)) {
            alreadyLoadedTemplates.add(template);
            Config fallback = template instanceof InheritingJobTemplate ? ((InheritingJobTemplate) template).getResolvedConfigHelper(config, alreadyLoadedTemplates) : template.getResolvedConfig(config);
            config = config.withFallback(fallback);
        }
    }
    return config;
}
Also used : Config(com.typesafe.config.Config) JobTemplate(org.apache.gobblin.runtime.api.JobTemplate)

Example 7 with JobTemplate

use of org.apache.gobblin.runtime.api.JobTemplate in project incubator-gobblin by apache.

the class ResolvedJobSpec method resolveConfig.

private static Config resolveConfig(JobSpec jobSpec, JobCatalog catalog) throws SpecNotFoundException, JobTemplate.TemplateException {
    Optional<URI> templateURIOpt = jobSpec.getTemplateURI();
    if (templateURIOpt.isPresent()) {
        JobCatalogWithTemplates catalogWithTemplates = new PackagedTemplatesJobCatalogDecorator(catalog);
        JobTemplate template = catalogWithTemplates.getTemplate(templateURIOpt.get());
        return template.getResolvedConfig(jobSpec.getConfig()).resolve();
    } else {
        return jobSpec.getConfig().resolve();
    }
}
Also used : JobCatalogWithTemplates(org.apache.gobblin.runtime.api.JobCatalogWithTemplates) PackagedTemplatesJobCatalogDecorator(org.apache.gobblin.runtime.job_catalog.PackagedTemplatesJobCatalogDecorator) JobTemplate(org.apache.gobblin.runtime.api.JobTemplate) URI(java.net.URI)

Aggregations

JobTemplate (org.apache.gobblin.runtime.api.JobTemplate)7 URI (java.net.URI)4 Config (com.typesafe.config.Config)3 JobCatalogWithTemplates (org.apache.gobblin.runtime.api.JobCatalogWithTemplates)2 Test (org.testng.annotations.Test)2 SpecNotFoundException (org.apache.gobblin.runtime.api.SpecNotFoundException)1 PackagedTemplatesJobCatalogDecorator (org.apache.gobblin.runtime.job_catalog.PackagedTemplatesJobCatalogDecorator)1 ResourceBasedJobTemplate (org.apache.gobblin.runtime.template.ResourceBasedJobTemplate)1