Search in sources :

Example 1 with HOCONInputStreamFlowTemplate

use of org.apache.gobblin.service.modules.template.HOCONInputStreamFlowTemplate in project incubator-gobblin by apache.

the class FSFlowTemplateCatalog method getFlowTemplate.

/**
 * @param flowTemplateDirURI URI of the flow template directory
 * @return a {@link FlowTemplate}
 * @throws SpecNotFoundException
 * @throws JobTemplate.TemplateException
 * @throws IOException
 */
public FlowTemplate getFlowTemplate(URI flowTemplateDirURI) throws SpecNotFoundException, JobTemplate.TemplateException, IOException, URISyntaxException {
    if (!validateTemplateURI(flowTemplateDirURI)) {
        throw new JobTemplate.TemplateException(String.format("The FlowTemplate %s is not valid", flowTemplateDirURI));
    }
    String templateCatalogDir = this.sysConfig.getString(ServiceConfigKeys.TEMPLATE_CATALOGS_FULLY_QUALIFIED_PATH_KEY);
    // path of uri is location of template file relative to the job configuration root directory
    Path templateDirPath = PathUtils.mergePaths(new Path(templateCatalogDir), new Path(flowTemplateDirURI.getPath()));
    Path templateFullPath = PathUtils.mergePaths(templateDirPath, new Path(FLOW_CONF_FILE_NAME));
    FileSystem fs = FileSystem.get(templateFullPath.toUri(), new Configuration());
    try (InputStream is = fs.open(templateFullPath)) {
        return new HOCONInputStreamFlowTemplate(is, flowTemplateDirURI, this);
    }
}
Also used : Path(org.apache.hadoop.fs.Path) Configuration(org.apache.hadoop.conf.Configuration) InputStream(java.io.InputStream) FileSystem(org.apache.hadoop.fs.FileSystem) HOCONInputStreamFlowTemplate(org.apache.gobblin.service.modules.template.HOCONInputStreamFlowTemplate)

Aggregations

InputStream (java.io.InputStream)1 HOCONInputStreamFlowTemplate (org.apache.gobblin.service.modules.template.HOCONInputStreamFlowTemplate)1 Configuration (org.apache.hadoop.conf.Configuration)1 FileSystem (org.apache.hadoop.fs.FileSystem)1 Path (org.apache.hadoop.fs.Path)1