Search in sources :

Example 11 with MetaverseAnalyzerException

use of org.pentaho.metaverse.api.MetaverseAnalyzerException in project pentaho-metaverse by pentaho.

the class KettleAnalyzerUtil method getFileFromRepo.

private static TransMeta getFileFromRepo(TransMeta parentTransMeta, Repository repo, String dir, String file) throws MetaverseAnalyzerException {
    TransMeta subTransMeta;
    try {
        RepositoryDirectoryInterface rdi = repo.findDirectory(dir);
        subTransMeta = repo.loadTransformation(file, rdi, null, true, null);
    } catch (KettleException e) {
        throw new MetaverseAnalyzerException(Messages.getString("ERROR.SubTransNotFoundInParentTrans", file, parentTransMeta.toString()), e);
    }
    return subTransMeta;
}
Also used : RepositoryDirectoryInterface(org.pentaho.di.repository.RepositoryDirectoryInterface) KettleException(org.pentaho.di.core.exception.KettleException) MetaverseAnalyzerException(org.pentaho.metaverse.api.MetaverseAnalyzerException) TransMeta(org.pentaho.di.trans.TransMeta)

Example 12 with MetaverseAnalyzerException

use of org.pentaho.metaverse.api.MetaverseAnalyzerException in project pentaho-metaverse by pentaho.

the class StepAnalyzer method getStepFieldOriginDescriptor.

protected IComponentDescriptor getStepFieldOriginDescriptor(IComponentDescriptor descriptor, String fieldName) throws MetaverseAnalyzerException {
    if (descriptor == null || stepFields == null) {
        return null;
    }
    ValueMetaInterface vmi = stepFields.searchValueMeta(fieldName);
    String origin = (vmi == null) ? fieldName : vmi.getOrigin();
    // if we can't determine the origin, throw an exception
    if (origin == null && !ArrayUtils.isEmpty(prevStepNames)) {
        throw new MetaverseAnalyzerException(Messages.getString("ERROR.NoOriginForField", fieldName));
    }
    IMetaverseNode tmpOriginNode = metaverseObjectFactory.createNodeObject(UUID.randomUUID().toString(), origin, DictionaryConst.NODE_TYPE_TRANS_STEP);
    tmpOriginNode.setProperty(DictionaryConst.PROPERTY_NAMESPACE, rootNode.getProperty(DictionaryConst.PROPERTY_NAMESPACE));
    INamespace stepFieldNamespace = new Namespace(tmpOriginNode.getLogicalId());
    MetaverseComponentDescriptor d = new MetaverseComponentDescriptor(fieldName, DictionaryConst.NODE_TYPE_TRANS_FIELD, tmpOriginNode, descriptor.getContext());
    return d;
}
Also used : MetaverseAnalyzerException(org.pentaho.metaverse.api.MetaverseAnalyzerException) INamespace(org.pentaho.metaverse.api.INamespace) IMetaverseNode(org.pentaho.metaverse.api.IMetaverseNode) INamespace(org.pentaho.metaverse.api.INamespace) Namespace(org.pentaho.metaverse.api.Namespace) ValueMetaInterface(org.pentaho.di.core.row.ValueMetaInterface) MetaverseComponentDescriptor(org.pentaho.metaverse.api.MetaverseComponentDescriptor)

Example 13 with MetaverseAnalyzerException

use of org.pentaho.metaverse.api.MetaverseAnalyzerException in project pentaho-metaverse by pentaho.

the class KettleAnalyzerUtil method getSubTransByFilename.

private static TransMeta getSubTransByFilename(ISubTransAwareMeta meta, TransMeta parentTransMeta, Repository repo, TransMeta subTransMeta) throws MetaverseAnalyzerException {
    String transPath = parentTransMeta.environmentSubstitute(meta.getFileName());
    // if repository is not null, try it first
    if (repo != null) {
        String dir = transPath.substring(0, transPath.lastIndexOf('/'));
        String file = transPath.substring(transPath.lastIndexOf('/') + 1);
        subTransMeta = getFileFromRepo(parentTransMeta, repo, dir, file);
    }
    // couldn't find in repo or no repo present, look in file system
    if (null == subTransMeta) {
        try {
            subTransMeta = getSubTransMeta(normalizeFilePathSafely(transPath));
        } catch (Exception e) {
            throw new MetaverseAnalyzerException(Messages.getString("ERROR.SubTransNotFoundInParentTrans", transPath, parentTransMeta.toString()), e);
        }
    }
    return subTransMeta;
}
Also used : MetaverseAnalyzerException(org.pentaho.metaverse.api.MetaverseAnalyzerException) KettleException(org.pentaho.di.core.exception.KettleException) KettleXMLException(org.pentaho.di.core.exception.KettleXMLException) MetaverseException(org.pentaho.metaverse.api.MetaverseException) KettleMissingPluginsException(org.pentaho.di.core.exception.KettleMissingPluginsException) KettleFileException(org.pentaho.di.core.exception.KettleFileException) FileSystemException(org.apache.commons.vfs2.FileSystemException) FileNotFoundException(java.io.FileNotFoundException) MetaverseAnalyzerException(org.pentaho.metaverse.api.MetaverseAnalyzerException)

Example 14 with MetaverseAnalyzerException

use of org.pentaho.metaverse.api.MetaverseAnalyzerException in project pentaho-metaverse by pentaho.

the class JobAnalyzer method analyze.

@Override
public synchronized IMetaverseNode analyze(final IComponentDescriptor documentDescriptor, final AbstractMeta meta, final IMetaverseNode node, final String documentPath) throws MetaverseAnalyzerException {
    final JobMeta jobMeta = (JobMeta) meta;
    Job j = new Job(null, jobMeta);
    j.setInternalKettleVariables(jobMeta);
    // pull out the standard fields
    String description = jobMeta.getDescription();
    if (description != null) {
        node.setProperty(DictionaryConst.PROPERTY_DESCRIPTION, description);
    }
    String extendedDescription = jobMeta.getExtendedDescription();
    if (extendedDescription != null) {
        node.setProperty("extendedDescription", extendedDescription);
    }
    Date createdDate = jobMeta.getCreatedDate();
    if (createdDate != null) {
        node.setProperty(DictionaryConst.PROPERTY_CREATED, Long.toString(createdDate.getTime()));
    }
    String createdUser = jobMeta.getCreatedUser();
    if (createdUser != null) {
        node.setProperty(DictionaryConst.PROPERTY_CREATED_BY, createdUser);
    }
    Date lastModifiedDate = jobMeta.getModifiedDate();
    if (lastModifiedDate != null) {
        node.setProperty(DictionaryConst.PROPERTY_LAST_MODIFIED, Long.toString(lastModifiedDate.getTime()));
    }
    String lastModifiedUser = jobMeta.getModifiedUser();
    if (lastModifiedUser != null) {
        node.setProperty(DictionaryConst.PROPERTY_LAST_MODIFIED_BY, lastModifiedUser);
    }
    String version = jobMeta.getJobversion();
    if (version != null) {
        node.setProperty(DictionaryConst.PROPERTY_ARTIFACT_VERSION, version);
    }
    String status = Messages.getString("INFO.JobOrTrans.Status_" + Integer.toString(jobMeta.getJobstatus()));
    if (status != null && !status.startsWith("!")) {
        node.setProperty(DictionaryConst.PROPERTY_STATUS, status);
    }
    node.setProperty(DictionaryConst.PROPERTY_PATH, documentPath);
    // Process job parameters
    String[] parameters = jobMeta.listParameters();
    if (parameters != null) {
        for (String parameter : parameters) {
            try {
                // Determine parameter properties and add them to a map, then the map to the list
                String defaultParameterValue = jobMeta.getParameterDefault(parameter);
                String parameterValue = jobMeta.getParameterValue(parameter);
                String parameterDescription = jobMeta.getParameterDescription(parameter);
                PropertiesHolder paramProperties = new PropertiesHolder();
                paramProperties.setProperty("defaultValue", defaultParameterValue);
                paramProperties.setProperty("value", parameterValue);
                paramProperties.setProperty("description", parameterDescription);
                node.setProperty("parameter_" + parameter, paramProperties.toString());
            } catch (UnknownParamException upe) {
                // This shouldn't happen as we're using the list provided by the meta
                throw new MetaverseAnalyzerException(upe);
            }
        }
    }
    // handle the entries
    for (int i = 0; i < jobMeta.nrJobEntries(); i++) {
        JobEntryCopy entry = jobMeta.getJobEntry(i);
        try {
            if (entry != null) {
                entry.getEntry().setParentJob(j);
                IMetaverseNode jobEntryNode = null;
                JobEntryInterface jobEntryInterface = entry.getEntry();
                IComponentDescriptor entryDescriptor = new MetaverseComponentDescriptor(entry.getName(), DictionaryConst.NODE_TYPE_JOB_ENTRY, node, documentDescriptor.getContext());
                Set<IJobEntryAnalyzer> jobEntryAnalyzers = getJobEntryAnalyzers(jobEntryInterface);
                if (jobEntryAnalyzers != null && !jobEntryAnalyzers.isEmpty()) {
                    for (IJobEntryAnalyzer jobEntryAnalyzer : jobEntryAnalyzers) {
                        // change while the job is being analyzed
                        if (jobEntryAnalyzer instanceof IClonableJobEntryAnalyzer) {
                            jobEntryAnalyzer = ((IClonableJobEntryAnalyzer) jobEntryAnalyzer).cloneAnalyzer();
                            ((IClonableJobEntryAnalyzer) jobEntryAnalyzer).setDocumentAnalyzer(this);
                            ((IClonableJobEntryAnalyzer) jobEntryAnalyzer).setDocumentDescriptor(documentDescriptor);
                            ((IClonableJobEntryAnalyzer) jobEntryAnalyzer).setDocumentPath(documentPath);
                        } else {
                            log.debug(Messages.getString("WARNING.CannotCloneAnalyzer"), jobEntryAnalyzer);
                        }
                        jobEntryAnalyzer.setMetaverseBuilder(metaverseBuilder);
                        jobEntryNode = (IMetaverseNode) jobEntryAnalyzer.analyze(entryDescriptor, entry.getEntry());
                    }
                } else if (new AnnotatedClassFields(jobEntryInterface, jobEntryInterface.getParentJobMeta()).hasMetaverseAnnotations()) {
                    AnnotationDrivenJobAnalyzer annotationDrivenJobAnalyzer = new AnnotationDrivenJobAnalyzer(jobEntryInterface);
                    annotationDrivenJobAnalyzer.setMetaverseBuilder(metaverseBuilder);
                    annotationDrivenJobAnalyzer.setDocumentAnalyzer(this);
                    annotationDrivenJobAnalyzer.setDocumentDescriptor(documentDescriptor);
                    annotationDrivenJobAnalyzer.setDocumentPath(documentPath);
                    jobEntryNode = annotationDrivenJobAnalyzer.analyze(entryDescriptor, jobEntryInterface);
                } else {
                    GenericJobEntryMetaAnalyzer defaultJobEntryAnalyzer = new GenericJobEntryMetaAnalyzer();
                    defaultJobEntryAnalyzer.setMetaverseBuilder(metaverseBuilder);
                    jobEntryNode = defaultJobEntryAnalyzer.analyze(entryDescriptor, jobEntryInterface);
                }
                if (jobEntryNode != null) {
                    metaverseBuilder.addLink(node, DictionaryConst.LINK_CONTAINS, jobEntryNode);
                }
            }
        } catch (Exception mae) {
            // Don't throw an exception, just log and carry on
            log.warn(Messages.getString("ERROR.ErrorDuringAnalysis", entry.getName(), Const.NVL(mae.getLocalizedMessage(), "Unspecified")));
            log.debug(Messages.getString("ERROR.ErrorDuringAnalysisStackTrace"), mae);
        }
    }
    // Model the hops between steps
    int numHops = jobMeta.nrJobHops();
    for (int i = 0; i < numHops; i++) {
        JobHopMeta hop = jobMeta.getJobHop(i);
        JobEntryCopy fromEntry = hop.getFromEntry();
        JobEntryCopy toEntry = hop.getToEntry();
        INamespace childNs = new Namespace(node.getLogicalId());
        // process legitimate hops
        if (fromEntry != null && toEntry != null) {
            IMetaverseNode fromEntryNode = metaverseObjectFactory.createNodeObject(childNs, fromEntry.getName(), DictionaryConst.NODE_TYPE_JOB_ENTRY);
            IMetaverseNode toEntryNode = metaverseObjectFactory.createNodeObject(childNs, toEntry.getName(), DictionaryConst.NODE_TYPE_JOB_ENTRY);
            metaverseBuilder.addLink(fromEntryNode, DictionaryConst.LINK_HOPSTO, toEntryNode);
        }
    }
    metaverseBuilder.addNode(node);
    addParentLink(documentDescriptor, node);
    return node;
}
Also used : JobMeta(org.pentaho.di.job.JobMeta) PropertiesHolder(org.pentaho.metaverse.api.PropertiesHolder) JobHopMeta(org.pentaho.di.job.JobHopMeta) MetaverseAnalyzerException(org.pentaho.metaverse.api.MetaverseAnalyzerException) JobEntryInterface(org.pentaho.di.job.entry.JobEntryInterface) IMetaverseNode(org.pentaho.metaverse.api.IMetaverseNode) IJobEntryAnalyzer(org.pentaho.metaverse.api.analyzer.kettle.jobentry.IJobEntryAnalyzer) Date(java.util.Date) AnnotatedClassFields(org.pentaho.metaverse.api.analyzer.kettle.annotations.AnnotatedClassFields) KettleXMLException(org.pentaho.di.core.exception.KettleXMLException) UnknownParamException(org.pentaho.di.core.parameters.UnknownParamException) MetaverseAnalyzerException(org.pentaho.metaverse.api.MetaverseAnalyzerException) INamespace(org.pentaho.metaverse.api.INamespace) Namespace(org.pentaho.metaverse.api.Namespace) MetaverseComponentDescriptor(org.pentaho.metaverse.api.MetaverseComponentDescriptor) JobEntryCopy(org.pentaho.di.job.entry.JobEntryCopy) IComponentDescriptor(org.pentaho.metaverse.api.IComponentDescriptor) INamespace(org.pentaho.metaverse.api.INamespace) IClonableJobEntryAnalyzer(org.pentaho.metaverse.api.analyzer.kettle.jobentry.IClonableJobEntryAnalyzer) UnknownParamException(org.pentaho.di.core.parameters.UnknownParamException) GenericJobEntryMetaAnalyzer(org.pentaho.metaverse.analyzer.kettle.jobentry.GenericJobEntryMetaAnalyzer) Job(org.pentaho.di.job.Job) AnnotationDrivenJobAnalyzer(org.pentaho.metaverse.api.analyzer.kettle.annotations.AnnotationDrivenJobAnalyzer)

Example 15 with MetaverseAnalyzerException

use of org.pentaho.metaverse.api.MetaverseAnalyzerException in project pentaho-metaverse by pentaho.

the class MetaverseUtil method getAnalyzerRunner.

public static Runnable getAnalyzerRunner(final IDocumentAnalyzer analyzer, final IDocument document) {
    return new Runnable() {

        @Override
        public void run() {
            try {
                MetaverseComponentDescriptor docDescriptor = new MetaverseComponentDescriptor(document.getName(), DictionaryConst.NODE_TYPE_TRANS, document.getNamespace());
                if (document.getContext() != null) {
                    docDescriptor.setContext(document.getContext());
                }
                analyzer.analyze(docDescriptor, document);
            } catch (MetaverseAnalyzerException mae) {
                throw new RuntimeException(Messages.getString("ERROR.AnalyzingDocument", document.getNamespaceId()), mae);
            }
        }
    };
}
Also used : MetaverseAnalyzerException(org.pentaho.metaverse.api.MetaverseAnalyzerException) MetaverseComponentDescriptor(org.pentaho.metaverse.api.MetaverseComponentDescriptor)

Aggregations

MetaverseAnalyzerException (org.pentaho.metaverse.api.MetaverseAnalyzerException)23 IMetaverseNode (org.pentaho.metaverse.api.IMetaverseNode)13 IComponentDescriptor (org.pentaho.metaverse.api.IComponentDescriptor)9 MetaverseComponentDescriptor (org.pentaho.metaverse.api.MetaverseComponentDescriptor)9 KettleException (org.pentaho.di.core.exception.KettleException)7 KettleXMLException (org.pentaho.di.core.exception.KettleXMLException)6 TransMeta (org.pentaho.di.trans.TransMeta)6 JobMeta (org.pentaho.di.job.JobMeta)5 INamespace (org.pentaho.metaverse.api.INamespace)5 Namespace (org.pentaho.metaverse.api.Namespace)5 FileNotFoundException (java.io.FileNotFoundException)4 KettleMissingPluginsException (org.pentaho.di.core.exception.KettleMissingPluginsException)4 RowMetaInterface (org.pentaho.di.core.row.RowMetaInterface)4 Repository (org.pentaho.di.repository.Repository)4 RepositoryDirectoryInterface (org.pentaho.di.repository.RepositoryDirectoryInterface)4 BaseStepMeta (org.pentaho.di.trans.step.BaseStepMeta)4 HashSet (java.util.HashSet)3 ProgressNullMonitorListener (org.pentaho.di.core.ProgressNullMonitorListener)3 KettleStepException (org.pentaho.di.core.exception.KettleStepException)3 IDocument (org.pentaho.metaverse.api.IDocument)3