Search in sources :

Example 1 with MetaverseException

use of org.pentaho.metaverse.api.MetaverseException in project pentaho-metaverse by pentaho.

the class IntegrationTestUtil method initializePentahoSystem.

public static synchronized void initializePentahoSystem(String solutionPath) throws Exception {
    // this setting is useful only for running the integration tests from within IntelliJ
    // this same property is set for integration tests via the pom when running with mvn
    String folderPaths = "target/spoon/plugins";
    File f = new File(folderPaths);
    System.setProperty("KETTLE_PLUGIN_BASE_FOLDERS", f.getAbsolutePath());
    StandaloneApplicationContext appContext = new StandaloneApplicationContext(solutionPath, "");
    PentahoSystem.setSystemSettingsService(new PathBasedSystemSettings());
    if (solutionPath == null) {
        throw new MetaverseException(Messages.getString("ERROR.MetaverseInit.BadConfigPath", solutionPath));
    }
    try {
        ClassLoader cl = Thread.currentThread().getContextClassLoader();
        Thread.currentThread().setContextClassLoader(MetaverseUtil.class.getClassLoader());
        IPentahoObjectFactory pentahoObjectFactory = new StandaloneSpringPentahoObjectFactory();
        pentahoObjectFactory.init(solutionPath, PentahoSystem.getApplicationContext());
        PentahoSystem.registerObjectFactory(pentahoObjectFactory);
        // Restore context classloader
        Thread.currentThread().setContextClassLoader(cl);
    } catch (Exception e) {
        throw new MetaverseException(Messages.getString("ERROR.MetaverseInit.CouldNotInit"), e);
    }
    PentahoSystem.init(appContext);
    PentahoSessionHolder.setSession(new StandaloneSession());
    registerKettlePlugins();
    try {
        KettleEnvironment.init();
    } catch (KettleException e) {
        e.printStackTrace();
    }
}
Also used : KettleException(org.pentaho.di.core.exception.KettleException) MetaverseUtil(org.pentaho.metaverse.util.MetaverseUtil) IPentahoObjectFactory(org.pentaho.platform.api.engine.IPentahoObjectFactory) StandaloneSession(org.pentaho.platform.engine.core.system.StandaloneSession) StandaloneApplicationContext(org.pentaho.platform.engine.core.system.StandaloneApplicationContext) File(java.io.File) PathBasedSystemSettings(org.pentaho.platform.engine.core.system.PathBasedSystemSettings) MetaverseException(org.pentaho.metaverse.api.MetaverseException) KettleException(org.pentaho.di.core.exception.KettleException) MetaverseException(org.pentaho.metaverse.api.MetaverseException) KettlePluginException(org.pentaho.di.core.exception.KettlePluginException) StandaloneSpringPentahoObjectFactory(org.pentaho.platform.engine.core.system.objfac.StandaloneSpringPentahoObjectFactory)

Example 2 with MetaverseException

use of org.pentaho.metaverse.api.MetaverseException in project pentaho-metaverse by pentaho.

the class TransChangedExtensionPoint method updateLineage.

protected void updateLineage(Object object) {
    if (object != null && object instanceof TransMeta) {
        try {
            TransMeta transMeta = (TransMeta) object;
            TransExtensionPointUtil.addLineageGraph(transMeta);
        } catch (MetaverseException me) {
        // Nothing we can do here
        }
    }
}
Also used : TransMeta(org.pentaho.di.trans.TransMeta) MetaverseException(org.pentaho.metaverse.api.MetaverseException)

Example 3 with MetaverseException

use of org.pentaho.metaverse.api.MetaverseException in project pentaho-metaverse by pentaho.

the class TransExtensionPointUtil method addLineageGraph.

public static void addLineageGraph(final TransMeta transMeta) throws MetaverseException {
    if (transMeta == null) {
        throw new MetaverseException(Messages.getString("ERROR.Document.IsNull"));
    }
    // Get the "natural" filename (repo-based if in repository, filesystem-based otherwise)
    String filename = getFilename(transMeta);
    final Graph graph = new TinkerGraph();
    final IMetaverseBuilder metaverseBuilder = new MetaverseBuilder(graph);
    final IMetaverseObjectFactory objFactory = MetaverseUtil.getDocumentController().getMetaverseObjectFactory();
    // Add the client design node
    final String clientName = KettleClientEnvironment.getInstance().getClient().toString();
    final INamespace namespace = new Namespace(clientName);
    final IMetaverseNode designNode = objFactory.createNodeObject(clientName, clientName, DictionaryConst.NODE_TYPE_LOCATOR);
    metaverseBuilder.addNode(designNode);
    // Create a document object containing the transMeta
    final IDocument document = MetaverseUtil.createDocument(namespace, transMeta, filename, transMeta.getName(), "ktr", URLConnection.getFileNameMap().getContentTypeFor("trans.ktr"));
    MetaverseUtil.addLineageGraph(document, graph);
}
Also used : TinkerGraph(com.tinkerpop.blueprints.impls.tg.TinkerGraph) Graph(com.tinkerpop.blueprints.Graph) TinkerGraph(com.tinkerpop.blueprints.impls.tg.TinkerGraph) INamespace(org.pentaho.metaverse.api.INamespace) IMetaverseNode(org.pentaho.metaverse.api.IMetaverseNode) IMetaverseObjectFactory(org.pentaho.metaverse.api.IMetaverseObjectFactory) IMetaverseBuilder(org.pentaho.metaverse.api.IMetaverseBuilder) IMetaverseBuilder(org.pentaho.metaverse.api.IMetaverseBuilder) MetaverseBuilder(org.pentaho.metaverse.impl.MetaverseBuilder) MetaverseException(org.pentaho.metaverse.api.MetaverseException) INamespace(org.pentaho.metaverse.api.INamespace) Namespace(org.pentaho.metaverse.api.Namespace) IDocument(org.pentaho.metaverse.api.IDocument)

Example 4 with MetaverseException

use of org.pentaho.metaverse.api.MetaverseException in project pentaho-metaverse by pentaho.

the class TransformationRuntimeExtensionPoint method startAnalyzer.

protected void startAnalyzer(Trans trans) throws KettleException {
    if (trans == null) {
        return;
    }
    // Create and populate an execution profile with what we know so far
    ExecutionProfile executionProfile = new ExecutionProfile();
    populateExecutionProfile(executionProfile, trans);
    IMetaverseBuilder builder = TransLineageHolderMap.getInstance().getMetaverseBuilder(trans);
    // Analyze the current transformation
    if (documentAnalyzer != null) {
        documentAnalyzer.setMetaverseBuilder(builder);
        // Create a document for the Trans
        final String clientName = executionProfile.getExecutionEngine().getName();
        final INamespace namespace = new Namespace(clientName);
        final IMetaverseNode designNode = builder.getMetaverseObjectFactory().createNodeObject(clientName, clientName, DictionaryConst.NODE_TYPE_LOCATOR);
        builder.addNode(designNode);
        final TransMeta transMeta = trans.getTransMeta();
        String id = TransExtensionPointUtil.getFilename(transMeta);
        IDocument metaverseDocument = builder.getMetaverseObjectFactory().createDocumentObject();
        metaverseDocument.setNamespace(namespace);
        metaverseDocument.setContent(transMeta);
        metaverseDocument.setStringID(id);
        metaverseDocument.setName(transMeta.getName());
        metaverseDocument.setExtension("ktr");
        metaverseDocument.setMimeType(URLConnection.getFileNameMap().getContentTypeFor("trans.ktr"));
        metaverseDocument.setContext(new AnalysisContext(DictionaryConst.CONTEXT_RUNTIME));
        String normalizedPath;
        try {
            normalizedPath = KettleAnalyzerUtil.normalizeFilePath(id);
        } catch (MetaverseException e) {
            normalizedPath = id;
        }
        metaverseDocument.setProperty(DictionaryConst.PROPERTY_NAME, trans.getName());
        metaverseDocument.setProperty(DictionaryConst.PROPERTY_PATH, normalizedPath);
        metaverseDocument.setProperty(DictionaryConst.PROPERTY_NAMESPACE, namespace.getNamespaceId());
        Runnable analyzerRunner = MetaverseUtil.getAnalyzerRunner(documentAnalyzer, metaverseDocument);
        MetaverseCompletionService.getInstance().submit(analyzerRunner, id);
    }
    // Save the lineage objects for later
    LineageHolder holder = TransLineageHolderMap.getInstance().getLineageHolder(trans);
    holder.setExecutionProfile(executionProfile);
    holder.setMetaverseBuilder(builder);
}
Also used : INamespace(org.pentaho.metaverse.api.INamespace) IMetaverseNode(org.pentaho.metaverse.api.IMetaverseNode) TransMeta(org.pentaho.di.trans.TransMeta) AnalysisContext(org.pentaho.metaverse.api.AnalysisContext) IMetaverseBuilder(org.pentaho.metaverse.api.IMetaverseBuilder) ExecutionProfile(org.pentaho.metaverse.impl.model.ExecutionProfile) IExecutionProfile(org.pentaho.metaverse.api.model.IExecutionProfile) INamespace(org.pentaho.metaverse.api.INamespace) Namespace(org.pentaho.metaverse.api.Namespace) IDocument(org.pentaho.metaverse.api.IDocument) MetaverseException(org.pentaho.metaverse.api.MetaverseException) LineageHolder(org.pentaho.metaverse.api.model.LineageHolder)

Example 5 with MetaverseException

use of org.pentaho.metaverse.api.MetaverseException in project pentaho-metaverse by pentaho.

the class MetaverseUtil method addLineageGraph.

public static void addLineageGraph(final IDocument document, Graph graph) throws MetaverseException {
    if (document == null) {
        throw new MetaverseException(Messages.getString("ERROR.Document.IsNull"));
    }
    // Find the transformation analyzer(s) and create Futures to analyze the transformation.
    // Right now we expect a single transformation analyzer in the system. If we need to support more,
    // the lineageGraphMap needs to map the TransMeta to a collection of Futures, etc.
    IDocumentController docController = MetaverseUtil.getDocumentController();
    if (docController != null) {
        // Create a new builder, setting it on the DocumentController if possible
        IMetaverseBuilder metaverseBuilder = new MetaverseBuilder(graph);
        docController.setMetaverseBuilder(metaverseBuilder);
        List<IDocumentAnalyzer> matchingAnalyzers = docController.getDocumentAnalyzers("ktr");
        if (matchingAnalyzers != null) {
            for (final IDocumentAnalyzer analyzer : matchingAnalyzers) {
                Runnable analyzerRunner = getAnalyzerRunner(analyzer, document);
                Graph g = (graph != null) ? graph : new TinkerGraph();
                Future<Graph> transAnalysis = LineageGraphCompletionService.getInstance().submit(analyzerRunner, g);
                // Save this Future, the client will call it when the analysis is needed
                LineageGraphMap.getInstance().put(document.getContent(), transAnalysis);
            }
        }
    }
}
Also used : TinkerGraph(com.tinkerpop.blueprints.impls.tg.TinkerGraph) Graph(com.tinkerpop.blueprints.Graph) IDocumentController(org.pentaho.metaverse.api.IDocumentController) TinkerGraph(com.tinkerpop.blueprints.impls.tg.TinkerGraph) IDocumentAnalyzer(org.pentaho.metaverse.api.IDocumentAnalyzer) IMetaverseBuilder(org.pentaho.metaverse.api.IMetaverseBuilder) MetaverseBuilder(org.pentaho.metaverse.impl.MetaverseBuilder) IMetaverseBuilder(org.pentaho.metaverse.api.IMetaverseBuilder) MetaverseException(org.pentaho.metaverse.api.MetaverseException)

Aggregations

MetaverseException (org.pentaho.metaverse.api.MetaverseException)11 Graph (com.tinkerpop.blueprints.Graph)4 IMetaverseBuilder (org.pentaho.metaverse.api.IMetaverseBuilder)4 IMetaverseNode (org.pentaho.metaverse.api.IMetaverseNode)4 TransMeta (org.pentaho.di.trans.TransMeta)3 IDocument (org.pentaho.metaverse.api.IDocument)3 INamespace (org.pentaho.metaverse.api.INamespace)3 Namespace (org.pentaho.metaverse.api.Namespace)3 Vertex (com.tinkerpop.blueprints.Vertex)2 TinkerGraph (com.tinkerpop.blueprints.impls.tg.TinkerGraph)2 GremlinPipeline (com.tinkerpop.gremlin.java.GremlinPipeline)2 File (java.io.File)2 ArrayList (java.util.ArrayList)2 HashMap (java.util.HashMap)2 HashSet (java.util.HashSet)2 List (java.util.List)2 Set (java.util.Set)2 KettleException (org.pentaho.di.core.exception.KettleException)2 AnalysisContext (org.pentaho.metaverse.api.AnalysisContext)2 IExecutionProfile (org.pentaho.metaverse.api.model.IExecutionProfile)2