Search in sources :

Example 1 with Resource

use of org.apache.zeppelin.resource.Resource in project zeppelin by apache.

the class ZeppelinContext method containsKey.

/**
   * Check if resource pool has the object
   * @param name
   * @return
   */
@ZeppelinApi
public boolean containsKey(String name) {
    ResourcePool resourcePool = interpreterContext.getResourcePool();
    Resource resource = resourcePool.get(name);
    return resource != null;
}
Also used : Resource(org.apache.zeppelin.resource.Resource) ResourcePool(org.apache.zeppelin.resource.ResourcePool) ZeppelinApi(org.apache.zeppelin.annotation.ZeppelinApi)

Example 2 with Resource

use of org.apache.zeppelin.resource.Resource in project zeppelin by apache.

the class RemoteInterpreterEventPoller method sendResourcePoolResponseGetAll.

private void sendResourcePoolResponseGetAll(ResourceSet resourceSet) {
    Client client = null;
    boolean broken = false;
    try {
        client = interpreterProcess.getClient();
        List<String> resourceList = new LinkedList<>();
        Gson gson = new Gson();
        for (Resource r : resourceSet) {
            resourceList.add(gson.toJson(r));
        }
        client.resourcePoolResponseGetAll(resourceList);
    } catch (Exception e) {
        logger.error(e.getMessage(), e);
        broken = true;
    } finally {
        if (client != null) {
            interpreterProcess.releaseClient(client, broken);
        }
    }
}
Also used : Resource(org.apache.zeppelin.resource.Resource) RemoteZeppelinServerResource(org.apache.zeppelin.interpreter.RemoteZeppelinServerResource) Gson(com.google.gson.Gson) Client(org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService.Client) LinkedList(java.util.LinkedList) TException(org.apache.thrift.TException) InvocationTargetException(java.lang.reflect.InvocationTargetException)

Example 3 with Resource

use of org.apache.zeppelin.resource.Resource in project zeppelin by apache.

the class RemoteInterpreterEventPoller method invokeResourceMethod.

private Object invokeResourceMethod(InvokeResourceMethodEventMessage message) {
    ResourceId resourceId = message.resourceId;
    InterpreterGroup intpGroup = InterpreterGroup.getByInterpreterGroupId(resourceId.getResourcePoolId());
    if (intpGroup == null) {
        return null;
    }
    RemoteInterpreterProcess remoteInterpreterProcess = intpGroup.getRemoteInterpreterProcess();
    if (remoteInterpreterProcess == null) {
        ResourcePool localPool = intpGroup.getResourcePool();
        if (localPool != null) {
            Resource res = localPool.get(resourceId.getName());
            if (res != null) {
                try {
                    return res.invokeMethod(message.methodName, message.getParamTypes(), message.params, message.returnResourceName);
                } catch (Exception e) {
                    logger.error(e.getMessage(), e);
                    return null;
                }
            } else {
                // object is null. can't invoke any method
                logger.error("Can't invoke method {} on null object", message.methodName);
                return null;
            }
        } else {
            logger.error("no resource pool");
            return null;
        }
    } else if (interpreterProcess.isRunning()) {
        Client client = null;
        boolean broken = false;
        try {
            client = remoteInterpreterProcess.getClient();
            ByteBuffer res = client.resourceInvokeMethod(resourceId.getNoteId(), resourceId.getParagraphId(), resourceId.getName(), gson.toJson(message));
            Object o = Resource.deserializeObject(res);
            return o;
        } catch (Exception e) {
            logger.error(e.getMessage(), e);
            broken = true;
        } finally {
            if (client != null) {
                intpGroup.getRemoteInterpreterProcess().releaseClient(client, broken);
            }
        }
    }
    return null;
}
Also used : ResourceId(org.apache.zeppelin.resource.ResourceId) InterpreterGroup(org.apache.zeppelin.interpreter.InterpreterGroup) Resource(org.apache.zeppelin.resource.Resource) RemoteZeppelinServerResource(org.apache.zeppelin.interpreter.RemoteZeppelinServerResource) ResourcePool(org.apache.zeppelin.resource.ResourcePool) AngularObject(org.apache.zeppelin.display.AngularObject) Client(org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService.Client) ByteBuffer(java.nio.ByteBuffer) TException(org.apache.thrift.TException) InvocationTargetException(java.lang.reflect.InvocationTargetException)

Example 4 with Resource

use of org.apache.zeppelin.resource.Resource in project zeppelin by apache.

the class Neo4jConnectionManager method execute.

public List<Record> execute(String cypherQuery, InterpreterContext interpreterContext) {
    Map<String, Object> params = new HashMap<>();
    if (interpreterContext != null) {
        ResourcePool resourcePool = interpreterContext.getResourcePool();
        Set<String> keys = extractParams(cypherQuery, PROPERTY_PATTERN, REPLACE_CURLY_BRACKETS);
        keys.addAll(extractParams(cypherQuery, $_PATTERN, REPLACE_$));
        for (String key : keys) {
            Resource resource = resourcePool.get(key);
            if (resource != null) {
                params.put(key, resource.get());
            }
        }
    }
    LOGGER.debug("Executing cypher query {} with params {}", cypherQuery, params);
    try (Session session = getSession()) {
        final Result result = params.isEmpty() ? session.run(cypherQuery) : session.run(cypherQuery, params);
        return result.list();
    }
}
Also used : HashMap(java.util.HashMap) Resource(org.apache.zeppelin.resource.Resource) ResourcePool(org.apache.zeppelin.resource.ResourcePool) Session(org.neo4j.driver.Session) Result(org.neo4j.driver.Result)

Example 5 with Resource

use of org.apache.zeppelin.resource.Resource in project zeppelin by apache.

the class RemoteInterpreterEventServer method invokeResourceMethod.

private Object invokeResourceMethod(String intpGroupId, final InvokeResourceMethodEventMessage message) {
    final ResourceId resourceId = message.resourceId;
    ManagedInterpreterGroup intpGroup = interpreterSettingManager.getInterpreterGroupById(resourceId.getResourcePoolId());
    if (intpGroup == null) {
        return null;
    }
    RemoteInterpreterProcess remoteInterpreterProcess = intpGroup.getRemoteInterpreterProcess();
    if (remoteInterpreterProcess == null) {
        ResourcePool localPool = intpGroup.getResourcePool();
        if (localPool != null) {
            Resource res = localPool.get(resourceId.getName());
            if (res != null) {
                try {
                    return res.invokeMethod(message.methodName, message.getParamTypes(), message.params, message.returnResourceName);
                } catch (Exception e) {
                    LOGGER.error(e.getMessage(), e);
                    return null;
                }
            } else {
                // object is null. can't invoke any method
                LOGGER.error("Can't invoke method {} on null object", message.methodName);
                return null;
            }
        } else {
            LOGGER.error("no resource pool");
            return null;
        }
    } else if (remoteInterpreterProcess.isRunning()) {
        ByteBuffer res = remoteInterpreterProcess.callRemoteFunction(client -> client.resourceInvokeMethod(resourceId.getNoteId(), resourceId.getParagraphId(), resourceId.getName(), message.toJson()));
        try {
            return Resource.deserializeObject(res);
        } catch (Exception e) {
            LOGGER.error(e.getMessage(), e);
        }
        return null;
    }
    return null;
}
Also used : RegisterInfo(org.apache.zeppelin.interpreter.thrift.RegisterInfo) ZeppelinConfiguration(org.apache.zeppelin.conf.ZeppelinConfiguration) RemoteInterpreterEventService(org.apache.zeppelin.interpreter.thrift.RemoteInterpreterEventService) TypeToken(com.google.gson.reflect.TypeToken) RemoteInterpreterProcess(org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess) ScheduledFuture(java.util.concurrent.ScheduledFuture) AppendOutputRunner(org.apache.zeppelin.interpreter.remote.AppendOutputRunner) Resource(org.apache.zeppelin.resource.Resource) TTransportException(org.apache.thrift.transport.TTransportException) LoggerFactory(org.slf4j.LoggerFactory) ResourcePool(org.apache.zeppelin.resource.ResourcePool) AuthenticationInfo(org.apache.zeppelin.user.AuthenticationInfo) StringUtils(org.apache.commons.lang3.StringUtils) ByteBuffer(java.nio.ByteBuffer) LibraryMetadata(org.apache.zeppelin.interpreter.thrift.LibraryMetadata) ParagraphInfo(org.apache.zeppelin.interpreter.thrift.ParagraphInfo) Gson(com.google.gson.Gson) TThreadPoolServer(org.apache.thrift.server.TThreadPoolServer) Map(java.util.Map) AngularObject(org.apache.zeppelin.display.AngularObject) InterpreterRPCException(org.apache.zeppelin.interpreter.thrift.InterpreterRPCException) Collection(java.util.Collection) Executors(java.util.concurrent.Executors) List(java.util.List) OutputAppendEvent(org.apache.zeppelin.interpreter.thrift.OutputAppendEvent) WebUrlInfo(org.apache.zeppelin.interpreter.thrift.WebUrlInfo) OutputUpdateEvent(org.apache.zeppelin.interpreter.thrift.OutputUpdateEvent) RemoteInterpreterUtils(org.apache.zeppelin.interpreter.remote.RemoteInterpreterUtils) RemoteInterpreterProcessListener(org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcessListener) ArrayList(java.util.ArrayList) RunParagraphsEvent(org.apache.zeppelin.interpreter.thrift.RunParagraphsEvent) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) RemoteAngularObject(org.apache.zeppelin.interpreter.remote.RemoteAngularObject) LinkedList(java.util.LinkedList) RemoteInterpreterResultMessage(org.apache.zeppelin.interpreter.thrift.RemoteInterpreterResultMessage) ResourceSet(org.apache.zeppelin.resource.ResourceSet) OutputUpdateAllEvent(org.apache.zeppelin.interpreter.thrift.OutputUpdateAllEvent) Logger(org.slf4j.Logger) ApplicationEventListener(org.apache.zeppelin.helium.ApplicationEventListener) AppOutputAppendEvent(org.apache.zeppelin.interpreter.thrift.AppOutputAppendEvent) ResourceId(org.apache.zeppelin.resource.ResourceId) Note(org.apache.zeppelin.notebook.Note) TException(org.apache.thrift.TException) FileUtils(org.apache.commons.io.FileUtils) IOException(java.io.IOException) File(java.io.File) TimeUnit(java.util.concurrent.TimeUnit) TServerSocket(org.apache.thrift.transport.TServerSocket) RemoteResource(org.apache.zeppelin.resource.RemoteResource) AppStatusUpdateEvent(org.apache.zeppelin.interpreter.thrift.AppStatusUpdateEvent) InvokeResourceMethodEventMessage(org.apache.zeppelin.interpreter.remote.InvokeResourceMethodEventMessage) AppOutputUpdateEvent(org.apache.zeppelin.interpreter.thrift.AppOutputUpdateEvent) Collections(java.util.Collections) ResourceId(org.apache.zeppelin.resource.ResourceId) Resource(org.apache.zeppelin.resource.Resource) RemoteResource(org.apache.zeppelin.resource.RemoteResource) RemoteInterpreterProcess(org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess) ResourcePool(org.apache.zeppelin.resource.ResourcePool) ByteBuffer(java.nio.ByteBuffer) TTransportException(org.apache.thrift.transport.TTransportException) InterpreterRPCException(org.apache.zeppelin.interpreter.thrift.InterpreterRPCException) TException(org.apache.thrift.TException) IOException(java.io.IOException)

Aggregations

Resource (org.apache.zeppelin.resource.Resource)23 ResourcePool (org.apache.zeppelin.resource.ResourcePool)11 Gson (com.google.gson.Gson)6 IOException (java.io.IOException)6 LinkedList (java.util.LinkedList)6 ResourceSet (org.apache.zeppelin.resource.ResourceSet)6 ZeppelinApi (org.apache.zeppelin.annotation.ZeppelinApi)5 ByteBuffer (java.nio.ByteBuffer)4 TException (org.apache.thrift.TException)4 AngularObject (org.apache.zeppelin.display.AngularObject)3 TypeToken (com.google.gson.reflect.TypeToken)2 File (java.io.File)2 InvocationTargetException (java.lang.reflect.InvocationTargetException)2 ArrayList (java.util.ArrayList)2 Collections (java.util.Collections)2 HashMap (java.util.HashMap)2 List (java.util.List)2 Matcher (java.util.regex.Matcher)2 TTransportException (org.apache.thrift.transport.TTransportException)2 InterpreterResult (org.apache.zeppelin.interpreter.InterpreterResult)2