Search in sources :

Example 1 with Client

use of org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService.Client in project zeppelin by apache.

the class RemoteInterpreterEventPoller method sendResourcePoolResponseGetAll.

private void sendResourcePoolResponseGetAll(ResourceSet resourceSet) {
    Client client = null;
    boolean broken = false;
    try {
        client = interpreterProcess.getClient();
        List<String> resourceList = new LinkedList<>();
        Gson gson = new Gson();
        for (Resource r : resourceSet) {
            resourceList.add(gson.toJson(r));
        }
        client.resourcePoolResponseGetAll(resourceList);
    } catch (Exception e) {
        logger.error(e.getMessage(), e);
        broken = true;
    } finally {
        if (client != null) {
            interpreterProcess.releaseClient(client, broken);
        }
    }
}
Also used : Resource(org.apache.zeppelin.resource.Resource) RemoteZeppelinServerResource(org.apache.zeppelin.interpreter.RemoteZeppelinServerResource) Gson(com.google.gson.Gson) Client(org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService.Client) LinkedList(java.util.LinkedList) TException(org.apache.thrift.TException) InvocationTargetException(java.lang.reflect.InvocationTargetException)

Example 2 with Client

use of org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService.Client in project zeppelin by apache.

the class RemoteInterpreterEventPoller method sendInvokeMethodResult.

public void sendInvokeMethodResult(InvokeResourceMethodEventMessage message, Object o) {
    Client client = null;
    boolean broken = false;
    try {
        client = interpreterProcess.getClient();
        Gson gson = new Gson();
        String invokeMessage = gson.toJson(message);
        ByteBuffer obj;
        if (o == null) {
            obj = ByteBuffer.allocate(0);
        } else {
            obj = Resource.serializeObject(o);
        }
        client.resourceResponseInvokeMethod(invokeMessage, obj);
    } catch (Exception e) {
        logger.error(e.getMessage(), e);
        broken = true;
    } finally {
        if (client != null) {
            interpreterProcess.releaseClient(client, broken);
        }
    }
}
Also used : Gson(com.google.gson.Gson) Client(org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService.Client) ByteBuffer(java.nio.ByteBuffer) TException(org.apache.thrift.TException) InvocationTargetException(java.lang.reflect.InvocationTargetException)

Example 3 with Client

use of org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService.Client in project zeppelin by apache.

the class RemoteInterpreterEventPoller method invokeResourceMethod.

private Object invokeResourceMethod(InvokeResourceMethodEventMessage message) {
    ResourceId resourceId = message.resourceId;
    InterpreterGroup intpGroup = InterpreterGroup.getByInterpreterGroupId(resourceId.getResourcePoolId());
    if (intpGroup == null) {
        return null;
    }
    RemoteInterpreterProcess remoteInterpreterProcess = intpGroup.getRemoteInterpreterProcess();
    if (remoteInterpreterProcess == null) {
        ResourcePool localPool = intpGroup.getResourcePool();
        if (localPool != null) {
            Resource res = localPool.get(resourceId.getName());
            if (res != null) {
                try {
                    return res.invokeMethod(message.methodName, message.getParamTypes(), message.params, message.returnResourceName);
                } catch (Exception e) {
                    logger.error(e.getMessage(), e);
                    return null;
                }
            } else {
                // object is null. can't invoke any method
                logger.error("Can't invoke method {} on null object", message.methodName);
                return null;
            }
        } else {
            logger.error("no resource pool");
            return null;
        }
    } else if (interpreterProcess.isRunning()) {
        Client client = null;
        boolean broken = false;
        try {
            client = remoteInterpreterProcess.getClient();
            ByteBuffer res = client.resourceInvokeMethod(resourceId.getNoteId(), resourceId.getParagraphId(), resourceId.getName(), gson.toJson(message));
            Object o = Resource.deserializeObject(res);
            return o;
        } catch (Exception e) {
            logger.error(e.getMessage(), e);
            broken = true;
        } finally {
            if (client != null) {
                intpGroup.getRemoteInterpreterProcess().releaseClient(client, broken);
            }
        }
    }
    return null;
}
Also used : ResourceId(org.apache.zeppelin.resource.ResourceId) InterpreterGroup(org.apache.zeppelin.interpreter.InterpreterGroup) Resource(org.apache.zeppelin.resource.Resource) RemoteZeppelinServerResource(org.apache.zeppelin.interpreter.RemoteZeppelinServerResource) ResourcePool(org.apache.zeppelin.resource.ResourcePool) AngularObject(org.apache.zeppelin.display.AngularObject) Client(org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService.Client) ByteBuffer(java.nio.ByteBuffer) TException(org.apache.thrift.TException) InvocationTargetException(java.lang.reflect.InvocationTargetException)

Example 4 with Client

use of org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService.Client in project zeppelin by apache.

the class RemoteInterpreterProcess method dereference.

public int dereference() {
    synchronized (referenceCount) {
        int r = referenceCount.decrementAndGet();
        if (r == 0) {
            logger.info("shutdown interpreter process");
            remoteInterpreterEventPoller.shutdown();
            // first try shutdown
            Client client = null;
            try {
                client = getClient();
                client.shutdown();
            } catch (Exception e) {
                // safely ignore exception while client.shutdown() may terminates remote process
                logger.info("Exception in RemoteInterpreterProcess while synchronized dereference, can " + "safely ignore exception while client.shutdown() may terminates remote process");
                logger.debug(e.getMessage(), e);
            } finally {
                if (client != null) {
                    // no longer used
                    releaseBrokenClient(client);
                }
            }
            clientPool.clear();
            clientPool.close();
            // wait for some time (connectTimeout) and force kill
            // remote process server.serve() loop is not always finishing gracefully
            long startTime = System.currentTimeMillis();
            while (System.currentTimeMillis() - startTime < connectTimeout) {
                if (this.isRunning()) {
                    try {
                        Thread.sleep(500);
                    } catch (InterruptedException e) {
                        logger.error("Exception in RemoteInterpreterProcess while synchronized dereference " + "Thread.sleep", e);
                    }
                } else {
                    break;
                }
            }
        }
        return r;
    }
}
Also used : Client(org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService.Client) TException(org.apache.thrift.TException)

Example 5 with Client

use of org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService.Client in project zeppelin by apache.

the class ClientFactory method create.

@Override
public Client create() throws Exception {
    TSocket transport = new TSocket(host, port);
    try {
        transport.open();
    } catch (TTransportException e) {
        throw new InterpreterException(e);
    }
    TProtocol protocol = new TBinaryProtocol(transport);
    Client client = new RemoteInterpreterService.Client(protocol);
    synchronized (clientSocketMap) {
        clientSocketMap.put(client, transport);
    }
    return client;
}
Also used : TBinaryProtocol(org.apache.thrift.protocol.TBinaryProtocol) TProtocol(org.apache.thrift.protocol.TProtocol) InterpreterException(org.apache.zeppelin.interpreter.InterpreterException) TTransportException(org.apache.thrift.transport.TTransportException) Client(org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService.Client) TSocket(org.apache.thrift.transport.TSocket)

Aggregations

Client (org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService.Client)22 TException (org.apache.thrift.TException)19 Gson (com.google.gson.Gson)8 InvocationTargetException (java.lang.reflect.InvocationTargetException)8 AngularObject (org.apache.zeppelin.display.AngularObject)6 ByteBuffer (java.nio.ByteBuffer)4 LinkedList (java.util.LinkedList)4 InterpreterGroup (org.apache.zeppelin.interpreter.InterpreterGroup)4 RemoteZeppelinServerResource (org.apache.zeppelin.interpreter.RemoteZeppelinServerResource)4 List (java.util.List)3 Map (java.util.Map)3 ResourcePool (org.apache.zeppelin.resource.ResourcePool)3 AngularObjectRegistry (org.apache.zeppelin.display.AngularObjectRegistry)2 InterpreterContextRunner (org.apache.zeppelin.interpreter.InterpreterContextRunner)2 Resource (org.apache.zeppelin.resource.Resource)2 ResourceId (org.apache.zeppelin.resource.ResourceId)2 ResourceSet (org.apache.zeppelin.resource.ResourceSet)2 Test (org.junit.Test)2 TypeToken (com.google.gson.reflect.TypeToken)1 HashMap (java.util.HashMap)1