Search in sources :

Example 6 with AngularObject

use of org.apache.zeppelin.display.AngularObject in project zeppelin by apache.

the class NotebookServer method removeAngularFromRemoteRegistry.

private AngularObject removeAngularFromRemoteRegistry(String noteId, String paragraphId, String varName, RemoteAngularObjectRegistry remoteRegistry, String interpreterGroupId, NotebookSocket conn) {
    final AngularObject ao = remoteRegistry.removeAndNotifyRemoteProcess(varName, noteId, paragraphId);
    connectionManager.broadcastExcept(noteId, new Message(OP.ANGULAR_OBJECT_REMOVE).put("angularObject", ao).put("interpreterGroupId", interpreterGroupId).put("noteId", noteId).put("paragraphId", paragraphId), conn);
    return ao;
}
Also used : OnMessage(javax.websocket.OnMessage) ClusterMessage(org.apache.zeppelin.cluster.event.ClusterMessage) Message(org.apache.zeppelin.common.Message) AngularObject(org.apache.zeppelin.display.AngularObject)

Example 7 with AngularObject

use of org.apache.zeppelin.display.AngularObject in project zeppelin by apache.

the class NotebookServer method angularObjectClientUnbind.

/**
 * 1. Remove the given Angular variable to the target interpreter(s) angular
 * registry given a noteId and an optional list of paragraph id(s).
 * 2. Delete AngularObject from note.
 */
protected void angularObjectClientUnbind(NotebookSocket conn, Message fromMessage) throws Exception {
    String noteId = fromMessage.getType("noteId");
    String varName = fromMessage.getType("name");
    String paragraphId = fromMessage.getType("paragraphId");
    if (paragraphId == null) {
        throw new IllegalArgumentException("target paragraph not specified for " + "angular value unBind");
    }
    getNotebook().processNote(noteId, note -> {
        if (note != null) {
            InterpreterGroup interpreterGroup;
            try {
                interpreterGroup = findInterpreterGroupForParagraph(note, paragraphId);
            } catch (Exception e) {
                LOG.error("No interpreter group found for noteId {} and paragraphId {}", noteId, paragraphId, e);
                return null;
            }
            final RemoteAngularObjectRegistry registry = (RemoteAngularObjectRegistry) interpreterGroup.getAngularObjectRegistry();
            AngularObject ao = removeAngularFromRemoteRegistry(noteId, paragraphId, varName, registry, interpreterGroup.getId(), conn);
            note.deleteAngularObject(interpreterGroup.getId(), noteId, paragraphId, varName);
        }
        return null;
    });
}
Also used : InterpreterGroup(org.apache.zeppelin.interpreter.InterpreterGroup) RemoteAngularObjectRegistry(org.apache.zeppelin.interpreter.remote.RemoteAngularObjectRegistry) AngularObject(org.apache.zeppelin.display.AngularObject) URISyntaxException(java.net.URISyntaxException) ForbiddenException(org.apache.zeppelin.rest.exception.ForbiddenException) TException(org.apache.thrift.TException) IOException(java.io.IOException) UnknownHostException(java.net.UnknownHostException) ServiceException(org.apache.zeppelin.interpreter.thrift.ServiceException)

Example 8 with AngularObject

use of org.apache.zeppelin.display.AngularObject in project zeppelin by apache.

the class RemoteInterpreterEventServer method updateAngularObject.

@Override
public void updateAngularObject(String intpGroupId, String json) throws InterpreterRPCException, TException {
    AngularObject<?> angularObject = AngularObject.fromJson(json);
    InterpreterGroup interpreterGroup = interpreterSettingManager.getInterpreterGroupById(intpGroupId);
    if (interpreterGroup == null) {
        throw new InterpreterRPCException("Invalid InterpreterGroupId: " + intpGroupId);
    }
    AngularObject localAngularObject = interpreterGroup.getAngularObjectRegistry().get(angularObject.getName(), angularObject.getNoteId(), angularObject.getParagraphId());
    if (localAngularObject instanceof RemoteAngularObject) {
        // to avoid ping-pong loop
        ((RemoteAngularObject) localAngularObject).set(angularObject.get(), true, false);
    } else {
        localAngularObject.set(angularObject.get());
    }
    if (angularObject.getNoteId() != null) {
        try {
            interpreterSettingManager.getNotebook().processNote(angularObject.getNoteId(), note -> {
                if (note != null) {
                    note.addOrUpdateAngularObject(intpGroupId, angularObject);
                    interpreterSettingManager.getNotebook().saveNote(note, AuthenticationInfo.ANONYMOUS);
                }
                return null;
            });
        } catch (IOException e) {
            LOGGER.error("Fail to get note: {}", angularObject.getNoteId());
        }
    }
}
Also used : InterpreterRPCException(org.apache.zeppelin.interpreter.thrift.InterpreterRPCException) RemoteAngularObject(org.apache.zeppelin.interpreter.remote.RemoteAngularObject) AngularObject(org.apache.zeppelin.display.AngularObject) RemoteAngularObject(org.apache.zeppelin.interpreter.remote.RemoteAngularObject) IOException(java.io.IOException)

Example 9 with AngularObject

use of org.apache.zeppelin.display.AngularObject in project zeppelin by apache.

the class NotebookServerTest method testAngularObjectSaveToNote.

@Test
public void testAngularObjectSaveToNote() throws IOException, InterruptedException {
    // create a notebook
    String note1Id = null;
    try {
        note1Id = notebook.createNote("note1", "angular", anonymous);
        // get reference to interpreterGroup
        InterpreterGroup interpreterGroup = null;
        List<InterpreterSetting> settings = notebook.processNote(note1Id, note1 -> note1.getBindedInterpreterSettings(new ArrayList<>()));
        for (InterpreterSetting setting : settings) {
            if (setting.getName().equals("angular")) {
                interpreterGroup = setting.getOrCreateInterpreterGroup("anonymous", note1Id);
                break;
            }
        }
        String p1Id = notebook.processNote(note1Id, note1 -> {
            // start interpreter process
            Paragraph p1 = note1.addNewParagraph(AuthenticationInfo.ANONYMOUS);
            p1.setText("%angular <h2>Bind here : {{COMMAND_TYPE}}</h2>");
            p1.setAuthenticationInfo(anonymous);
            note1.run(p1.getId());
            return p1.getId();
        });
        // wait for paragraph finished
        Status status = notebook.processNote(note1Id, note1 -> note1.getParagraph(p1Id).getStatus());
        while (true) {
            if (status == Job.Status.FINISHED) {
                break;
            }
            Thread.sleep(100);
            status = notebook.processNote(note1Id, note1 -> note1.getParagraph(p1Id).getStatus());
        }
        // sleep for 1 second to make sure job running thread finish to fire event. See ZEPPELIN-3277
        Thread.sleep(1000);
        // create two sockets and open it
        NotebookSocket sock1 = createWebSocket();
        notebookServer.onOpen(sock1);
        // getNote, getAngularObject
        verify(sock1, times(0)).send(anyString());
        // open the same notebook from sockets
        notebookServer.onMessage(sock1, new Message(OP.GET_NOTE).put("id", note1Id).toJson());
        reset(sock1);
        // bind object from sock1
        notebookServer.onMessage(sock1, new Message(OP.ANGULAR_OBJECT_CLIENT_BIND).put("noteId", note1Id).put("paragraphId", p1Id).put("name", "COMMAND_TYPE").put("value", "COMMAND_TYPE_VALUE").put("interpreterGroupId", interpreterGroup.getId()).toJson());
        List<AngularObject> list = notebook.processNote(note1Id, note1 -> note1.getAngularObjects("angular-shared_process"));
        assertEquals(1, list.size());
        assertEquals(note1Id, list.get(0).getNoteId());
        assertEquals(p1Id, list.get(0).getParagraphId());
        assertEquals("COMMAND_TYPE", list.get(0).getName());
        assertEquals("COMMAND_TYPE_VALUE", list.get(0).get());
        // Check if the interpreterGroup AngularObjectRegistry is updated
        Map<String, Map<String, AngularObject>> mapRegistry = interpreterGroup.getAngularObjectRegistry().getRegistry();
        AngularObject ao = mapRegistry.get(note1Id + "_" + p1Id).get("COMMAND_TYPE");
        assertEquals("COMMAND_TYPE", ao.getName());
        assertEquals("COMMAND_TYPE_VALUE", ao.get());
        // update bind object from sock1
        notebookServer.onMessage(sock1, new Message(OP.ANGULAR_OBJECT_UPDATED).put("noteId", note1Id).put("paragraphId", p1Id).put("name", "COMMAND_TYPE").put("value", "COMMAND_TYPE_VALUE_UPDATE").put("interpreterGroupId", interpreterGroup.getId()).toJson());
        list = notebook.processNote(note1Id, note1 -> note1.getAngularObjects("angular-shared_process"));
        assertEquals(1, list.size());
        assertEquals(note1Id, list.get(0).getNoteId());
        assertEquals(p1Id, list.get(0).getParagraphId());
        assertEquals("COMMAND_TYPE", list.get(0).getName());
        assertEquals("COMMAND_TYPE_VALUE_UPDATE", list.get(0).get());
        // Check if the interpreterGroup AngularObjectRegistry is updated
        mapRegistry = interpreterGroup.getAngularObjectRegistry().getRegistry();
        AngularObject ao1 = mapRegistry.get(note1Id + "_" + p1Id).get("COMMAND_TYPE");
        assertEquals("COMMAND_TYPE", ao1.getName());
        assertEquals("COMMAND_TYPE_VALUE_UPDATE", ao1.get());
        // unbind object from sock1
        notebookServer.onMessage(sock1, new Message(OP.ANGULAR_OBJECT_CLIENT_UNBIND).put("noteId", note1Id).put("paragraphId", p1Id).put("name", "COMMAND_TYPE").put("value", "COMMAND_TYPE_VALUE").put("interpreterGroupId", interpreterGroup.getId()).toJson());
        list = notebook.processNote(note1Id, note1 -> note1.getAngularObjects("angular-shared_process"));
        assertEquals(0, list.size());
        // Check if the interpreterGroup AngularObjectRegistry is delete
        mapRegistry = interpreterGroup.getAngularObjectRegistry().getRegistry();
        AngularObject ao2 = mapRegistry.get(note1Id + "_" + p1Id).get("COMMAND_TYPE");
        assertNull(ao2);
    } finally {
        if (note1Id != null) {
            notebook.removeNote(note1Id, anonymous);
        }
    }
}
Also used : Status(org.apache.zeppelin.scheduler.Job.Status) ZeppelinConfiguration(org.apache.zeppelin.conf.ZeppelinConfiguration) TestUtils(org.apache.zeppelin.utils.TestUtils) Arrays(java.util.Arrays) AuthenticationInfo(org.apache.zeppelin.user.AuthenticationInfo) InetAddress(java.net.InetAddress) ServiceContext(org.apache.zeppelin.service.ServiceContext) ParagraphInfo(org.apache.zeppelin.interpreter.thrift.ParagraphInfo) Arrays.asList(java.util.Arrays.asList) Matchers.eq(org.mockito.Matchers.eq) Map(java.util.Map) Job(org.apache.zeppelin.scheduler.Job) InterpreterSetting(org.apache.zeppelin.interpreter.InterpreterSetting) AuthorizationService(org.apache.zeppelin.notebook.AuthorizationService) NotebookRepoWithVersionControl(org.apache.zeppelin.notebook.repo.NotebookRepoWithVersionControl) AngularObjectBuilder(org.apache.zeppelin.display.AngularObjectBuilder) AngularObject(org.apache.zeppelin.display.AngularObject) AfterClass(org.junit.AfterClass) NoteProcessor(org.apache.zeppelin.notebook.Notebook.NoteProcessor) NoteInfo(org.apache.zeppelin.notebook.NoteInfo) StandardCharsets(java.nio.charset.StandardCharsets) IOUtils(org.apache.commons.io.IOUtils) RemoteAngularObjectRegistry(org.apache.zeppelin.interpreter.remote.RemoteAngularObjectRegistry) List(java.util.List) Assert.assertFalse(org.junit.Assert.assertFalse) Status(org.apache.zeppelin.scheduler.Job.Status) Mockito.mock(org.mockito.Mockito.mock) BeforeClass(org.junit.BeforeClass) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) HttpServletRequest(javax.servlet.http.HttpServletRequest) RETURNS_DEEP_STUBS(org.mockito.Mockito.RETURNS_DEEP_STUBS) Message(org.apache.zeppelin.common.Message) Mockito.anyString(org.mockito.Mockito.anyString) OP(org.apache.zeppelin.common.Message.OP) InterpreterGroup(org.apache.zeppelin.interpreter.InterpreterGroup) Before(org.junit.Before) Paragraph(org.apache.zeppelin.notebook.Paragraph) NotebookService(org.apache.zeppelin.service.NotebookService) Assert.assertNotNull(org.junit.Assert.assertNotNull) Note(org.apache.zeppelin.notebook.Note) Assert.assertTrue(org.junit.Assert.assertTrue) TException(org.apache.thrift.TException) Mockito.times(org.mockito.Mockito.times) IOException(java.io.IOException) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) UnknownHostException(java.net.UnknownHostException) Notebook(org.apache.zeppelin.notebook.Notebook) Assert.assertNotEquals(org.junit.Assert.assertNotEquals) ServiceException(org.apache.zeppelin.interpreter.thrift.ServiceException) AbstractTestRestApi(org.apache.zeppelin.rest.AbstractTestRestApi) Mockito.verify(org.mockito.Mockito.verify) Mockito(org.mockito.Mockito) Mockito.never(org.mockito.Mockito.never) Assert.assertNull(org.junit.Assert.assertNull) Mockito.reset(org.mockito.Mockito.reset) Assert.assertEquals(org.junit.Assert.assertEquals) Message(org.apache.zeppelin.common.Message) InterpreterSetting(org.apache.zeppelin.interpreter.InterpreterSetting) ArrayList(java.util.ArrayList) AngularObject(org.apache.zeppelin.display.AngularObject) Mockito.anyString(org.mockito.Mockito.anyString) Paragraph(org.apache.zeppelin.notebook.Paragraph) InterpreterGroup(org.apache.zeppelin.interpreter.InterpreterGroup) Map(java.util.Map) Test(org.junit.Test)

Example 10 with AngularObject

use of org.apache.zeppelin.display.AngularObject in project zeppelin by apache.

the class NotebookServerTest method testLoadAngularObjectFromNote.

@Test
public void testLoadAngularObjectFromNote() throws IOException, InterruptedException {
    // create a notebook
    String note1Id = null;
    try {
        note1Id = notebook.createNote("note1", anonymous);
        // get reference to interpreterGroup
        InterpreterGroup interpreterGroup = null;
        List<InterpreterSetting> settings = notebook.getInterpreterSettingManager().get();
        for (InterpreterSetting setting : settings) {
            if (setting.getName().equals("angular")) {
                interpreterGroup = setting.getOrCreateInterpreterGroup("anonymous", note1Id);
                break;
            }
        }
        String p1Id = notebook.processNote(note1Id, note1 -> {
            // start interpreter process
            Paragraph p1 = note1.addNewParagraph(AuthenticationInfo.ANONYMOUS);
            p1.setText("%angular <h2>Bind here : {{COMMAND_TYPE}}</h2>");
            p1.setAuthenticationInfo(anonymous);
            note1.run(p1.getId());
            return p1.getId();
        });
        // wait for paragraph finished
        Status status = notebook.processNote(note1Id, note1 -> note1.getParagraph(p1Id).getStatus());
        while (true) {
            System.out.println("loop");
            if (status == Job.Status.FINISHED) {
                break;
            }
            Thread.sleep(100);
            status = notebook.processNote(note1Id, note1 -> note1.getParagraph(p1Id).getStatus());
        }
        // sleep for 1 second to make sure job running thread finish to fire event. See ZEPPELIN-3277
        Thread.sleep(1000);
        // set note AngularObject
        AngularObject ao = new AngularObject("COMMAND_TYPE", "COMMAND_TYPE_VALUE", note1Id, p1Id, null);
        notebook.processNote(note1Id, note1 -> {
            note1.addOrUpdateAngularObject("angular-shared_process", ao);
            return null;
        });
        // create sockets and open it
        NotebookSocket sock1 = createWebSocket();
        notebookServer.onOpen(sock1);
        // Check the AngularObjectRegistry of the interpreterGroup before executing GET_NOTE
        Map<String, Map<String, AngularObject>> mapRegistry1 = interpreterGroup.getAngularObjectRegistry().getRegistry();
        assertEquals(0, mapRegistry1.size());
        // open the notebook from sockets, AngularObjectRegistry that triggers the update of the interpreterGroup
        notebookServer.onMessage(sock1, new Message(OP.GET_NOTE).put("id", note1Id).toJson());
        Thread.sleep(1000);
        // After executing GET_NOTE, check the AngularObjectRegistry of the interpreterGroup
        Map<String, Map<String, AngularObject>> mapRegistry2 = interpreterGroup.getAngularObjectRegistry().getRegistry();
        assertEquals(2, mapRegistry1.size());
        AngularObject ao1 = mapRegistry2.get(note1Id + "_" + p1Id).get("COMMAND_TYPE");
        assertEquals("COMMAND_TYPE", ao1.getName());
        assertEquals("COMMAND_TYPE_VALUE", ao1.get());
    } finally {
        if (note1Id != null) {
            notebook.removeNote(note1Id, anonymous);
        }
    }
}
Also used : Status(org.apache.zeppelin.scheduler.Job.Status) ZeppelinConfiguration(org.apache.zeppelin.conf.ZeppelinConfiguration) TestUtils(org.apache.zeppelin.utils.TestUtils) Arrays(java.util.Arrays) AuthenticationInfo(org.apache.zeppelin.user.AuthenticationInfo) InetAddress(java.net.InetAddress) ServiceContext(org.apache.zeppelin.service.ServiceContext) ParagraphInfo(org.apache.zeppelin.interpreter.thrift.ParagraphInfo) Arrays.asList(java.util.Arrays.asList) Matchers.eq(org.mockito.Matchers.eq) Map(java.util.Map) Job(org.apache.zeppelin.scheduler.Job) InterpreterSetting(org.apache.zeppelin.interpreter.InterpreterSetting) AuthorizationService(org.apache.zeppelin.notebook.AuthorizationService) NotebookRepoWithVersionControl(org.apache.zeppelin.notebook.repo.NotebookRepoWithVersionControl) AngularObjectBuilder(org.apache.zeppelin.display.AngularObjectBuilder) AngularObject(org.apache.zeppelin.display.AngularObject) AfterClass(org.junit.AfterClass) NoteProcessor(org.apache.zeppelin.notebook.Notebook.NoteProcessor) NoteInfo(org.apache.zeppelin.notebook.NoteInfo) StandardCharsets(java.nio.charset.StandardCharsets) IOUtils(org.apache.commons.io.IOUtils) RemoteAngularObjectRegistry(org.apache.zeppelin.interpreter.remote.RemoteAngularObjectRegistry) List(java.util.List) Assert.assertFalse(org.junit.Assert.assertFalse) Status(org.apache.zeppelin.scheduler.Job.Status) Mockito.mock(org.mockito.Mockito.mock) BeforeClass(org.junit.BeforeClass) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) HttpServletRequest(javax.servlet.http.HttpServletRequest) RETURNS_DEEP_STUBS(org.mockito.Mockito.RETURNS_DEEP_STUBS) Message(org.apache.zeppelin.common.Message) Mockito.anyString(org.mockito.Mockito.anyString) OP(org.apache.zeppelin.common.Message.OP) InterpreterGroup(org.apache.zeppelin.interpreter.InterpreterGroup) Before(org.junit.Before) Paragraph(org.apache.zeppelin.notebook.Paragraph) NotebookService(org.apache.zeppelin.service.NotebookService) Assert.assertNotNull(org.junit.Assert.assertNotNull) Note(org.apache.zeppelin.notebook.Note) Assert.assertTrue(org.junit.Assert.assertTrue) TException(org.apache.thrift.TException) Mockito.times(org.mockito.Mockito.times) IOException(java.io.IOException) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) UnknownHostException(java.net.UnknownHostException) Notebook(org.apache.zeppelin.notebook.Notebook) Assert.assertNotEquals(org.junit.Assert.assertNotEquals) ServiceException(org.apache.zeppelin.interpreter.thrift.ServiceException) AbstractTestRestApi(org.apache.zeppelin.rest.AbstractTestRestApi) Mockito.verify(org.mockito.Mockito.verify) Mockito(org.mockito.Mockito) Mockito.never(org.mockito.Mockito.never) Assert.assertNull(org.junit.Assert.assertNull) Mockito.reset(org.mockito.Mockito.reset) Assert.assertEquals(org.junit.Assert.assertEquals) Message(org.apache.zeppelin.common.Message) InterpreterGroup(org.apache.zeppelin.interpreter.InterpreterGroup) InterpreterSetting(org.apache.zeppelin.interpreter.InterpreterSetting) AngularObject(org.apache.zeppelin.display.AngularObject) Mockito.anyString(org.mockito.Mockito.anyString) Map(java.util.Map) Paragraph(org.apache.zeppelin.notebook.Paragraph) Test(org.junit.Test)

Aggregations

AngularObject (org.apache.zeppelin.display.AngularObject)38 AngularObjectRegistry (org.apache.zeppelin.display.AngularObjectRegistry)21 IOException (java.io.IOException)12 Map (java.util.Map)11 TException (org.apache.thrift.TException)11 RemoteAngularObjectRegistry (org.apache.zeppelin.interpreter.remote.RemoteAngularObjectRegistry)10 List (java.util.List)8 InterpreterGroup (org.apache.zeppelin.interpreter.InterpreterGroup)8 InterpreterSetting (org.apache.zeppelin.interpreter.InterpreterSetting)8 UnknownHostException (java.net.UnknownHostException)7 HashMap (java.util.HashMap)7 Message (org.apache.zeppelin.common.Message)7 ServiceException (org.apache.zeppelin.interpreter.thrift.ServiceException)7 AuthenticationInfo (org.apache.zeppelin.user.AuthenticationInfo)7 ArrayList (java.util.ArrayList)6 HashSet (java.util.HashSet)6 ZeppelinConfiguration (org.apache.zeppelin.conf.ZeppelinConfiguration)5 Arrays (java.util.Arrays)4 LinkedList (java.util.LinkedList)4 OP (org.apache.zeppelin.common.Message.OP)4