Search in sources :

Example 11 with HostnamePort

use of org.neo4j.helpers.HostnamePort in project neo4j by neo4j.

the class ClusterJoin method joinByConfig.

private void joinByConfig() throws TimeoutException {
    List<HostnamePort> hosts = config.getInitialHosts();
    cluster.addClusterListener(new UnknownJoiningMemberWarning(hosts));
    if (hosts == null || hosts.size() == 0) {
        userLog.info("No cluster hosts specified. Creating cluster %s", config.getClusterName());
        cluster.create(config.getClusterName());
    } else {
        URI[] memberURIs = hosts.stream().map(member -> URI.create("cluster://" + resolvePortOnlyHost(member))).toArray(URI[]::new);
        while (true) {
            userLog.info("Attempting to join cluster of %s", hosts.toString());
            Future<ClusterConfiguration> clusterConfig = cluster.join(this.config.getClusterName(), memberURIs);
            try {
                ClusterConfiguration clusterConf = config.getClusterJoinTimeout() > 0 ? clusterConfig.get(config.getClusterJoinTimeout(), TimeUnit.MILLISECONDS) : clusterConfig.get();
                userLog.info("Joined cluster: %s", clusterConf);
                return;
            } catch (InterruptedException e) {
                userLog.warn("Could not join cluster, interrupted. Retrying...");
            } catch (ExecutionException e) {
                messagesLog.debug("Could not join cluster " + this.config.getClusterName());
                if (e.getCause() instanceof IllegalStateException) {
                    throw (IllegalStateException) e.getCause();
                }
                if (config.isAllowedToCreateCluster()) {
                    // Failed to join cluster, create new one
                    userLog.info("Could not join cluster of %s", hosts.toString());
                    userLog.info("Creating new cluster with name [%s]...", config.getClusterName());
                    cluster.create(config.getClusterName());
                    break;
                }
                userLog.warn("Could not join cluster, timed out. Retrying...");
            }
        }
    }
}
Also used : InstanceId(org.neo4j.cluster.InstanceId) ProtocolServer(org.neo4j.cluster.ProtocolServer) LifecycleAdapter(org.neo4j.kernel.lifecycle.LifecycleAdapter) Log(org.neo4j.logging.Log) Cluster(org.neo4j.cluster.protocol.cluster.Cluster) Semaphore(java.util.concurrent.Semaphore) TimeoutException(java.util.concurrent.TimeoutException) ClusterListener(org.neo4j.cluster.protocol.cluster.ClusterListener) LogService(org.neo4j.kernel.impl.logging.LogService) UnknownHostException(java.net.UnknownHostException) InetAddress(java.net.InetAddress) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) Future(java.util.concurrent.Future) ClusterConfiguration(org.neo4j.cluster.protocol.cluster.ClusterConfiguration) HostnamePort(org.neo4j.helpers.HostnamePort) URI(java.net.URI) HostnamePort(org.neo4j.helpers.HostnamePort) ClusterConfiguration(org.neo4j.cluster.protocol.cluster.ClusterConfiguration) ExecutionException(java.util.concurrent.ExecutionException) URI(java.net.URI)

Example 12 with HostnamePort

use of org.neo4j.helpers.HostnamePort in project neo4j by neo4j.

the class CertificatesIT method shouldUseConfiguredCertificate.

@Test
public void shouldUseConfiguredCertificate() throws Exception {
    // GIVEN
    SecureSocketConnection connection = new SecureSocketConnection();
    // WHEN
    connection.connect(new HostnamePort("localhost:7687")).send(TransportTestUtil.acceptedVersions(1, 0, 0, 0));
    // THEN
    Set<X509Certificate> certificatesSeen = connection.getServerCertificatesSeen();
    assertThat(certificatesSeen, contains(loadCertificateFromDisk()));
}
Also used : SecureSocketConnection(org.neo4j.bolt.v1.transport.socket.client.SecureSocketConnection) HostnamePort(org.neo4j.helpers.HostnamePort) X509Certificate(java.security.cert.X509Certificate) Test(org.junit.Test)

Example 13 with HostnamePort

use of org.neo4j.helpers.HostnamePort in project neo4j by neo4j.

the class SocketConnectionTest method shouldOnlyReadOnceIfAllBytesAreRead.

@Test
public void shouldOnlyReadOnceIfAllBytesAreRead() throws Exception {
    // GIVEN
    Socket socket = mock(Socket.class);
    InputStream stream = mock(InputStream.class);
    when(socket.getInputStream()).thenReturn(stream);
    when(stream.read(any(byte[].class), anyInt(), anyInt())).thenReturn(4);
    SocketConnection connection = new SocketConnection(socket);
    connection.connect(new HostnamePort("my.domain", 1234));
    // WHEN
    connection.recv(4);
    // THEN
    verify(stream, times(1)).read(any(byte[].class), anyInt(), anyInt());
}
Also used : InputStream(java.io.InputStream) HostnamePort(org.neo4j.helpers.HostnamePort) Socket(java.net.Socket) Test(org.junit.Test)

Example 14 with HostnamePort

use of org.neo4j.helpers.HostnamePort in project neo4j by neo4j.

the class SocketConnectionTest method shouldOnlyReadUntilAllBytesAreRead.

@Test
public void shouldOnlyReadUntilAllBytesAreRead() throws Exception {
    // GIVEN
    Socket socket = mock(Socket.class);
    InputStream stream = mock(InputStream.class);
    when(socket.getInputStream()).thenReturn(stream);
    when(stream.read(any(byte[].class), anyInt(), anyInt())).thenReturn(4).thenReturn(4).thenReturn(2).thenReturn(-1);
    SocketConnection connection = new SocketConnection(socket);
    connection.connect(new HostnamePort("my.domain", 1234));
    // WHEN
    connection.recv(10);
    // THEN
    verify(stream, times(3)).read(any(byte[].class), anyInt(), anyInt());
}
Also used : InputStream(java.io.InputStream) HostnamePort(org.neo4j.helpers.HostnamePort) Socket(java.net.Socket) Test(org.junit.Test)

Example 15 with HostnamePort

use of org.neo4j.helpers.HostnamePort in project neo4j by neo4j.

the class NetworkSenderReceiverTest method senderThatStartsAfterReceiverShouldEventuallyConnectSuccessfully.

@Test
public void senderThatStartsAfterReceiverShouldEventuallyConnectSuccessfully() throws Throwable {
    /*
         * This test verifies that a closed channel from a sender to a receiver is removed from the connections
         * mapping in the sender. It starts a sender, connects it to a receiver and sends a message.
         *
         * We should be testing this without resorting to using a NetworkReceiver. But, as prophets Mick Jagger and
         * Keith Richards mention in their scriptures, you can't always get what you want. In this case,
         * NetworkSender creates on its own the things required to communicate with the outside world, and this
         * means it creates actual sockets. To interact with it then, we need to setup listeners for those sockets
         * and respond properly. Hence, NetworkReceiver. Yes, this means that this test requires to open actual
         * network sockets.
         *
         * Read on for further hacks in place.
         */
    NetworkSender sender = null;
    NetworkReceiver receiver = null;
    try {
        LogProvider logProviderMock = mock(LogProvider.class);
        Log logMock = mock(Log.class);
        when(logProviderMock.getLog(Matchers.<Class>any())).thenReturn(logMock);
        final Semaphore sem = new Semaphore(0);
        /*
             * A semaphore AND a boolean? Weird, you may think, as the purpose is clearly to step through the
             * connection setup/teardown process. So, let's discuss what happens here more clearly.
             *
             * The sender and receiver are started. Trapped by the semaphore release on listeningAt()
             * The sender sends through the first message, it is received by the receiver. Trapped by the semaphore
             *      release on listeningAt() which is triggered on the first message receive on the receiver
             * The receiver is stopped, trapped by the overridden stop() method of the logging service.
             * The sender sends a message through, which will trigger the ChannelClosedException. This is where it
             *      gets tricky. See, normally, since we waited for the semaphore on NetworkReceiver.stop() and an
             *      happensBefore edge exists and all these good things, it should be certain that the Receiver is
             *      actually stopped and the message would fail to be sent. That would be too easy though. In reality,
             *      netty will not wait for all listening threads to stop before returning, so the receiver is not
             *      guaranteed to not be listening for incoming connections when stop() returns. This happens rarely,
             *      but the result is that the message "HelloWorld2" should fail with an exception (triggering the warn
             *      method on the logger) but it doesn't. So we can't block, but we must retry until we know the
             *      message failed to be sent and the exception happened, which is what this test is all about. We do
             *      that with a boolean that is tested upon continuously with sent messages until the error happens.
             *      Then we proceed with...
             * The receiver is started. Trapped by the listeningAt() callback.
             * The sender sends a message.
             * The receiver receives it, trapped by the dummy processor added to the receiver.
             */
        final AtomicBoolean senderChannelClosed = new AtomicBoolean(false);
        doAnswer(new Answer<Object>() {

            @Override
            public Object answer(InvocationOnMock invocation) throws Throwable {
                senderChannelClosed.set(true);
                return null;
            }
        }).when(logMock).warn(anyString());
        receiver = new NetworkReceiver(mock(NetworkReceiver.Monitor.class), new NetworkReceiver.Configuration() {

            @Override
            public HostnamePort clusterServer() {
                return new HostnamePort("127.0.0.1:1235");
            }

            @Override
            public int defaultPort() {
                return 5001;
            }

            @Override
            public String name() {
                return null;
            }
        }, NullLogProvider.getInstance()) {

            @Override
            public void stop() throws Throwable {
                super.stop();
                sem.release();
            }
        };
        sender = new NetworkSender(mock(NetworkSender.Monitor.class), new NetworkSender.Configuration() {

            @Override
            public int port() {
                return 1235;
            }

            @Override
            public int defaultPort() {
                return 5001;
            }
        }, receiver, logProviderMock);
        sender.init();
        sender.start();
        receiver.addNetworkChannelsListener(new NetworkReceiver.NetworkChannelsListener() {

            @Override
            public void listeningAt(URI me) {
                sem.release();
            }

            @Override
            public void channelOpened(URI to) {
            }

            @Override
            public void channelClosed(URI to) {
            }
        });
        final AtomicBoolean received = new AtomicBoolean(false);
        receiver.addMessageProcessor(new MessageProcessor() {

            @Override
            public boolean process(Message<? extends MessageType> message) {
                received.set(true);
                sem.release();
                return true;
            }
        });
        receiver.init();
        receiver.start();
        // wait for start from listeningAt() in the NetworkChannelsListener
        sem.acquire();
        sender.process(Message.to(TestMessage.helloWorld, URI.create("cluster://127.0.0.1:1235"), "Hello World"));
        // wait for process from the MessageProcessor
        sem.acquire();
        receiver.stop();
        // wait for overridden stop method in receiver
        sem.acquire();
        /*
             * This is the infernal loop of doom. We keep sending messages until one fails with a ClosedChannelException
             * which we have no better way to grab other than through the logger.warn() call which will occur.
             *
             * This code will hang if the warn we rely on is removed or if the receiver never stops - in general, if
             * the closed channel exception is not thrown. This is not an ideal failure mode but it's the best we can
             * do, given that NetworkSender is provided with very few things from its environment.
             */
        while (!senderChannelClosed.get()) {
            sender.process(Message.to(TestMessage.helloWorld, URI.create("cluster://127.0.0.1:1235"), "Hello World2"));
            /*
                 * This sleep is not necessary, it's just nice. If it's ommitted, everything will work, but we'll
                 * spam messages over the network as fast as possible. Even when the race between send and
                 * receiver.stop() does not occur, we will still send 3-4 messages through at full speed. If it
                 * does occur, then we are looking at hundreds. So we just back off a bit and let things work out.
                 */
            Thread.sleep(5);
        }
        receiver.start();
        // wait for receiver.listeningAt()
        sem.acquire();
        received.set(false);
        sender.process(Message.to(TestMessage.helloWorld, URI.create("cluster://127.0.0.1:1235"), "Hello World3"));
        // wait for receiver.process();
        sem.acquire();
        assertTrue(received.get());
    } finally {
        if (sender != null) {
            sender.stop();
            sender.shutdown();
        }
        if (receiver != null) {
            receiver.stop();
            receiver.shutdown();
        }
    }
}
Also used : Log(org.neo4j.logging.Log) HostnamePort(org.neo4j.helpers.HostnamePort) Semaphore(java.util.concurrent.Semaphore) URI(java.net.URI) NetworkReceiver(org.neo4j.cluster.com.NetworkReceiver) LogProvider(org.neo4j.logging.LogProvider) NullLogProvider(org.neo4j.logging.NullLogProvider) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) InvocationOnMock(org.mockito.invocation.InvocationOnMock) NetworkSender(org.neo4j.cluster.com.NetworkSender) Test(org.junit.Test)

Aggregations

HostnamePort (org.neo4j.helpers.HostnamePort)41 Test (org.junit.Test)29 File (java.io.File)9 Config (org.neo4j.kernel.configuration.Config)9 URI (java.net.URI)6 CoreMatchers.containsString (org.hamcrest.CoreMatchers.containsString)6 Cluster (org.neo4j.cluster.protocol.cluster.Cluster)6 PrintStream (java.io.PrintStream)5 InetSocketAddress (java.net.InetSocketAddress)5 ServerBootstrap (org.jboss.netty.bootstrap.ServerBootstrap)5 Matchers.anyString (org.mockito.Matchers.anyString)5 DefaultSlaveFactory (org.neo4j.kernel.ha.com.master.DefaultSlaveFactory)5 HashMap (java.util.HashMap)4 SocketConnection (org.neo4j.bolt.v1.transport.socket.client.SocketConnection)4 SlaveFactory (org.neo4j.kernel.ha.com.master.SlaveFactory)4 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)4 InputStream (java.io.InputStream)3 Socket (java.net.Socket)3 Channel (org.jboss.netty.channel.Channel)3 ChannelException (org.jboss.netty.channel.ChannelException)3