Search in sources :

Example 11 with StoreId

use of org.neo4j.kernel.impl.store.StoreId in project neo4j by neo4j.

the class Protocol method deserializeResponse.

public <PAYLOAD> Response<PAYLOAD> deserializeResponse(BlockingReadHandler<ChannelBuffer> reader, ByteBuffer input, long timeout, Deserializer<PAYLOAD> payloadDeserializer, ResourceReleaser channelReleaser, final LogEntryReader<ReadableClosablePositionAwareChannel> entryReader) throws IOException {
    final DechunkingChannelBuffer dechunkingBuffer = new DechunkingChannelBuffer(reader, timeout, internalProtocolVersion, applicationProtocolVersion);
    PAYLOAD response = payloadDeserializer.read(dechunkingBuffer, input);
    StoreId storeId = readStoreId(dechunkingBuffer, input);
    // Response type is what previously was a byte saying how many data sources there were in the
    // coming transaction stream response. For backwards compatibility we keep it as a byte and we introduce
    // the transaction obligation response type as -1
    byte responseType = dechunkingBuffer.readByte();
    if (responseType == TransactionObligationResponse.RESPONSE_TYPE) {
        // It is a transaction obligation response
        long obligationTxId = dechunkingBuffer.readLong();
        return new TransactionObligationResponse<>(response, storeId, obligationTxId, channelReleaser);
    }
    // It's a transaction stream in this response
    TransactionStream transactions = visitor -> {
        NetworkReadableClosableChannel channel = new NetworkReadableClosableChannel(dechunkingBuffer);
        try (PhysicalTransactionCursor<ReadableClosablePositionAwareChannel> cursor = new PhysicalTransactionCursor<>(channel, entryReader)) {
            while (cursor.next() && !visitor.visit(cursor.get())) {
            }
        }
    };
    return new TransactionStreamResponse<>(response, storeId, transactions, channelReleaser);
}
Also used : StoreId(org.neo4j.kernel.impl.store.StoreId) RecordFormat(org.neo4j.kernel.impl.store.format.RecordFormat) LengthFieldBasedFrameDecoder(org.jboss.netty.handler.codec.frame.LengthFieldBasedFrameDecoder) NeoStoreDataSource(org.neo4j.kernel.NeoStoreDataSource) TransactionRepresentation(org.neo4j.kernel.impl.transaction.TransactionRepresentation) ChannelBuffer(org.jboss.netty.buffer.ChannelBuffer) LogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader) BlockingReadHandler(org.jboss.netty.handler.queue.BlockingReadHandler) StoreWriter(org.neo4j.com.storecopy.StoreWriter) IOException(java.io.IOException) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) Channel(org.jboss.netty.channel.Channel) ByteBuffer(java.nio.ByteBuffer) StorageCommand(org.neo4j.storageengine.api.StorageCommand) ChannelPipeline(org.jboss.netty.channel.ChannelPipeline) List(java.util.List) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) LinkedList(java.util.LinkedList) PhysicalTransactionRepresentation(org.neo4j.kernel.impl.transaction.log.PhysicalTransactionRepresentation) PhysicalTransactionCursor(org.neo4j.kernel.impl.transaction.log.PhysicalTransactionCursor) LengthFieldPrepender(org.jboss.netty.handler.codec.frame.LengthFieldPrepender) PhysicalTransactionCursor(org.neo4j.kernel.impl.transaction.log.PhysicalTransactionCursor) StoreId(org.neo4j.kernel.impl.store.StoreId)

Example 12 with StoreId

use of org.neo4j.kernel.impl.store.StoreId in project neo4j by neo4j.

the class SwitchToSlave method switchToSlave.

/**
     * Performs a switch to the slave state. Starts the communication endpoints, switches components to the slave state
     * and ensures that the current database is appropriate for this cluster. It also broadcasts the appropriate
     * Slave Is Available event
     *
     * @param haCommunicationLife The LifeSupport instance to register the network facilities required for
     *                            communication with the rest of the cluster
     * @param me The URI this instance must bind to
     * @param masterUri The URI of the master for which this instance must become slave to
     * @param cancellationRequest A handle for gracefully aborting the switch
     * @return The URI that was broadcasted as the slave endpoint or null if the task was cancelled
     */
public URI switchToSlave(LifeSupport haCommunicationLife, URI me, URI masterUri, CancellationRequest cancellationRequest) throws Throwable {
    URI slaveUri;
    boolean success = false;
    monitor.switchToSlaveStarted();
    // Wait a short while for current transactions to stop first, just to be nice.
    // We can't wait forever since switching to our designated role is quite important.
    Clock clock = Clocks.systemClock();
    long deadline = clock.millis() + config.get(HaSettings.internal_state_switch_timeout);
    while (transactionCounters.getNumberOfActiveTransactions() > 0 && clock.millis() < deadline) {
        parkNanos(MILLISECONDS.toNanos(10));
    }
    try {
        InstanceId myId = config.get(ClusterSettings.server_id);
        userLog.info("ServerId %s, moving to slave for master %s", myId, masterUri);
        // since we are here it must already have been set from outside
        assert masterUri != null;
        idGeneratorFactory.switchToSlave();
        copyStoreFromMasterIfNeeded(masterUri, me, cancellationRequest);
        /*
             * The following check is mandatory, since the store copy can be cancelled and if it was actually
             * happening then we can't continue, as there is no store in place
             */
        if (cancellationRequest.cancellationRequested()) {
            msgLog.info("Switch to slave cancelled during store copy if no local store is present.");
            return null;
        }
        /*
             * We get here either with a fresh store from the master copy above so we need to
             * start the ds or we already had a store, so we have already started the ds. Either way,
             * make sure it's there.
             */
        NeoStoreDataSource neoDataSource = neoDataSourceSupplier.get();
        neoDataSource.afterModeSwitch();
        StoreId myStoreId = neoDataSource.getStoreId();
        boolean consistencyChecksExecutedSuccessfully = executeConsistencyChecks(transactionIdStoreSupplier.get(), masterUri, me, myStoreId, cancellationRequest);
        if (!consistencyChecksExecutedSuccessfully) {
            msgLog.info("Switch to slave cancelled due to consistency check failure.");
            return null;
        }
        if (cancellationRequest.cancellationRequested()) {
            msgLog.info("Switch to slave cancelled after consistency checks.");
            return null;
        }
        // no exception were thrown and we can proceed
        slaveUri = startHaCommunication(haCommunicationLife, neoDataSource, me, masterUri, myStoreId, cancellationRequest);
        if (slaveUri == null) {
            msgLog.info("Switch to slave unable to connect.");
            return null;
        }
        success = true;
        userLog.info("ServerId %s, successfully moved to slave for master %s", myId, masterUri);
    } finally {
        monitor.switchToSlaveCompleted(success);
    }
    return slaveUri;
}
Also used : StoreId(org.neo4j.kernel.impl.store.StoreId) InstanceId(org.neo4j.cluster.InstanceId) ClusterMembers.hasInstanceId(org.neo4j.kernel.ha.cluster.member.ClusterMembers.hasInstanceId) NeoStoreDataSource(org.neo4j.kernel.NeoStoreDataSource) Clock(java.time.Clock) URI(java.net.URI)

Example 13 with StoreId

use of org.neo4j.kernel.impl.store.StoreId in project neo4j by neo4j.

the class MasterClient214 method handshake.

@Override
public Response<HandshakeResult> handshake(final long txId, StoreId storeId) {
    Serializer serializer = buffer -> buffer.writeLong(txId);
    Deserializer<HandshakeResult> deserializer = (buffer, temporaryBuffer) -> new HandshakeResult(buffer.readLong(), buffer.readLong());
    return sendRequest(requestTypes.type(HaRequestTypes.Type.HANDSHAKE), RequestContext.EMPTY, serializer, deserializer, storeId, ResponseUnpacker.TxHandler.NO_OP_TX_HANDLER);
}
Also used : VOID_DESERIALIZER(org.neo4j.com.Protocol.VOID_DESERIALIZER) StoreId(org.neo4j.kernel.impl.store.StoreId) IdAllocation(org.neo4j.kernel.ha.id.IdAllocation) RequestType(org.neo4j.com.RequestType) LockResult(org.neo4j.kernel.ha.lock.LockResult) ObjectSerializer(org.neo4j.com.ObjectSerializer) ChannelBuffer(org.jboss.netty.buffer.ChannelBuffer) ResourceType(org.neo4j.storageengine.api.lock.ResourceType) Exceptions(org.neo4j.helpers.Exceptions) StoreWriter(org.neo4j.com.storecopy.StoreWriter) LogProvider(org.neo4j.logging.LogProvider) Serializer(org.neo4j.com.Serializer) HandshakeResult(org.neo4j.kernel.ha.com.master.HandshakeResult) IdRange(org.neo4j.kernel.impl.store.id.IdRange) RequestContext(org.neo4j.com.RequestContext) RequestMonitor(org.neo4j.com.monitor.RequestMonitor) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) INTERNAL_PROTOCOL_VERSION(org.neo4j.com.ProtocolVersion.INTERNAL_PROTOCOL_VERSION) MasterClient(org.neo4j.kernel.ha.com.slave.MasterClient) ByteCounterMonitor(org.neo4j.kernel.monitoring.ByteCounterMonitor) Client(org.neo4j.com.Client) EMPTY_SERIALIZER(org.neo4j.com.Protocol.EMPTY_SERIALIZER) Response(org.neo4j.com.Response) Protocol(org.neo4j.com.Protocol) TransactionRepresentation(org.neo4j.kernel.impl.transaction.TransactionRepresentation) LogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader) IOException(java.io.IOException) Protocol214(org.neo4j.com.Protocol214) MasterServer(org.neo4j.kernel.ha.com.master.MasterServer) LockStatus(org.neo4j.kernel.ha.lock.LockStatus) String.format(java.lang.String.format) IdType(org.neo4j.kernel.impl.store.id.IdType) ResponseUnpacker(org.neo4j.com.storecopy.ResponseUnpacker) Protocol.writeString(org.neo4j.com.Protocol.writeString) ProtocolVersion(org.neo4j.com.ProtocolVersion) Protocol.readString(org.neo4j.com.Protocol.readString) Deserializer(org.neo4j.com.Deserializer) Master(org.neo4j.kernel.ha.com.master.Master) HandshakeResult(org.neo4j.kernel.ha.com.master.HandshakeResult) ObjectSerializer(org.neo4j.com.ObjectSerializer) Serializer(org.neo4j.com.Serializer)

Example 14 with StoreId

use of org.neo4j.kernel.impl.store.StoreId in project neo4j by neo4j.

the class MasterClientTest method endLockSessionDoesNotUnpackResponse.

@Test
public void endLockSessionDoesNotUnpackResponse() throws Throwable {
    StoreId storeId = new StoreId(1, 2, 3, 4, 5);
    long txChecksum = 123;
    long lastAppliedTx = 5;
    ResponseUnpacker responseUnpacker = mock(ResponseUnpacker.class);
    MasterImpl.SPI masterImplSPI = MasterImplTest.mockedSpi(storeId);
    when(masterImplSPI.packTransactionObligationResponse(any(RequestContext.class), Matchers.anyObject())).thenReturn(Response.empty());
    when(masterImplSPI.getTransactionChecksum(anyLong())).thenReturn(txChecksum);
    newMasterServer(masterImplSPI);
    MasterClient client = newMasterClient320(storeId, responseUnpacker);
    HandshakeResult handshakeResult;
    try (Response<HandshakeResult> handshakeResponse = client.handshake(1, storeId)) {
        handshakeResult = handshakeResponse.response();
    }
    verify(responseUnpacker).unpackResponse(any(Response.class), any(TxHandler.class));
    reset(responseUnpacker);
    RequestContext context = new RequestContext(handshakeResult.epoch(), 1, 1, lastAppliedTx, txChecksum);
    client.endLockSession(context, false);
    verify(responseUnpacker, never()).unpackResponse(any(Response.class), any(TxHandler.class));
}
Also used : Response(org.neo4j.com.Response) TransactionStreamResponse(org.neo4j.com.TransactionStreamResponse) HandshakeResult(org.neo4j.kernel.ha.com.master.HandshakeResult) MasterImpl(org.neo4j.kernel.ha.com.master.MasterImpl) TxHandler(org.neo4j.com.storecopy.ResponseUnpacker.TxHandler) StoreId(org.neo4j.kernel.impl.store.StoreId) MasterClient(org.neo4j.kernel.ha.com.slave.MasterClient) RequestContext(org.neo4j.com.RequestContext) TransactionCommittingResponseUnpacker(org.neo4j.com.storecopy.TransactionCommittingResponseUnpacker) ResponseUnpacker(org.neo4j.com.storecopy.ResponseUnpacker) MasterImplTest(org.neo4j.kernel.ha.com.master.MasterImplTest) Test(org.junit.Test)

Example 15 with StoreId

use of org.neo4j.kernel.impl.store.StoreId in project neo4j by neo4j.

the class MasterClientTest method newClientsShouldNotIgnoreStoreIdDifferences.

@Test(expected = MismatchingStoreIdException.class)
public void newClientsShouldNotIgnoreStoreIdDifferences() throws Throwable {
    // Given
    MasterImpl.SPI masterImplSPI = MasterImplTest.mockedSpi(StoreIdTestFactory.newStoreIdForCurrentVersion(1, 2, 3, 4));
    when(masterImplSPI.getTransactionChecksum(anyLong())).thenReturn(5L);
    newMasterServer(masterImplSPI);
    StoreId storeId = StoreIdTestFactory.newStoreIdForCurrentVersion(5, 6, 7, 8);
    MasterClient masterClient = newMasterClient320(storeId);
    // When
    masterClient.handshake(1, storeId);
}
Also used : MasterImpl(org.neo4j.kernel.ha.com.master.MasterImpl) StoreId(org.neo4j.kernel.impl.store.StoreId) MasterClient(org.neo4j.kernel.ha.com.slave.MasterClient) MasterImplTest(org.neo4j.kernel.ha.com.master.MasterImplTest) Test(org.junit.Test)

Aggregations

StoreId (org.neo4j.kernel.impl.store.StoreId)15 Test (org.junit.Test)10 MasterClient (org.neo4j.kernel.ha.com.slave.MasterClient)10 HandshakeResult (org.neo4j.kernel.ha.com.master.HandshakeResult)8 URI (java.net.URI)7 TransactionId (org.neo4j.kernel.impl.store.TransactionId)7 TransactionIdStore (org.neo4j.kernel.impl.transaction.log.TransactionIdStore)7 BranchedDataPolicy (org.neo4j.kernel.ha.BranchedDataPolicy)6 IOException (java.io.IOException)5 InstanceId (org.neo4j.cluster.InstanceId)5 NeoStoreDataSource (org.neo4j.kernel.NeoStoreDataSource)5 MismatchingStoreIdException (org.neo4j.kernel.impl.store.MismatchingStoreIdException)5 BranchedDataException (org.neo4j.kernel.ha.BranchedDataException)4 File (java.io.File)3 InetSocketAddress (java.net.InetSocketAddress)3 Response (org.neo4j.com.Response)3 CancellationRequest (org.neo4j.helpers.CancellationRequest)3 ClusterMember (org.neo4j.kernel.ha.cluster.member.ClusterMember)3 ClusterMembers (org.neo4j.kernel.ha.cluster.member.ClusterMembers)3 MasterImpl (org.neo4j.kernel.ha.com.master.MasterImpl)3