Search in sources :

Example 11 with Response

use of org.neo4j.com.Response in project neo4j by neo4j.

the class ResponsePacker method packTransactionStreamResponse.

public <T> Response<T> packTransactionStreamResponse(RequestContext context, T response) {
    final long toStartFrom = context.lastAppliedTransaction() + 1;
    final long toEndAt = transactionIdStore.getLastCommittedTransactionId();
    TransactionStream transactions = visitor -> {
        if (toStartFrom > BASE_TX_ID && toStartFrom <= toEndAt) {
            extractTransactions(toStartFrom, filterVisitor(visitor, toEndAt));
        }
    };
    return new TransactionStreamResponse<>(response, storeId.get(), transactions, ResourceReleaser.NO_OP);
}
Also used : Response(org.neo4j.com.Response) StoreId(org.neo4j.kernel.impl.store.StoreId) TransactionStream(org.neo4j.com.TransactionStream) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) Supplier(java.util.function.Supplier) RequestContext(org.neo4j.com.RequestContext) IOCursor(org.neo4j.cursor.IOCursor) TransactionStreamResponse(org.neo4j.com.TransactionStreamResponse) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) ResourceReleaser(org.neo4j.com.ResourceReleaser) Visitor(org.neo4j.helpers.collection.Visitor) TransactionObligationResponse(org.neo4j.com.TransactionObligationResponse) BASE_TX_ID(org.neo4j.kernel.impl.transaction.log.TransactionIdStore.BASE_TX_ID) TransactionStreamResponse(org.neo4j.com.TransactionStreamResponse) TransactionStream(org.neo4j.com.TransactionStream)

Example 12 with Response

use of org.neo4j.com.Response in project neo4j by neo4j.

the class ResponsePackerIT method shouldPackTheHighestTxCommittedAsObligation.

@Test
public void shouldPackTheHighestTxCommittedAsObligation() throws Exception {
    // GIVEN
    LogicalTransactionStore transactionStore = mock(LogicalTransactionStore.class);
    FileSystemAbstraction fs = fsRule.get();
    PageCache pageCache = pageCacheRule.getPageCache(fs);
    try (NeoStores neoStore = createNeoStore(fs, pageCache)) {
        MetaDataStore store = neoStore.getMetaDataStore();
        store.transactionCommitted(2, 111, BASE_TX_COMMIT_TIMESTAMP);
        store.transactionCommitted(3, 222, BASE_TX_COMMIT_TIMESTAMP);
        store.transactionCommitted(4, 333, BASE_TX_COMMIT_TIMESTAMP);
        store.transactionCommitted(5, 444, BASE_TX_COMMIT_TIMESTAMP);
        store.transactionCommitted(6, 555, BASE_TX_COMMIT_TIMESTAMP);
        // skip 7 to emulate the fact we have an hole in the committed tx ids list
        final long expectedTxId = 8L;
        store.transactionCommitted(expectedTxId, 777, BASE_TX_COMMIT_TIMESTAMP);
        ResponsePacker packer = new ResponsePacker(transactionStore, store, Suppliers.singleton(newStoreIdForCurrentVersion()));
        // WHEN
        Response<Object> response = packer.packTransactionObligationResponse(new RequestContext(0, 0, 0, 0, 0), new Object());
        // THEN
        assertTrue(response instanceof TransactionObligationResponse);
        ((TransactionObligationResponse) response).accept(new Response.Handler() {

            @Override
            public void obligation(long txId) throws IOException {
                assertEquals(expectedTxId, txId);
            }

            @Override
            public Visitor<CommittedTransactionRepresentation, Exception> transactions() {
                throw new UnsupportedOperationException("not expected");
            }
        });
    }
}
Also used : FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) Visitor(org.neo4j.helpers.collection.Visitor) MetaDataStore(org.neo4j.kernel.impl.store.MetaDataStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) IOException(java.io.IOException) Response(org.neo4j.com.Response) TransactionObligationResponse(org.neo4j.com.TransactionObligationResponse) NeoStores(org.neo4j.kernel.impl.store.NeoStores) TransactionObligationResponse(org.neo4j.com.TransactionObligationResponse) RequestContext(org.neo4j.com.RequestContext) PageCache(org.neo4j.io.pagecache.PageCache) Test(org.junit.Test)

Example 13 with Response

use of org.neo4j.com.Response in project neo4j by neo4j.

the class ResponsePackerTest method shouldHaveFixedTargetTransactionIdEvenIfLastTransactionIdIsMoving.

@Test
public void shouldHaveFixedTargetTransactionIdEvenIfLastTransactionIdIsMoving() throws Exception {
    // GIVEN
    LogicalTransactionStore transactionStore = mock(LogicalTransactionStore.class);
    long lastAppliedTransactionId = 5L;
    TransactionCursor endlessCursor = new EndlessCursor(lastAppliedTransactionId + 1);
    when(transactionStore.getTransactions(anyLong())).thenReturn(endlessCursor);
    final long targetTransactionId = 8L;
    final TransactionIdStore transactionIdStore = new DeadSimpleTransactionIdStore(targetTransactionId, 0, BASE_TX_COMMIT_TIMESTAMP, 0, 0);
    ResponsePacker packer = new ResponsePacker(transactionStore, transactionIdStore, Suppliers.singleton(StoreIdTestFactory.newStoreIdForCurrentVersion()));
    // WHEN
    Response<Object> response = packer.packTransactionStreamResponse(requestContextStartingAt(5L), null);
    final AtomicLong nextExpectedVisit = new AtomicLong(lastAppliedTransactionId);
    response.accept(new Response.Handler() {

        @Override
        public void obligation(long txId) throws IOException {
            fail("Should not be called");
        }

        @Override
        public Visitor<CommittedTransactionRepresentation, Exception> transactions() {
            return new Visitor<CommittedTransactionRepresentation, Exception>() {

                @Override
                public boolean visit(CommittedTransactionRepresentation element) {
                    // THEN
                    long txId = element.getCommitEntry().getTxId();
                    assertThat(txId, lessThanOrEqualTo(targetTransactionId));
                    assertEquals(nextExpectedVisit.incrementAndGet(), txId);
                    // Move the target transaction id forward one step, effectively always keeping it out of reach
                    transactionIdStore.setLastCommittedAndClosedTransactionId(transactionIdStore.getLastCommittedTransactionId() + 1, 0, BASE_TX_COMMIT_TIMESTAMP, 3, 4);
                    return true;
                }
            };
        }
    });
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) DeadSimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.DeadSimpleTransactionIdStore) Visitor(org.neo4j.helpers.collection.Visitor) DeadSimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.DeadSimpleTransactionIdStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) IOException(java.io.IOException) IOException(java.io.IOException) Response(org.neo4j.com.Response) AtomicLong(java.util.concurrent.atomic.AtomicLong) TransactionCursor(org.neo4j.kernel.impl.transaction.log.TransactionCursor) Test(org.junit.Test)

Example 14 with Response

use of org.neo4j.com.Response in project neo4j by neo4j.

the class SwitchToSlave method copyStoreFromMaster.

void copyStoreFromMaster(MasterClient masterClient, CancellationRequest cancellationRequest, MoveAfterCopy moveAfterCopy) throws Throwable {
    try {
        userLog.info("Copying store from master");
        StoreCopyClient.StoreCopyRequester requester = new StoreCopyClient.StoreCopyRequester() {

            @Override
            public Response<?> copyStore(StoreWriter writer) {
                return masterClient.copyStore(new RequestContext(0, config.get(ClusterSettings.server_id).toIntegerIndex(), 0, BASE_TX_ID, 0), writer);
            }

            @Override
            public void done() {
            // Nothing to clean up here
            }
        };
        MoveAfterCopy moveAfterCopyWithLogging = (moves, fromDirectory, toDirectory) -> {
            userLog.info("Copied store from master to " + fromDirectory);
            msgLog.info("Starting post copy operation to move store from " + fromDirectory + " to " + storeDir);
            moveAfterCopy.move(moves, fromDirectory, toDirectory);
        };
        storeCopyClient.copyStore(requester, cancellationRequest, moveAfterCopyWithLogging);
        startServicesAgain();
        userLog.info("Finished copying store from master");
    } catch (Throwable t) {
        // Delete store so that we can copy from master without conflicts next time
        cleanStoreDir();
        throw t;
    }
}
Also used : InstanceId(org.neo4j.cluster.InstanceId) StoreId(org.neo4j.kernel.impl.store.StoreId) NeoStoreDataSource(org.neo4j.kernel.NeoStoreDataSource) MasterClientResolver(org.neo4j.kernel.ha.com.slave.MasterClientResolver) Log(org.neo4j.logging.Log) OnlineBackupKernelExtension(org.neo4j.backup.OnlineBackupKernelExtension) DelegateInvocationHandler(org.neo4j.kernel.ha.DelegateInvocationHandler) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) RequestContext(org.neo4j.com.RequestContext) MasterClient(org.neo4j.kernel.ha.com.slave.MasterClient) HighAvailabilityModeSwitcher(org.neo4j.kernel.ha.cluster.modeswitch.HighAvailabilityModeSwitcher) TransactionObligationFulfiller(org.neo4j.com.storecopy.TransactionObligationFulfiller) URI(java.net.URI) MismatchingStoreIdException(org.neo4j.kernel.impl.store.MismatchingStoreIdException) StoreUtil(org.neo4j.com.storecopy.StoreUtil) ClusterMembers.hasInstanceId(org.neo4j.kernel.ha.cluster.member.ClusterMembers.hasInstanceId) Response(org.neo4j.com.Response) PageCache(org.neo4j.io.pagecache.PageCache) Iterables.filter(org.neo4j.helpers.collection.Iterables.filter) MissingLogDataException(org.neo4j.kernel.impl.transaction.log.MissingLogDataException) LogService(org.neo4j.kernel.impl.logging.LogService) MILLISECONDS(java.util.concurrent.TimeUnit.MILLISECONDS) TransactionCommittingResponseUnpacker(org.neo4j.com.storecopy.TransactionCommittingResponseUnpacker) UpdatePuller(org.neo4j.kernel.ha.UpdatePuller) UpdatePullerScheduler(org.neo4j.kernel.ha.UpdatePullerScheduler) UnableToCopyStoreFromOldMasterException(org.neo4j.kernel.ha.store.UnableToCopyStoreFromOldMasterException) ClusterMemberAvailability(org.neo4j.cluster.member.ClusterMemberAvailability) Iterables.firstOrNull(org.neo4j.helpers.collection.Iterables.firstOrNull) DependencyResolver(org.neo4j.graphdb.DependencyResolver) RequestContextFactory(org.neo4j.kernel.ha.com.RequestContextFactory) BranchedDataException(org.neo4j.kernel.ha.BranchedDataException) MoveAfterCopy(org.neo4j.com.storecopy.MoveAfterCopy) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) StoreWriter(org.neo4j.com.storecopy.StoreWriter) Monitors(org.neo4j.kernel.monitoring.Monitors) LockSupport.parkNanos(java.util.concurrent.locks.LockSupport.parkNanos) HandshakeResult(org.neo4j.kernel.ha.com.master.HandshakeResult) Function(java.util.function.Function) Supplier(java.util.function.Supplier) PullerFactory(org.neo4j.kernel.ha.PullerFactory) ServerUtil(org.neo4j.com.ServerUtil) TransactionId(org.neo4j.kernel.impl.store.TransactionId) ClusterMembers(org.neo4j.kernel.ha.cluster.member.ClusterMembers) StoreLockerLifecycleAdapter(org.neo4j.kernel.internal.StoreLockerLifecycleAdapter) SlaveServer(org.neo4j.kernel.ha.com.slave.SlaveServer) HaIdGeneratorFactory(org.neo4j.kernel.ha.id.HaIdGeneratorFactory) CancellationRequest(org.neo4j.helpers.CancellationRequest) HighAvailabilityModeSwitcher.getServerId(org.neo4j.kernel.ha.cluster.modeswitch.HighAvailabilityModeSwitcher.getServerId) NeoStores.isStorePresent(org.neo4j.kernel.impl.store.NeoStores.isStorePresent) BASE_TX_ID(org.neo4j.kernel.impl.transaction.log.TransactionIdStore.BASE_TX_ID) Lifecycle(org.neo4j.kernel.lifecycle.Lifecycle) Config(org.neo4j.kernel.configuration.Config) IOException(java.io.IOException) HaSettings(org.neo4j.kernel.ha.HaSettings) DataSourceManager(org.neo4j.kernel.impl.transaction.state.DataSourceManager) File(java.io.File) StoreOutOfDateException(org.neo4j.kernel.ha.StoreOutOfDateException) Server(org.neo4j.com.Server) SlaveImpl(org.neo4j.kernel.ha.com.slave.SlaveImpl) IndexConfigStore(org.neo4j.kernel.impl.index.IndexConfigStore) Clock(java.time.Clock) ClusterSettings(org.neo4j.cluster.ClusterSettings) Slave(org.neo4j.kernel.ha.com.master.Slave) StoreCopyClient(org.neo4j.com.storecopy.StoreCopyClient) TransactionStats(org.neo4j.kernel.impl.transaction.TransactionStats) ClusterMember(org.neo4j.kernel.ha.cluster.member.ClusterMember) FileSystemWatcherService(org.neo4j.kernel.impl.util.watcher.FileSystemWatcherService) Clocks(org.neo4j.time.Clocks) Master(org.neo4j.kernel.ha.com.master.Master) StoreWriter(org.neo4j.com.storecopy.StoreWriter) StoreCopyClient(org.neo4j.com.storecopy.StoreCopyClient) RequestContext(org.neo4j.com.RequestContext) MoveAfterCopy(org.neo4j.com.storecopy.MoveAfterCopy)

Example 15 with Response

use of org.neo4j.com.Response in project neo4j by neo4j.

the class MasterClient214 method createPropertyKey.

@Override
public Response<Integer> createPropertyKey(RequestContext context, final String name) {
    Serializer serializer = buffer -> writeString(buffer, name);
    Deserializer<Integer> deserializer = (buffer, temporaryBuffer) -> buffer.readInt();
    return sendRequest(requestTypes.type(HaRequestTypes.Type.CREATE_PROPERTY_KEY), context, serializer, deserializer);
}
Also used : VOID_DESERIALIZER(org.neo4j.com.Protocol.VOID_DESERIALIZER) StoreId(org.neo4j.kernel.impl.store.StoreId) IdAllocation(org.neo4j.kernel.ha.id.IdAllocation) RequestType(org.neo4j.com.RequestType) LockResult(org.neo4j.kernel.ha.lock.LockResult) ObjectSerializer(org.neo4j.com.ObjectSerializer) ChannelBuffer(org.jboss.netty.buffer.ChannelBuffer) ResourceType(org.neo4j.storageengine.api.lock.ResourceType) Exceptions(org.neo4j.helpers.Exceptions) StoreWriter(org.neo4j.com.storecopy.StoreWriter) LogProvider(org.neo4j.logging.LogProvider) Serializer(org.neo4j.com.Serializer) HandshakeResult(org.neo4j.kernel.ha.com.master.HandshakeResult) IdRange(org.neo4j.kernel.impl.store.id.IdRange) RequestContext(org.neo4j.com.RequestContext) RequestMonitor(org.neo4j.com.monitor.RequestMonitor) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) INTERNAL_PROTOCOL_VERSION(org.neo4j.com.ProtocolVersion.INTERNAL_PROTOCOL_VERSION) MasterClient(org.neo4j.kernel.ha.com.slave.MasterClient) ByteCounterMonitor(org.neo4j.kernel.monitoring.ByteCounterMonitor) Client(org.neo4j.com.Client) EMPTY_SERIALIZER(org.neo4j.com.Protocol.EMPTY_SERIALIZER) Response(org.neo4j.com.Response) Protocol(org.neo4j.com.Protocol) TransactionRepresentation(org.neo4j.kernel.impl.transaction.TransactionRepresentation) LogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader) IOException(java.io.IOException) Protocol214(org.neo4j.com.Protocol214) MasterServer(org.neo4j.kernel.ha.com.master.MasterServer) LockStatus(org.neo4j.kernel.ha.lock.LockStatus) String.format(java.lang.String.format) IdType(org.neo4j.kernel.impl.store.id.IdType) ResponseUnpacker(org.neo4j.com.storecopy.ResponseUnpacker) Protocol.writeString(org.neo4j.com.Protocol.writeString) ProtocolVersion(org.neo4j.com.ProtocolVersion) Protocol.readString(org.neo4j.com.Protocol.readString) Deserializer(org.neo4j.com.Deserializer) Master(org.neo4j.kernel.ha.com.master.Master) ObjectSerializer(org.neo4j.com.ObjectSerializer) Serializer(org.neo4j.com.Serializer)

Aggregations

Response (org.neo4j.com.Response)17 IOException (java.io.IOException)12 StoreId (org.neo4j.kernel.impl.store.StoreId)12 RequestContext (org.neo4j.com.RequestContext)11 HandshakeResult (org.neo4j.kernel.ha.com.master.HandshakeResult)10 MasterClient (org.neo4j.kernel.ha.com.slave.MasterClient)10 Test (org.junit.Test)7 ResponseUnpacker (org.neo4j.com.storecopy.ResponseUnpacker)7 StoreWriter (org.neo4j.com.storecopy.StoreWriter)7 Master (org.neo4j.kernel.ha.com.master.Master)7 String.format (java.lang.String.format)6 ChannelBuffer (org.jboss.netty.buffer.ChannelBuffer)6 Client (org.neo4j.com.Client)6 Deserializer (org.neo4j.com.Deserializer)6 ObjectSerializer (org.neo4j.com.ObjectSerializer)6 Protocol (org.neo4j.com.Protocol)6 EMPTY_SERIALIZER (org.neo4j.com.Protocol.EMPTY_SERIALIZER)6 VOID_DESERIALIZER (org.neo4j.com.Protocol.VOID_DESERIALIZER)6 Protocol.readString (org.neo4j.com.Protocol.readString)6 Protocol.writeString (org.neo4j.com.Protocol.writeString)6