Search in sources :

Example 31 with RequestContext

use of org.neo4j.com.RequestContext in project neo4j by neo4j.

the class SwitchToSlave method copyStoreFromMaster.

void copyStoreFromMaster(MasterClient masterClient, CancellationRequest cancellationRequest, MoveAfterCopy moveAfterCopy) throws Throwable {
    try {
        userLog.info("Copying store from master");
        StoreCopyClient.StoreCopyRequester requester = new StoreCopyClient.StoreCopyRequester() {

            @Override
            public Response<?> copyStore(StoreWriter writer) {
                return masterClient.copyStore(new RequestContext(0, config.get(ClusterSettings.server_id).toIntegerIndex(), 0, BASE_TX_ID, 0), writer);
            }

            @Override
            public void done() {
            // Nothing to clean up here
            }
        };
        MoveAfterCopy moveAfterCopyWithLogging = (moves, fromDirectory, toDirectory) -> {
            userLog.info("Copied store from master to " + fromDirectory);
            msgLog.info("Starting post copy operation to move store from " + fromDirectory + " to " + storeDir);
            moveAfterCopy.move(moves, fromDirectory, toDirectory);
        };
        storeCopyClient.copyStore(requester, cancellationRequest, moveAfterCopyWithLogging);
        startServicesAgain();
        userLog.info("Finished copying store from master");
    } catch (Throwable t) {
        // Delete store so that we can copy from master without conflicts next time
        cleanStoreDir();
        throw t;
    }
}
Also used : InstanceId(org.neo4j.cluster.InstanceId) StoreId(org.neo4j.kernel.impl.store.StoreId) NeoStoreDataSource(org.neo4j.kernel.NeoStoreDataSource) MasterClientResolver(org.neo4j.kernel.ha.com.slave.MasterClientResolver) Log(org.neo4j.logging.Log) OnlineBackupKernelExtension(org.neo4j.backup.OnlineBackupKernelExtension) DelegateInvocationHandler(org.neo4j.kernel.ha.DelegateInvocationHandler) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) RequestContext(org.neo4j.com.RequestContext) MasterClient(org.neo4j.kernel.ha.com.slave.MasterClient) HighAvailabilityModeSwitcher(org.neo4j.kernel.ha.cluster.modeswitch.HighAvailabilityModeSwitcher) TransactionObligationFulfiller(org.neo4j.com.storecopy.TransactionObligationFulfiller) URI(java.net.URI) MismatchingStoreIdException(org.neo4j.kernel.impl.store.MismatchingStoreIdException) StoreUtil(org.neo4j.com.storecopy.StoreUtil) ClusterMembers.hasInstanceId(org.neo4j.kernel.ha.cluster.member.ClusterMembers.hasInstanceId) Response(org.neo4j.com.Response) PageCache(org.neo4j.io.pagecache.PageCache) Iterables.filter(org.neo4j.helpers.collection.Iterables.filter) MissingLogDataException(org.neo4j.kernel.impl.transaction.log.MissingLogDataException) LogService(org.neo4j.kernel.impl.logging.LogService) MILLISECONDS(java.util.concurrent.TimeUnit.MILLISECONDS) TransactionCommittingResponseUnpacker(org.neo4j.com.storecopy.TransactionCommittingResponseUnpacker) UpdatePuller(org.neo4j.kernel.ha.UpdatePuller) UpdatePullerScheduler(org.neo4j.kernel.ha.UpdatePullerScheduler) UnableToCopyStoreFromOldMasterException(org.neo4j.kernel.ha.store.UnableToCopyStoreFromOldMasterException) ClusterMemberAvailability(org.neo4j.cluster.member.ClusterMemberAvailability) Iterables.firstOrNull(org.neo4j.helpers.collection.Iterables.firstOrNull) DependencyResolver(org.neo4j.graphdb.DependencyResolver) RequestContextFactory(org.neo4j.kernel.ha.com.RequestContextFactory) BranchedDataException(org.neo4j.kernel.ha.BranchedDataException) MoveAfterCopy(org.neo4j.com.storecopy.MoveAfterCopy) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) StoreWriter(org.neo4j.com.storecopy.StoreWriter) Monitors(org.neo4j.kernel.monitoring.Monitors) LockSupport.parkNanos(java.util.concurrent.locks.LockSupport.parkNanos) HandshakeResult(org.neo4j.kernel.ha.com.master.HandshakeResult) Function(java.util.function.Function) Supplier(java.util.function.Supplier) PullerFactory(org.neo4j.kernel.ha.PullerFactory) ServerUtil(org.neo4j.com.ServerUtil) TransactionId(org.neo4j.kernel.impl.store.TransactionId) ClusterMembers(org.neo4j.kernel.ha.cluster.member.ClusterMembers) StoreLockerLifecycleAdapter(org.neo4j.kernel.internal.StoreLockerLifecycleAdapter) SlaveServer(org.neo4j.kernel.ha.com.slave.SlaveServer) HaIdGeneratorFactory(org.neo4j.kernel.ha.id.HaIdGeneratorFactory) CancellationRequest(org.neo4j.helpers.CancellationRequest) HighAvailabilityModeSwitcher.getServerId(org.neo4j.kernel.ha.cluster.modeswitch.HighAvailabilityModeSwitcher.getServerId) NeoStores.isStorePresent(org.neo4j.kernel.impl.store.NeoStores.isStorePresent) BASE_TX_ID(org.neo4j.kernel.impl.transaction.log.TransactionIdStore.BASE_TX_ID) Lifecycle(org.neo4j.kernel.lifecycle.Lifecycle) Config(org.neo4j.kernel.configuration.Config) IOException(java.io.IOException) HaSettings(org.neo4j.kernel.ha.HaSettings) DataSourceManager(org.neo4j.kernel.impl.transaction.state.DataSourceManager) File(java.io.File) StoreOutOfDateException(org.neo4j.kernel.ha.StoreOutOfDateException) Server(org.neo4j.com.Server) SlaveImpl(org.neo4j.kernel.ha.com.slave.SlaveImpl) IndexConfigStore(org.neo4j.kernel.impl.index.IndexConfigStore) Clock(java.time.Clock) ClusterSettings(org.neo4j.cluster.ClusterSettings) Slave(org.neo4j.kernel.ha.com.master.Slave) StoreCopyClient(org.neo4j.com.storecopy.StoreCopyClient) TransactionStats(org.neo4j.kernel.impl.transaction.TransactionStats) ClusterMember(org.neo4j.kernel.ha.cluster.member.ClusterMember) FileSystemWatcherService(org.neo4j.kernel.impl.util.watcher.FileSystemWatcherService) Clocks(org.neo4j.time.Clocks) Master(org.neo4j.kernel.ha.com.master.Master) StoreWriter(org.neo4j.com.storecopy.StoreWriter) StoreCopyClient(org.neo4j.com.storecopy.StoreCopyClient) RequestContext(org.neo4j.com.RequestContext) MoveAfterCopy(org.neo4j.com.storecopy.MoveAfterCopy)

Example 32 with RequestContext

use of org.neo4j.com.RequestContext in project neo4j by neo4j.

the class MasterClient214 method createPropertyKey.

@Override
public Response<Integer> createPropertyKey(RequestContext context, final String name) {
    Serializer serializer = buffer -> writeString(buffer, name);
    Deserializer<Integer> deserializer = (buffer, temporaryBuffer) -> buffer.readInt();
    return sendRequest(requestTypes.type(HaRequestTypes.Type.CREATE_PROPERTY_KEY), context, serializer, deserializer);
}
Also used : VOID_DESERIALIZER(org.neo4j.com.Protocol.VOID_DESERIALIZER) StoreId(org.neo4j.kernel.impl.store.StoreId) IdAllocation(org.neo4j.kernel.ha.id.IdAllocation) RequestType(org.neo4j.com.RequestType) LockResult(org.neo4j.kernel.ha.lock.LockResult) ObjectSerializer(org.neo4j.com.ObjectSerializer) ChannelBuffer(org.jboss.netty.buffer.ChannelBuffer) ResourceType(org.neo4j.storageengine.api.lock.ResourceType) Exceptions(org.neo4j.helpers.Exceptions) StoreWriter(org.neo4j.com.storecopy.StoreWriter) LogProvider(org.neo4j.logging.LogProvider) Serializer(org.neo4j.com.Serializer) HandshakeResult(org.neo4j.kernel.ha.com.master.HandshakeResult) IdRange(org.neo4j.kernel.impl.store.id.IdRange) RequestContext(org.neo4j.com.RequestContext) RequestMonitor(org.neo4j.com.monitor.RequestMonitor) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) INTERNAL_PROTOCOL_VERSION(org.neo4j.com.ProtocolVersion.INTERNAL_PROTOCOL_VERSION) MasterClient(org.neo4j.kernel.ha.com.slave.MasterClient) ByteCounterMonitor(org.neo4j.kernel.monitoring.ByteCounterMonitor) Client(org.neo4j.com.Client) EMPTY_SERIALIZER(org.neo4j.com.Protocol.EMPTY_SERIALIZER) Response(org.neo4j.com.Response) Protocol(org.neo4j.com.Protocol) TransactionRepresentation(org.neo4j.kernel.impl.transaction.TransactionRepresentation) LogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader) IOException(java.io.IOException) Protocol214(org.neo4j.com.Protocol214) MasterServer(org.neo4j.kernel.ha.com.master.MasterServer) LockStatus(org.neo4j.kernel.ha.lock.LockStatus) String.format(java.lang.String.format) IdType(org.neo4j.kernel.impl.store.id.IdType) ResponseUnpacker(org.neo4j.com.storecopy.ResponseUnpacker) Protocol.writeString(org.neo4j.com.Protocol.writeString) ProtocolVersion(org.neo4j.com.ProtocolVersion) Protocol.readString(org.neo4j.com.Protocol.readString) Deserializer(org.neo4j.com.Deserializer) Master(org.neo4j.kernel.ha.com.master.Master) ObjectSerializer(org.neo4j.com.ObjectSerializer) Serializer(org.neo4j.com.Serializer)

Example 33 with RequestContext

use of org.neo4j.com.RequestContext in project neo4j by neo4j.

the class SlaveUpdatePuller method doPullUpdates.

private void doPullUpdates() {
    try {
        RequestContext context = requestContextFactory.newRequestContext();
        try (Response<Void> ignored = master.pullUpdates(context)) {
            // Updates would be applied as part of response processing
            monitor.pulledUpdates(context.lastAppliedTransaction());
        }
        invalidEpochCappedLogger.reset();
        comExceptionCappedLogger.reset();
    } catch (InvalidEpochException e) {
        invalidEpochHandler.handle();
        invalidEpochCappedLogger.warn("Pull updates by " + this + " failed at the epoch check", e);
    } catch (ComException e) {
        invalidEpochCappedLogger.warn("Pull updates by " + this + " failed due to network error.", e);
    } catch (Throwable e) {
        logger.error("Pull updates by " + this + " failed", e);
    }
    lastUpdateTime.setLastUpdateTime(currentTimeMillis());
}
Also used : ComException(org.neo4j.com.ComException) InvalidEpochException(org.neo4j.kernel.ha.com.master.InvalidEpochException) RequestContext(org.neo4j.com.RequestContext)

Example 34 with RequestContext

use of org.neo4j.com.RequestContext in project neo4j by neo4j.

the class BackupImpl method fullBackup.

@Override
public Response<Void> fullBackup(StoreWriter writer, boolean forensics) {
    try (StoreWriter storeWriter = writer) {
        logger.log("Full backup started...");
        RequestContext copyStartContext = storeCopyServer.flushStoresAndStreamStoreFiles(FULL_BACKUP_CHECKPOINT_TRIGGER, storeWriter, forensics);
        ResponsePacker responsePacker = new StoreCopyResponsePacker(logicalTransactionStore, transactionIdStore, logFileInformation, storeId, copyStartContext.lastAppliedTransaction() + 1, // mandatory transaction id
        storeCopyServer.monitor());
        long optionalTransactionId = copyStartContext.lastAppliedTransaction();
        return responsePacker.packTransactionStreamResponse(anonymous(optionalTransactionId), null);
    } finally {
        logger.log("Full backup finished.");
    }
}
Also used : StoreWriter(org.neo4j.com.storecopy.StoreWriter) ResponsePacker(org.neo4j.com.storecopy.ResponsePacker) RequestContext(org.neo4j.com.RequestContext)

Example 35 with RequestContext

use of org.neo4j.com.RequestContext in project neo4j by neo4j.

the class MasterImpl method getOngoingTransactions.

// =====================================================================
// Just some methods which aren't really used when running a HA cluster,
// but exposed so that other tools can reach that information.
// =====================================================================
public Map<Integer, Collection<RequestContext>> getOngoingTransactions() {
    Map<Integer, Collection<RequestContext>> result = new HashMap<>();
    Set<RequestContext> contexts = conversationManager.getActiveContexts();
    for (RequestContext context : contexts.toArray(new RequestContext[contexts.size()])) {
        Collection<RequestContext> txs = result.computeIfAbsent(context.machineId(), k -> new ArrayList<>());
        txs.add(context);
    }
    return result;
}
Also used : HashMap(java.util.HashMap) Collection(java.util.Collection) RequestContext(org.neo4j.com.RequestContext)

Aggregations

RequestContext (org.neo4j.com.RequestContext)38 Test (org.junit.Test)21 Config (org.neo4j.kernel.configuration.Config)13 SPI (org.neo4j.kernel.ha.com.master.MasterImpl.SPI)12 Response (org.neo4j.com.Response)10 LockResult (org.neo4j.kernel.ha.lock.LockResult)10 StoreId (org.neo4j.kernel.impl.store.StoreId)9 IOException (java.io.IOException)8 StoreWriter (org.neo4j.com.storecopy.StoreWriter)8 DefaultConversationSPI (org.neo4j.kernel.ha.cluster.DefaultConversationSPI)8 MasterClient (org.neo4j.kernel.ha.com.slave.MasterClient)8 Client (org.neo4j.kernel.impl.locking.Locks.Client)8 TransactionRepresentation (org.neo4j.kernel.impl.transaction.TransactionRepresentation)8 ResponseUnpacker (org.neo4j.com.storecopy.ResponseUnpacker)7 HandshakeResult (org.neo4j.kernel.ha.com.master.HandshakeResult)7 Master (org.neo4j.kernel.ha.com.master.Master)7 RequestMonitor (org.neo4j.com.monitor.RequestMonitor)6 ReadableClosablePositionAwareChannel (org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)6 String.format (java.lang.String.format)5 ChannelBuffer (org.jboss.netty.buffer.ChannelBuffer)4