Search in sources :

Example 1 with Checkpoint

use of org.hyperledger.besu.ethereum.eth.sync.fastsync.checkpoint.Checkpoint in project besu by hyperledger.

the class CheckpointSyncDownloadPipelineFactory method createDownloadCheckPointPipeline.

protected Pipeline<Hash> createDownloadCheckPointPipeline(final SyncState syncState, final SyncTarget target) {
    final Checkpoint checkpoint = syncState.getCheckpoint().orElseThrow();
    final CheckpointSource checkPointSource = new CheckpointSource(syncState, target.peer(), protocolSchedule.getByBlockNumber(checkpoint.blockNumber()).getBlockHeaderFunctions());
    final CheckpointBlockImportStep checkPointBlockImportStep = new CheckpointBlockImportStep(checkPointSource, checkpoint, protocolContext.getBlockchain());
    final CheckpointDownloadBlockStep checkPointDownloadBlockStep = new CheckpointDownloadBlockStep(protocolSchedule, ethContext, checkpoint, metricsSystem);
    return PipelineBuilder.createPipelineFrom("fetchCheckpoints", checkPointSource, 1, metricsSystem.createLabelledCounter(BesuMetricCategory.SYNCHRONIZER, "chain_download_pipeline_processed_total", "Number of header process by each chain download pipeline stage", "step", "action"), true, "checkpointSync").thenProcessAsyncOrdered("downloadBlock", checkPointDownloadBlockStep::downloadBlock, 1).andFinishWith("importBlock", checkPointBlockImportStep);
}
Also used : Checkpoint(org.hyperledger.besu.ethereum.eth.sync.fastsync.checkpoint.Checkpoint)

Example 2 with Checkpoint

use of org.hyperledger.besu.ethereum.eth.sync.fastsync.checkpoint.Checkpoint in project besu by hyperledger.

the class CheckPointSyncChainDownloaderTest method setup.

@Before
public void setup() {
    when(validationPolicy.getValidationModeForNextBlock()).thenReturn(LIGHT_SKIP_DETACHED);
    when(worldStateStorage.isWorldStateAvailable(any(), any())).thenReturn(true);
    final BlockchainSetupUtil localBlockchainSetup = BlockchainSetupUtil.forTesting(storageFormat);
    localBlockchain = localBlockchainSetup.getBlockchain();
    otherBlockchainSetup = BlockchainSetupUtil.forTesting(storageFormat);
    otherBlockchain = otherBlockchainSetup.getBlockchain();
    protocolSchedule = localBlockchainSetup.getProtocolSchedule();
    protocolContext = localBlockchainSetup.getProtocolContext();
    ethProtocolManager = EthProtocolManagerTestUtil.create(localBlockchain, new EthScheduler(1, 1, 1, 1, new NoOpMetricsSystem()));
    ethContext = ethProtocolManager.ethContext();
    final int blockNumber = 10;
    checkpoint = ImmutableCheckpoint.builder().blockNumber(blockNumber).blockHash(localBlockchainSetup.getBlocks().get(blockNumber).getHash()).totalDifficulty(Difficulty.ONE).build();
    syncState = new SyncState(protocolContext.getBlockchain(), ethContext.getEthPeers(), true, Optional.of(checkpoint));
}
Also used : EthScheduler(org.hyperledger.besu.ethereum.eth.manager.EthScheduler) SyncState(org.hyperledger.besu.ethereum.eth.sync.state.SyncState) FastSyncState(org.hyperledger.besu.ethereum.eth.sync.fastsync.FastSyncState) BlockchainSetupUtil(org.hyperledger.besu.ethereum.core.BlockchainSetupUtil) NoOpMetricsSystem(org.hyperledger.besu.metrics.noop.NoOpMetricsSystem) ImmutableCheckpoint(org.hyperledger.besu.ethereum.eth.sync.fastsync.checkpoint.ImmutableCheckpoint) Checkpoint(org.hyperledger.besu.ethereum.eth.sync.fastsync.checkpoint.Checkpoint) Before(org.junit.Before)

Example 3 with Checkpoint

use of org.hyperledger.besu.ethereum.eth.sync.fastsync.checkpoint.Checkpoint in project besu by hyperledger.

the class BesuControllerBuilder method build.

public BesuController build() {
    checkNotNull(genesisConfig, "Missing genesis config");
    checkNotNull(syncConfig, "Missing sync config");
    checkNotNull(ethereumWireProtocolConfiguration, "Missing ethereum protocol configuration");
    checkNotNull(networkId, "Missing network ID");
    checkNotNull(miningParameters, "Missing mining parameters");
    checkNotNull(metricsSystem, "Missing metrics system");
    checkNotNull(privacyParameters, "Missing privacy parameters");
    // Why do we need this?
    checkNotNull(dataDirectory, "Missing data directory");
    checkNotNull(clock, "Missing clock");
    checkNotNull(transactionPoolConfiguration, "Missing transaction pool configuration");
    checkNotNull(nodeKey, "Missing node key");
    checkNotNull(storageProvider, "Must supply a storage provider");
    checkNotNull(gasLimitCalculator, "Missing gas limit calculator");
    checkNotNull(evmConfiguration, "Missing evm config");
    prepForBuild();
    final ProtocolSchedule protocolSchedule = createProtocolSchedule();
    final GenesisState genesisState = GenesisState.fromConfig(genesisConfig, protocolSchedule);
    final WorldStateStorage worldStateStorage = storageProvider.createWorldStateStorage(dataStorageConfiguration.getDataStorageFormat());
    final BlockchainStorage blockchainStorage = storageProvider.createBlockchainStorage(protocolSchedule);
    final MutableBlockchain blockchain = DefaultBlockchain.createMutable(genesisState.getBlock(), blockchainStorage, metricsSystem, reorgLoggingThreshold, dataDirectory.toString());
    final WorldStateArchive worldStateArchive = createWorldStateArchive(worldStateStorage, blockchain);
    if (blockchain.getChainHeadBlockNumber() < 1) {
        genesisState.writeStateTo(worldStateArchive.getMutable());
    }
    final ProtocolContext protocolContext = createProtocolContext(blockchain, worldStateArchive, protocolSchedule, this::createConsensusContext);
    validateContext(protocolContext);
    protocolSchedule.setPublicWorldStateArchiveForPrivacyBlockProcessor(protocolContext.getWorldStateArchive());
    Optional<Pruner> maybePruner = Optional.empty();
    if (isPruningEnabled) {
        if (!storageProvider.isWorldStateIterable()) {
            LOG.warn("Cannot enable pruning with current database version. Disabling. Resync to get the latest database version or disable pruning explicitly on the command line to remove this warning.");
        } else if (dataStorageConfiguration.getDataStorageFormat().equals(DataStorageFormat.BONSAI)) {
            LOG.warn("Cannot enable pruning with Bonsai data storage format. Disabling. Change the data storage format or disable pruning explicitly on the command line to remove this warning.");
        } else {
            maybePruner = Optional.of(new Pruner(new MarkSweepPruner(((DefaultWorldStateArchive) worldStateArchive).getWorldStateStorage(), blockchain, storageProvider.getStorageBySegmentIdentifier(KeyValueSegmentIdentifier.PRUNING_STATE), metricsSystem), blockchain, prunerConfiguration));
        }
    }
    final EthPeers ethPeers = new EthPeers(getSupportedProtocol(), clock, metricsSystem, maxPeers, messagePermissioningProviders);
    final EthMessages ethMessages = new EthMessages();
    final EthMessages snapMessages = new EthMessages();
    final EthScheduler scheduler = new EthScheduler(syncConfig.getDownloaderParallelism(), syncConfig.getTransactionsParallelism(), syncConfig.getComputationParallelism(), metricsSystem);
    final GenesisConfigOptions configOptions = genesisConfig.getConfigOptions(genesisConfigOverrides);
    Optional<Checkpoint> checkpoint = Optional.empty();
    if (configOptions.getCheckpointOptions().isValid()) {
        checkpoint = Optional.of(ImmutableCheckpoint.builder().blockHash(Hash.fromHexString(// NOSONAR
        configOptions.getCheckpointOptions().getHash().get())).blockNumber(configOptions.getCheckpointOptions().getNumber().getAsLong()).totalDifficulty(Difficulty.fromHexString(configOptions.getCheckpointOptions().getTotalDifficulty().get())).build());
    }
    final EthContext ethContext = new EthContext(ethPeers, ethMessages, snapMessages, scheduler);
    final boolean fastSyncEnabled = !SyncMode.isFullSync(syncConfig.getSyncMode());
    final SyncState syncState = new SyncState(blockchain, ethPeers, fastSyncEnabled, checkpoint);
    syncState.subscribeTTDReached(new PandaPrinter());
    final TransactionPool transactionPool = TransactionPoolFactory.createTransactionPool(protocolSchedule, protocolContext, ethContext, clock, metricsSystem, syncState, miningParameters, transactionPoolConfiguration);
    final List<PeerValidator> peerValidators = createPeerValidators(protocolSchedule);
    final EthProtocolManager ethProtocolManager = createEthProtocolManager(protocolContext, fastSyncEnabled, transactionPool, ethereumWireProtocolConfiguration, ethPeers, ethContext, ethMessages, scheduler, peerValidators);
    final Optional<SnapProtocolManager> maybeSnapProtocolManager = createSnapProtocolManager(peerValidators, ethPeers, snapMessages, worldStateArchive);
    final PivotBlockSelector pivotBlockSelector = createPivotSelector(protocolContext);
    final Synchronizer synchronizer = createSynchronizer(protocolSchedule, worldStateStorage, protocolContext, maybePruner, ethContext, syncState, ethProtocolManager, pivotBlockSelector);
    final MiningCoordinator miningCoordinator = createMiningCoordinator(protocolSchedule, protocolContext, transactionPool, miningParameters, syncState, ethProtocolManager);
    final PluginServiceFactory additionalPluginServices = createAdditionalPluginServices(blockchain, protocolContext);
    final SubProtocolConfiguration subProtocolConfiguration = createSubProtocolConfiguration(ethProtocolManager, maybeSnapProtocolManager);
    final JsonRpcMethods additionalJsonRpcMethodFactory = createAdditionalJsonRpcMethodFactory(protocolContext);
    final List<Closeable> closeables = new ArrayList<>();
    closeables.add(storageProvider);
    if (privacyParameters.getPrivateStorageProvider() != null) {
        closeables.add(privacyParameters.getPrivateStorageProvider());
    }
    return new BesuController(protocolSchedule, protocolContext, ethProtocolManager, configOptionsSupplier.get(), subProtocolConfiguration, synchronizer, syncState, transactionPool, miningCoordinator, privacyParameters, miningParameters, additionalJsonRpcMethodFactory, nodeKey, closeables, additionalPluginServices);
}
Also used : GenesisState(org.hyperledger.besu.ethereum.chain.GenesisState) WorldStateStorage(org.hyperledger.besu.ethereum.worldstate.WorldStateStorage) EthContext(org.hyperledger.besu.ethereum.eth.manager.EthContext) Closeable(java.io.Closeable) ArrayList(java.util.ArrayList) EthPeers(org.hyperledger.besu.ethereum.eth.manager.EthPeers) BlockchainStorage(org.hyperledger.besu.ethereum.chain.BlockchainStorage) MiningCoordinator(org.hyperledger.besu.ethereum.blockcreation.MiningCoordinator) JsonRpcMethods(org.hyperledger.besu.ethereum.api.jsonrpc.methods.JsonRpcMethods) SnapProtocolManager(org.hyperledger.besu.ethereum.eth.manager.snap.SnapProtocolManager) EthProtocolManager(org.hyperledger.besu.ethereum.eth.manager.EthProtocolManager) SyncState(org.hyperledger.besu.ethereum.eth.sync.state.SyncState) ProtocolContext(org.hyperledger.besu.ethereum.ProtocolContext) MarkSweepPruner(org.hyperledger.besu.ethereum.worldstate.MarkSweepPruner) EthMessages(org.hyperledger.besu.ethereum.eth.manager.EthMessages) TransactionPool(org.hyperledger.besu.ethereum.eth.transactions.TransactionPool) Synchronizer(org.hyperledger.besu.ethereum.core.Synchronizer) DefaultSynchronizer(org.hyperledger.besu.ethereum.eth.sync.DefaultSynchronizer) Pruner(org.hyperledger.besu.ethereum.worldstate.Pruner) MarkSweepPruner(org.hyperledger.besu.ethereum.worldstate.MarkSweepPruner) ProtocolSchedule(org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule) GenesisConfigOptions(org.hyperledger.besu.config.GenesisConfigOptions) PivotBlockSelector(org.hyperledger.besu.ethereum.eth.sync.PivotBlockSelector) ImmutableCheckpoint(org.hyperledger.besu.ethereum.eth.sync.fastsync.checkpoint.ImmutableCheckpoint) Checkpoint(org.hyperledger.besu.ethereum.eth.sync.fastsync.checkpoint.Checkpoint) DaoForkPeerValidator(org.hyperledger.besu.ethereum.eth.peervalidation.DaoForkPeerValidator) ClassicForkPeerValidator(org.hyperledger.besu.ethereum.eth.peervalidation.ClassicForkPeerValidator) CheckpointBlocksPeerValidator(org.hyperledger.besu.ethereum.eth.peervalidation.CheckpointBlocksPeerValidator) RequiredBlocksPeerValidator(org.hyperledger.besu.ethereum.eth.peervalidation.RequiredBlocksPeerValidator) PeerValidator(org.hyperledger.besu.ethereum.eth.peervalidation.PeerValidator) SubProtocolConfiguration(org.hyperledger.besu.ethereum.p2p.config.SubProtocolConfiguration) EthScheduler(org.hyperledger.besu.ethereum.eth.manager.EthScheduler) DefaultWorldStateArchive(org.hyperledger.besu.ethereum.worldstate.DefaultWorldStateArchive) WorldStateArchive(org.hyperledger.besu.ethereum.worldstate.WorldStateArchive) BonsaiWorldStateArchive(org.hyperledger.besu.ethereum.bonsai.BonsaiWorldStateArchive) MutableBlockchain(org.hyperledger.besu.ethereum.chain.MutableBlockchain) PandaPrinter(org.hyperledger.besu.consensus.merge.PandaPrinter)

Aggregations

Checkpoint (org.hyperledger.besu.ethereum.eth.sync.fastsync.checkpoint.Checkpoint)3 EthScheduler (org.hyperledger.besu.ethereum.eth.manager.EthScheduler)2 ImmutableCheckpoint (org.hyperledger.besu.ethereum.eth.sync.fastsync.checkpoint.ImmutableCheckpoint)2 SyncState (org.hyperledger.besu.ethereum.eth.sync.state.SyncState)2 Closeable (java.io.Closeable)1 ArrayList (java.util.ArrayList)1 GenesisConfigOptions (org.hyperledger.besu.config.GenesisConfigOptions)1 PandaPrinter (org.hyperledger.besu.consensus.merge.PandaPrinter)1 ProtocolContext (org.hyperledger.besu.ethereum.ProtocolContext)1 JsonRpcMethods (org.hyperledger.besu.ethereum.api.jsonrpc.methods.JsonRpcMethods)1 MiningCoordinator (org.hyperledger.besu.ethereum.blockcreation.MiningCoordinator)1 BonsaiWorldStateArchive (org.hyperledger.besu.ethereum.bonsai.BonsaiWorldStateArchive)1 BlockchainStorage (org.hyperledger.besu.ethereum.chain.BlockchainStorage)1 GenesisState (org.hyperledger.besu.ethereum.chain.GenesisState)1 MutableBlockchain (org.hyperledger.besu.ethereum.chain.MutableBlockchain)1 BlockchainSetupUtil (org.hyperledger.besu.ethereum.core.BlockchainSetupUtil)1 Synchronizer (org.hyperledger.besu.ethereum.core.Synchronizer)1 EthContext (org.hyperledger.besu.ethereum.eth.manager.EthContext)1 EthMessages (org.hyperledger.besu.ethereum.eth.manager.EthMessages)1 EthPeers (org.hyperledger.besu.ethereum.eth.manager.EthPeers)1