Search in sources :

Example 1 with SszListSchema

use of tech.pegasys.teku.infrastructure.ssz.schema.SszListSchema in project teku by ConsenSys.

the class DepositProvider method getDepositsWithProof.

/**
 * @param fromDepositIndex inclusive
 * @param toDepositIndex exclusive
 * @param eth1DepositCount number of deposits in the merkle tree according to Eth1Data in state
 * @return
 */
private SszList<Deposit> getDepositsWithProof(UInt64 fromDepositIndex, UInt64 toDepositIndex, UInt64 eth1DepositCount, long maxDeposits) {
    final AtomicReference<UInt64> expectedDepositIndex = new AtomicReference<>(fromDepositIndex);
    SszListSchema<Deposit, ?> depositsSchema = depositsSchemaCache.get(maxDeposits);
    return depositNavigableMap.subMap(fromDepositIndex, true, toDepositIndex, false).values().stream().map(deposit -> {
        if (!deposit.getIndex().equals(expectedDepositIndex.get())) {
            throw MissingDepositsException.missingRange(expectedDepositIndex.get(), deposit.getIndex());
        }
        expectedDepositIndex.set(deposit.getIndex().plus(ONE));
        SszBytes32Vector proof = Deposit.SSZ_SCHEMA.getProofSchema().of(depositMerkleTree.getProofWithViewBoundary(deposit.getIndex().intValue(), eth1DepositCount.intValue()));
        return new DepositWithIndex(proof, deposit.getData(), deposit.getIndex());
    }).collect(depositsSchema.collector());
}
Also used : SszBytes32Vector(tech.pegasys.teku.infrastructure.ssz.collections.SszBytes32Vector) SszList(tech.pegasys.teku.infrastructure.ssz.SszList) MinGenesisTimeBlockEvent(tech.pegasys.teku.ethereum.pow.api.MinGenesisTimeBlockEvent) ONE(tech.pegasys.teku.infrastructure.unsigned.UInt64.ONE) AtomicReference(java.util.concurrent.atomic.AtomicReference) Eth1Data(tech.pegasys.teku.spec.datastructures.blocks.Eth1Data) SszListSchema(tech.pegasys.teku.infrastructure.ssz.schema.SszListSchema) UInt64(tech.pegasys.teku.infrastructure.unsigned.UInt64) MerkleTree(tech.pegasys.teku.spec.datastructures.util.MerkleTree) DepositUtil(tech.pegasys.teku.spec.datastructures.util.DepositUtil) DepositsFromBlockEvent(tech.pegasys.teku.ethereum.pow.api.DepositsFromBlockEvent) Spec(tech.pegasys.teku.spec.Spec) FinalizedCheckpointChannel(tech.pegasys.teku.storage.api.FinalizedCheckpointChannel) Bytes32(org.apache.tuweni.bytes.Bytes32) NavigableMap(java.util.NavigableMap) Deposit(tech.pegasys.teku.spec.datastructures.operations.Deposit) DepositWithIndex(tech.pegasys.teku.spec.datastructures.operations.DepositWithIndex) Logger(org.apache.logging.log4j.Logger) TekuMetricCategory(tech.pegasys.teku.infrastructure.metrics.TekuMetricCategory) TreeMap(java.util.TreeMap) RecentChainData(tech.pegasys.teku.storage.client.RecentChainData) MetricsSystem(org.hyperledger.besu.plugin.services.MetricsSystem) LogManager(org.apache.logging.log4j.LogManager) Eth1EventsChannel(tech.pegasys.teku.pow.api.Eth1EventsChannel) Checkpoint(tech.pegasys.teku.spec.datastructures.state.Checkpoint) OptimizedMerkleTree(tech.pegasys.teku.spec.datastructures.util.OptimizedMerkleTree) Counter(org.hyperledger.besu.plugin.services.metrics.Counter) BeaconState(tech.pegasys.teku.spec.datastructures.state.beaconstate.BeaconState) Deposit(tech.pegasys.teku.spec.datastructures.operations.Deposit) SszBytes32Vector(tech.pegasys.teku.infrastructure.ssz.collections.SszBytes32Vector) DepositWithIndex(tech.pegasys.teku.spec.datastructures.operations.DepositWithIndex) AtomicReference(java.util.concurrent.atomic.AtomicReference) UInt64(tech.pegasys.teku.infrastructure.unsigned.UInt64)

Example 2 with SszListSchema

use of tech.pegasys.teku.infrastructure.ssz.schema.SszListSchema in project teku by ConsenSys.

the class BlockProcessorTest method processDepositHelper.

private BeaconState processDepositHelper(BeaconState beaconState, DepositData depositData) throws BlockProcessingException {
    // Add the deposit to a Merkle tree so that we can get the root to put into the state Eth1 data
    MerkleTree depositMerkleTree = new OptimizedMerkleTree(specConfig.getDepositContractTreeDepth());
    depositMerkleTree.add(depositData.hashTreeRoot());
    beaconState = beaconState.updated(state -> state.setEth1_data(new Eth1Data(depositMerkleTree.getRoot(), UInt64.valueOf(1), Bytes32.ZERO)));
    SszListSchema<Deposit, ?> schema = SszListSchema.create(DepositWithIndex.SSZ_SCHEMA, specConfig.getMaxDeposits());
    SszBytes32Vector proof = Deposit.SSZ_SCHEMA.getProofSchema().of(depositMerkleTree.getProof(0));
    SszList<Deposit> deposits = schema.of(new DepositWithIndex(proof, depositData, UInt64.valueOf(0)));
    // Attempt to process deposit with above data.
    return beaconState.updated(state -> blockProcessor.processDeposits(state, deposits));
}
Also used : BouncyCastleExtension(org.apache.tuweni.junit.BouncyCastleExtension) SszBytes32Vector(tech.pegasys.teku.infrastructure.ssz.collections.SszBytes32Vector) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) SszList(tech.pegasys.teku.infrastructure.ssz.SszList) Bytes(org.apache.tuweni.bytes.Bytes) Fork(tech.pegasys.teku.spec.datastructures.state.Fork) SpecVersion(tech.pegasys.teku.spec.SpecVersion) ArrayList(java.util.ArrayList) ExtendWith(org.junit.jupiter.api.extension.ExtendWith) Eth1Data(tech.pegasys.teku.spec.datastructures.blocks.Eth1Data) SszListSchema(tech.pegasys.teku.infrastructure.ssz.schema.SszListSchema) UInt64(tech.pegasys.teku.infrastructure.unsigned.UInt64) MerkleTree(tech.pegasys.teku.spec.datastructures.util.MerkleTree) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) Spec(tech.pegasys.teku.spec.Spec) Bytes32(org.apache.tuweni.bytes.Bytes32) BlockProcessingException(tech.pegasys.teku.spec.logic.common.statetransition.exceptions.BlockProcessingException) Validator(tech.pegasys.teku.spec.datastructures.state.Validator) BLSSignature(tech.pegasys.teku.bls.BLSSignature) BLSPublicKey(tech.pegasys.teku.bls.BLSPublicKey) SpecConfig(tech.pegasys.teku.spec.config.SpecConfig) Deposit(tech.pegasys.teku.spec.datastructures.operations.Deposit) Test(org.junit.jupiter.api.Test) List(java.util.List) DepositWithIndex(tech.pegasys.teku.spec.datastructures.operations.DepositWithIndex) DataStructureUtil(tech.pegasys.teku.spec.util.DataStructureUtil) Bytes48(org.apache.tuweni.bytes.Bytes48) DepositMessage(tech.pegasys.teku.spec.datastructures.operations.DepositMessage) DepositData(tech.pegasys.teku.spec.datastructures.operations.DepositData) OptimizedMerkleTree(tech.pegasys.teku.spec.datastructures.util.OptimizedMerkleTree) BeaconState(tech.pegasys.teku.spec.datastructures.state.beaconstate.BeaconState) OptimizedMerkleTree(tech.pegasys.teku.spec.datastructures.util.OptimizedMerkleTree) Deposit(tech.pegasys.teku.spec.datastructures.operations.Deposit) MerkleTree(tech.pegasys.teku.spec.datastructures.util.MerkleTree) OptimizedMerkleTree(tech.pegasys.teku.spec.datastructures.util.OptimizedMerkleTree) SszBytes32Vector(tech.pegasys.teku.infrastructure.ssz.collections.SszBytes32Vector) DepositWithIndex(tech.pegasys.teku.spec.datastructures.operations.DepositWithIndex) Eth1Data(tech.pegasys.teku.spec.datastructures.blocks.Eth1Data)

Example 3 with SszListSchema

use of tech.pegasys.teku.infrastructure.ssz.schema.SszListSchema in project teku by ConsenSys.

the class SszListHintsTest method testList.

<TElement extends SszData> void testList(SszListSchema<TElement, ?> type, Supplier<TElement> listElementsFactory, Consumer<SszList<TElement>> results) {
    SszList<TElement> def = type.getDefault();
    assertEmptyListVariants(type, def);
    results.accept(def);
    SszMutableList<TElement> w0 = def.createWritableCopy();
    assertEmptyListVariants(type, w0);
    SszList<TElement> r0 = w0.commitChanges();
    assertEmptyListVariants(type, r0);
    results.accept(r0);
    SszMutableList<TElement> w1 = r0.createWritableCopy();
    assertEmptyListVariants(type, w1);
    TElement elem1 = listElementsFactory.get();
    w1.append(elem1);
    assertListElementsVariants(type, w1, List.of(elem1));
    SszList<TElement> r1 = w1.commitChanges();
    assertListElementsVariants(type, r1, List.of(elem1));
    results.accept(r1);
    SszMutableList<TElement> w2 = r1.createWritableCopy();
    assertListElementsVariants(type, r1, List.of(elem1));
    TElement elem2 = listElementsFactory.get();
    w2.append(elem2);
    assertListElementsVariants(type, w2, List.of(elem1, elem2));
    SszList<TElement> r2 = w2.commitChanges();
    assertListElementsVariants(type, r2, List.of(elem1, elem2));
    results.accept(r2);
    SszMutableList<TElement> w3 = r2.createWritableCopy();
    assertListElementsVariants(type, w3, List.of(elem1, elem2));
    TElement elem3 = listElementsFactory.get();
    w3.set(0, elem3);
    assertListElementsVariants(type, w3, List.of(elem3, elem2));
    SszList<TElement> r3 = w3.commitChanges();
    assertListElementsVariants(type, r3, List.of(elem3, elem2));
    results.accept(r3);
    SszMutableList<TElement> w4 = r2.createWritableCopy();
    w3.set(0, elem1);
    SszList<TElement> r4 = w4.commitChanges();
    assertListElementsVariants(type, r4, List.of(elem1, elem2));
    assertListEqualsVariants(type, r4, r2);
    results.accept(r4);
    IntStream sizes = IntStream.range(2, 18);
    sizes = IntStream.concat(sizes, IntStream.of(63, 64, 65, 127, 128, 129, 255, 256, 511, 512, 513, 1023, 1024, 1025));
    sizes = IntStream.concat(sizes, type.getMaxLength() > (1 << 16) ? IntStream.empty() : IntStream.of((int) type.getMaxLength() - 1, (int) type.getMaxLength()));
    sizes.filter(s -> s <= type.getMaxLength()).forEach(size -> {
        SszMutableList<TElement> w1_0 = def.createWritableCopy();
        List<TElement> elements = new ArrayList<>();
        for (int i = 0; i < size; i++) {
            TElement el = listElementsFactory.get();
            elements.add(el);
            w1_0.append(el);
        }
        SszList<TElement> r1_0 = w1_0.commitChanges();
        results.accept(r1_0);
        assertListElementsVariants(type, r1_0, elements);
        IntStream changeIndices = IntStream.of(0, 1, 2, 3, 4, 7, 8, size - 1, size - 2).filter(i -> i < size);
        SszMutableList<TElement> w1_1 = r1_0.createWritableCopy();
        changeIndices.forEach(chIdx -> {
            TElement newElem = listElementsFactory.get();
            elements.set(chIdx, newElem);
            w1_1.set(chIdx, newElem);
        });
        SszList<TElement> r1_1 = w1_1.commitChanges();
        assertListElementsVariants(type, r1_1, elements);
        results.accept(r1_0);
    });
    if (type.getMaxLength() <= (1 << 10)) {
        // check max capacity if the max len is not too huge
        SszMutableList<TElement> w5 = def.createWritableCopy();
        for (long i = 0; i < type.getMaxLength(); i++) {
            w5.append(listElementsFactory.get());
        }
        assertThatThrownBy(() -> w5.append(listElementsFactory.get())).isInstanceOf(IndexOutOfBoundsException.class);
    }
}
Also used : IntStream(java.util.stream.IntStream) SszSchemaHints(tech.pegasys.teku.infrastructure.ssz.schema.SszSchemaHints) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) Random(java.util.Random) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) SszReader(tech.pegasys.teku.infrastructure.ssz.sos.SszReader) Assertions.assertThatThrownBy(org.assertj.core.api.Assertions.assertThatThrownBy) SszListSchema(tech.pegasys.teku.infrastructure.ssz.schema.SszListSchema) TestDoubleSuperContainer(tech.pegasys.teku.infrastructure.ssz.TestContainers.TestDoubleSuperContainer) UInt64(tech.pegasys.teku.infrastructure.unsigned.UInt64) TestSmallContainer(tech.pegasys.teku.infrastructure.ssz.TestContainers.TestSmallContainer) Bytes32(org.apache.tuweni.bytes.Bytes32) TestByteVectorContainer(tech.pegasys.teku.infrastructure.ssz.TestContainers.TestByteVectorContainer) TestContainer(tech.pegasys.teku.infrastructure.ssz.TestContainers.TestContainer) MethodSource(org.junit.jupiter.params.provider.MethodSource) SszSchema(tech.pegasys.teku.infrastructure.ssz.schema.SszSchema) Arguments(org.junit.jupiter.params.provider.Arguments) Collectors(java.util.stream.Collectors) Consumer(java.util.function.Consumer) List(java.util.List) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) Stream(java.util.stream.Stream) TestSubContainer(tech.pegasys.teku.infrastructure.ssz.TestContainers.TestSubContainer) ArrayDeque(java.util.ArrayDeque) ArrayList(java.util.ArrayList) IntStream(java.util.stream.IntStream)

Example 4 with SszListSchema

use of tech.pegasys.teku.infrastructure.ssz.schema.SszListSchema in project teku by ConsenSys.

the class AggregatingAttestationPool method getAttestationsForBlock.

public synchronized SszList<Attestation> getAttestationsForBlock(final BeaconState stateAtBlockSlot, final AttestationForkChecker forkChecker, final AttestationWorthinessChecker worthinessChecker) {
    final UInt64 currentEpoch = spec.getCurrentEpoch(stateAtBlockSlot);
    final int previousEpochLimit = spec.getPreviousEpochAttestationCapacity(stateAtBlockSlot);
    final SszListSchema<Attestation, ?> attestationsSchema = spec.atSlot(stateAtBlockSlot.getSlot()).getSchemaDefinitions().getBeaconBlockBodySchema().getAttestationsSchema();
    final AtomicInteger prevEpochCount = new AtomicInteger(0);
    return dataHashBySlot.descendingMap().values().stream().flatMap(Collection::stream).map(attestationGroupByDataHash::get).filter(Objects::nonNull).filter(group -> isValid(stateAtBlockSlot, group.getAttestationData())).filter(forkChecker::areAttestationsFromCorrectFork).filter(group -> worthinessChecker.areAttestationsWorthy(group.getAttestationData())).flatMap(MatchingDataAttestationGroup::stream).limit(attestationsSchema.getMaxLength()).map(ValidateableAttestation::getAttestation).filter(att -> {
        if (spec.computeEpochAtSlot(att.getData().getSlot()).isLessThan(currentEpoch)) {
            final int currentCount = prevEpochCount.getAndIncrement();
            return currentCount < previousEpochLimit;
        }
        return true;
    }).collect(attestationsSchema.collector());
}
Also used : SszList(tech.pegasys.teku.infrastructure.ssz.SszList) HashMap(java.util.HashMap) Bytes(org.apache.tuweni.bytes.Bytes) HashSet(java.util.HashSet) SettableGauge(tech.pegasys.teku.infrastructure.metrics.SettableGauge) AttestationData(tech.pegasys.teku.spec.datastructures.operations.AttestationData) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Map(java.util.Map) SszListSchema(tech.pegasys.teku.infrastructure.ssz.schema.SszListSchema) UInt64(tech.pegasys.teku.infrastructure.unsigned.UInt64) Spec(tech.pegasys.teku.spec.Spec) Bytes32(org.apache.tuweni.bytes.Bytes32) Attestation(tech.pegasys.teku.spec.datastructures.operations.Attestation) SlotEventsChannel(tech.pegasys.teku.ethereum.events.SlotEventsChannel) Predicate(java.util.function.Predicate) Collection(java.util.Collection) Set(java.util.Set) NavigableMap(java.util.NavigableMap) AttestationWorthinessChecker(tech.pegasys.teku.spec.logic.common.statetransition.attestation.AttestationWorthinessChecker) Objects(java.util.Objects) Stream(java.util.stream.Stream) TekuMetricCategory(tech.pegasys.teku.infrastructure.metrics.TekuMetricCategory) TreeMap(java.util.TreeMap) ValidateableAttestation(tech.pegasys.teku.spec.datastructures.attestation.ValidateableAttestation) Optional(java.util.Optional) MetricsSystem(org.hyperledger.besu.plugin.services.MetricsSystem) BeaconState(tech.pegasys.teku.spec.datastructures.state.beaconstate.BeaconState) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Objects(java.util.Objects) Collection(java.util.Collection) UInt64(tech.pegasys.teku.infrastructure.unsigned.UInt64) ValidateableAttestation(tech.pegasys.teku.spec.datastructures.attestation.ValidateableAttestation) Attestation(tech.pegasys.teku.spec.datastructures.operations.Attestation) ValidateableAttestation(tech.pegasys.teku.spec.datastructures.attestation.ValidateableAttestation)

Aggregations

Bytes32 (org.apache.tuweni.bytes.Bytes32)4 SszListSchema (tech.pegasys.teku.infrastructure.ssz.schema.SszListSchema)4 UInt64 (tech.pegasys.teku.infrastructure.unsigned.UInt64)4 SszList (tech.pegasys.teku.infrastructure.ssz.SszList)3 Spec (tech.pegasys.teku.spec.Spec)3 BeaconState (tech.pegasys.teku.spec.datastructures.state.beaconstate.BeaconState)3 ArrayList (java.util.ArrayList)2 List (java.util.List)2 NavigableMap (java.util.NavigableMap)2 TreeMap (java.util.TreeMap)2 Stream (java.util.stream.Stream)2 Bytes (org.apache.tuweni.bytes.Bytes)2 Assertions.assertThat (org.assertj.core.api.Assertions.assertThat)2 MetricsSystem (org.hyperledger.besu.plugin.services.MetricsSystem)2 SszBytes32Vector (tech.pegasys.teku.infrastructure.ssz.collections.SszBytes32Vector)2 Eth1Data (tech.pegasys.teku.spec.datastructures.blocks.Eth1Data)2 Deposit (tech.pegasys.teku.spec.datastructures.operations.Deposit)2 DepositWithIndex (tech.pegasys.teku.spec.datastructures.operations.DepositWithIndex)2 MerkleTree (tech.pegasys.teku.spec.datastructures.util.MerkleTree)2 OptimizedMerkleTree (tech.pegasys.teku.spec.datastructures.util.OptimizedMerkleTree)2