Search in sources :

Example 81 with MutableInteger

use of org.agrona.collections.MutableInteger in project aeron by real-logic.

the class RedirectingNameResolver method updateNameResolutionStatus.

public static boolean updateNameResolutionStatus(final CountersReader counters, final String hostname, final int operationValue) {
    final MutableInteger nameCounterId = new MutableInteger(NULL_VALUE);
    counters.forEach((counterId, typeId, keyBuffer, label) -> {
        if (typeId == NAME_ENTRY_COUNTER_TYPE_ID && hostname.equals(keyBuffer.getStringAscii(0))) {
            nameCounterId.set(counterId);
        }
    });
    final boolean counterFound = NULL_VALUE != nameCounterId.get();
    if (counterFound) {
        final AtomicCounter nameCounter = new AtomicCounter(counters.valuesBuffer(), nameCounterId.get());
        nameCounter.set(operationValue);
    }
    return counterFound;
}
Also used : MutableInteger(org.agrona.collections.MutableInteger) AtomicCounter(org.agrona.concurrent.status.AtomicCounter)

Example 82 with MutableInteger

use of org.agrona.collections.MutableInteger in project aeron by real-logic.

the class SystemTestWatcher method countClusterMarkFileErrors.

private int countClusterMarkFileErrors(final List<Path> paths, final Predicate<String> filter) {
    final MutableInteger errorCount = new MutableInteger();
    for (final Path path : paths) {
        try (ClusterMarkFile clusterMarkFile = openClusterMarkFile(path)) {
            final AtomicBuffer buffer = clusterMarkFile.errorBuffer();
            ErrorLogReader.read(buffer, (observationCount, firstObservationTimestamp, lastObservationTimestamp, encodedException) -> {
                if (filter.test(encodedException)) {
                    errorCount.set(errorCount.get() + observationCount);
                }
            });
        }
    }
    return errorCount.get();
}
Also used : Path(java.nio.file.Path) ClusterMarkFile(io.aeron.cluster.service.ClusterMarkFile) MutableInteger(org.agrona.collections.MutableInteger) AtomicBuffer(org.agrona.concurrent.AtomicBuffer)

Example 83 with MutableInteger

use of org.agrona.collections.MutableInteger in project aeron by real-logic.

the class ArchiveMigration_2_3 method migrateCatalogFile.

private int migrateCatalogFile(final Catalog catalog, final int version, final MappedByteBuffer mappedByteBuffer) {
    final UnsafeBuffer buffer = new UnsafeBuffer(mappedByteBuffer);
    final int alignment = CACHE_LINE_LENGTH;
    // Create CatalogHeader for version 3.0.0
    final int catalogHeaderLength = writeCatalogHeader(buffer, version, catalog.nextRecordingId(), alignment);
    final MutableInteger offset = new MutableInteger(catalogHeaderLength);
    catalog.forEach((recordingDescriptorOffset, headerEncoder, headerDecoder, descriptorEncoder, descriptorDecoder) -> {
        final int strippedChannelLength = descriptorDecoder.strippedChannelLength();
        descriptorDecoder.skipStrippedChannel();
        final int originalChannelLength = descriptorDecoder.originalChannelLength();
        descriptorDecoder.skipOriginalChannel();
        final int sourceIdentityLength = descriptorDecoder.sourceIdentityLength();
        final int frameLength = align(DESCRIPTOR_HEADER_LENGTH + 7 * SIZE_OF_LONG + 6 * SIZE_OF_INT + SIZE_OF_INT + strippedChannelLength + SIZE_OF_INT + originalChannelLength + SIZE_OF_INT + sourceIdentityLength, alignment);
        final DirectBuffer srcBuffer = headerDecoder.buffer();
        final int headerLength = headerDecoder.encodedLength();
        // Copy recording header
        int index = offset.get();
        buffer.putBytes(index, srcBuffer, 0, headerLength);
        index += headerLength;
        // Correct length
        buffer.putInt(offset.get(), frameLength - headerLength, LITTLE_ENDIAN);
        // Copy recording descriptor
        buffer.putBytes(index, srcBuffer, headerLength, frameLength - headerLength);
        offset.addAndGet(frameLength);
    });
    return offset.get();
}
Also used : MutableInteger(org.agrona.collections.MutableInteger) UnsafeBuffer(org.agrona.concurrent.UnsafeBuffer)

Example 84 with MutableInteger

use of org.agrona.collections.MutableInteger in project aeron by real-logic.

the class ArchiveTool method compact.

static void compact(final PrintStream out, final File archiveDir, final EpochClock epochClock) {
    final File compactFile = new File(archiveDir, CATALOG_FILE_NAME + ".compact");
    try {
        final Path compactFilePath = compactFile.toPath();
        try (FileChannel channel = FileChannel.open(compactFilePath, READ, WRITE, CREATE_NEW);
            Catalog catalog = openCatalogReadOnly(archiveDir, epochClock)) {
            final MappedByteBuffer mappedByteBuffer = channel.map(READ_WRITE, 0, MAX_CATALOG_LENGTH);
            mappedByteBuffer.order(CatalogHeaderEncoder.BYTE_ORDER);
            try {
                final UnsafeBuffer unsafeBuffer = new UnsafeBuffer(mappedByteBuffer);
                new CatalogHeaderEncoder().wrap(unsafeBuffer, 0).version(catalog.version()).length(CatalogHeaderEncoder.BLOCK_LENGTH).nextRecordingId(catalog.nextRecordingId()).alignment(catalog.alignment());
                final MutableInteger offset = new MutableInteger(CatalogHeaderEncoder.BLOCK_LENGTH);
                final MutableInteger deletedRecords = new MutableInteger();
                final MutableInteger reclaimedBytes = new MutableInteger();
                catalog.forEach((recordingDescriptorOffset, headerEncoder, headerDecoder, descriptorEncoder, descriptorDecoder) -> {
                    final int frameLength = headerDecoder.encodedLength() + headerDecoder.length();
                    if (INVALID == headerDecoder.state()) {
                        deletedRecords.increment();
                        reclaimedBytes.addAndGet(frameLength);
                        final ArrayList<String> segmentFiles = listSegmentFiles(archiveDir, descriptorDecoder.recordingId());
                        for (final String segmentFile : segmentFiles) {
                            IoUtil.deleteIfExists(new File(archiveDir, segmentFile));
                        }
                    } else {
                        final int index = offset.getAndAdd(frameLength);
                        unsafeBuffer.putBytes(index, headerDecoder.buffer(), 0, frameLength);
                    }
                });
                out.println("Compaction result: deleted " + deletedRecords.get() + " records and reclaimed " + reclaimedBytes.get() + " bytes");
            } finally {
                BufferUtil.free(mappedByteBuffer);
            }
        }
        final Path catalogFilePath = compactFilePath.resolveSibling(CATALOG_FILE_NAME);
        Files.delete(catalogFilePath);
        Files.move(compactFilePath, catalogFilePath);
    } catch (final IOException ex) {
        ex.printStackTrace(out);
    } finally {
        IoUtil.deleteIfExists(compactFile);
    }
}
Also used : Path(java.nio.file.Path) MappedByteBuffer(java.nio.MappedByteBuffer) FileChannel(java.nio.channels.FileChannel) MutableInteger(org.agrona.collections.MutableInteger) MigrationUtils.fullVersionString(io.aeron.archive.MigrationUtils.fullVersionString) IOException(java.io.IOException) UnsafeBuffer(org.agrona.concurrent.UnsafeBuffer) File(java.io.File) Catalog(io.aeron.archive.Catalog)

Example 85 with MutableInteger

use of org.agrona.collections.MutableInteger in project aeron by real-logic.

the class ArchiveTool method createVerifyEntryProcessor.

private static CatalogEntryProcessor createVerifyEntryProcessor(final PrintStream out, final File archiveDir, final Set<VerifyOption> options, final Catalog catalog, final Checksum checksum, final EpochClock epochClock, final MutableInteger errorCount, final ActionConfirmation<File> truncateOnPageStraddle) {
    final ByteBuffer buffer = BufferUtil.allocateDirectAligned(FILE_IO_MAX_LENGTH_DEFAULT, CACHE_LINE_LENGTH);
    buffer.order(LITTLE_ENDIAN);
    final DataHeaderFlyweight headerFlyweight = new DataHeaderFlyweight(buffer);
    return (recordingDescriptorOffset, headerEncoder, headerDecoder, descriptorEncoder, descriptorDecoder) -> verifyRecording(out, archiveDir, options, catalog, checksum, epochClock, errorCount, truncateOnPageStraddle, headerFlyweight, recordingDescriptorOffset, headerEncoder, headerDecoder, descriptorEncoder, descriptorDecoder);
}
Also used : CACHE_LINE_LENGTH(org.agrona.BitUtil.CACHE_LINE_LENGTH) MigrationUtils.fullVersionString(io.aeron.archive.MigrationUtils.fullVersionString) NATIVE_BYTE_ORDER(org.agrona.BufferUtil.NATIVE_BYTE_ORDER) IntConsumer(java.util.function.IntConsumer) UnsafeBuffer(org.agrona.concurrent.UnsafeBuffer) ByteBuffer(java.nio.ByteBuffer) FILE_IO_MAX_LENGTH_DEFAULT(io.aeron.archive.Archive.Configuration.FILE_IO_MAX_LENGTH_DEFAULT) org.agrona(org.agrona) Collectors.toMap(java.util.stream.Collectors.toMap) Path(java.nio.file.Path) HDR_TYPE_PAD(io.aeron.protocol.HeaderFlyweight.HDR_TYPE_PAD) NULL_POSITION(io.aeron.archive.client.AeronArchive.NULL_POSITION) LogBufferDescriptor.positionBitsToShift(io.aeron.logbuffer.LogBufferDescriptor.positionBitsToShift) HEADER_LENGTH(io.aeron.protocol.DataHeaderFlyweight.HEADER_LENGTH) VERIFY_ALL_SEGMENT_FILES(io.aeron.archive.ArchiveTool.VerifyOption.VERIFY_ALL_SEGMENT_FILES) StandardOpenOption(java.nio.file.StandardOpenOption) Math.min(java.lang.Math.min) INVALID(io.aeron.archive.codecs.RecordingState.INVALID) VALID(io.aeron.archive.codecs.RecordingState.VALID) Stream(java.util.stream.Stream) LITTLE_ENDIAN(java.nio.ByteOrder.LITTLE_ENDIAN) EpochClock(org.agrona.concurrent.EpochClock) DataHeaderFlyweight(io.aeron.protocol.DataHeaderFlyweight) ReplaySession.isInvalidHeader(io.aeron.archive.ReplaySession.isInvalidHeader) java.util(java.util) Checksums.newInstance(io.aeron.archive.checksum.Checksums.newInstance) Catalog(io.aeron.archive.Catalog) HeaderFlyweight(io.aeron.protocol.HeaderFlyweight) Configuration(io.aeron.driver.Configuration) READ_WRITE(java.nio.channels.FileChannel.MapMode.READ_WRITE) io.aeron.archive.codecs(io.aeron.archive.codecs) MutableInteger(org.agrona.collections.MutableInteger) PrintStream(java.io.PrintStream) FrameDescriptor(io.aeron.logbuffer.FrameDescriptor) Collections.emptySet(java.util.Collections.emptySet) Files(java.nio.file.Files) SESSION_ID_FIELD_OFFSET(io.aeron.protocol.DataHeaderFlyweight.SESSION_ID_FIELD_OFFSET) AeronArchive.segmentFileBasePosition(io.aeron.archive.client.AeronArchive.segmentFileBasePosition) IOException(java.io.IOException) CommonContext(io.aeron.CommonContext) Checksum(io.aeron.archive.checksum.Checksum) AeronException(io.aeron.exceptions.AeronException) File(java.io.File) INSTANCE(org.agrona.concurrent.SystemEpochClock.INSTANCE) TimeUnit(java.util.concurrent.TimeUnit) Consumer(java.util.function.Consumer) APPLY_CHECKSUM(io.aeron.archive.ArchiveTool.VerifyOption.APPLY_CHECKSUM) CncFileDescriptor(io.aeron.CncFileDescriptor) MarkFileHeaderDecoder(io.aeron.archive.codecs.mark.MarkFileHeaderDecoder) HDR_TYPE_DATA(io.aeron.protocol.HeaderFlyweight.HDR_TYPE_DATA) FileChannel(java.nio.channels.FileChannel) BitUtil.align(org.agrona.BitUtil.align) CATALOG_FILE_NAME(io.aeron.archive.Archive.Configuration.CATALOG_FILE_NAME) MappedByteBuffer(java.nio.MappedByteBuffer) LogBufferDescriptor.computeTermIdFromPosition(io.aeron.logbuffer.LogBufferDescriptor.computeTermIdFromPosition) DataHeaderFlyweight(io.aeron.protocol.DataHeaderFlyweight) ByteBuffer(java.nio.ByteBuffer) MappedByteBuffer(java.nio.MappedByteBuffer)

Aggregations

MutableInteger (org.agrona.collections.MutableInteger)151 UnsafeBuffer (org.agrona.concurrent.UnsafeBuffer)76 DirectBuffer (org.agrona.DirectBuffer)64 Test (org.junit.jupiter.api.Test)61 InterruptAfter (io.aeron.test.InterruptAfter)52 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)42 MethodSource (org.junit.jupiter.params.provider.MethodSource)38 MediaDriver (io.aeron.driver.MediaDriver)34 ExtendWith (org.junit.jupiter.api.extension.ExtendWith)34 Test (org.junit.Test)33 ByteBuffer (java.nio.ByteBuffer)32 CloseHelper (org.agrona.CloseHelper)32 Array32FW (io.aklivity.zilla.specs.binding.kafka.internal.types.Array32FW)28 HEADER (io.aklivity.zilla.specs.binding.kafka.internal.types.KafkaConditionType.HEADER)28 HEADERS (io.aklivity.zilla.specs.binding.kafka.internal.types.KafkaConditionType.HEADERS)28 KEY (io.aklivity.zilla.specs.binding.kafka.internal.types.KafkaConditionType.KEY)28 NOT (io.aklivity.zilla.specs.binding.kafka.internal.types.KafkaConditionType.NOT)28 KafkaDeltaType (io.aklivity.zilla.specs.binding.kafka.internal.types.KafkaDeltaType)28 KafkaOffsetFW (io.aklivity.zilla.specs.binding.kafka.internal.types.KafkaOffsetFW)28 KafkaSkip (io.aklivity.zilla.specs.binding.kafka.internal.types.KafkaSkip)28