Search in sources :

Example 51 with JournalException

use of com.questdb.std.ex.JournalException in project questdb by bluestreak01.

the class JournalIterators method createRanges.

private static <T> ObjList<JournalIteratorRange> createRanges(Journal<T> journal, long lo) {
    ObjList<JournalIteratorRange> ranges = new ObjList<>();
    int loPartitionID = Rows.toPartitionIndex(lo);
    long loLocalRowID = Rows.toLocalRowID(lo);
    try {
        int count = journal.getPartitionCount();
        for (int i = loPartitionID; i < count; i++) {
            long localRowID = 0;
            if (i == loPartitionID) {
                localRowID = loLocalRowID;
            }
            Partition<T> p = journal.getPartition(i, true);
            long size = p.size();
            if (size > 0) {
                ranges.add(new JournalIteratorRange(p.getPartitionIndex(), localRowID, size - 1));
            }
        }
        return ranges;
    } catch (JournalException e) {
        throw new JournalRuntimeException(e);
    }
}
Also used : ObjList(com.questdb.std.ObjList) JournalException(com.questdb.std.ex.JournalException) JournalRuntimeException(com.questdb.common.JournalRuntimeException)

Example 52 with JournalException

use of com.questdb.std.ex.JournalException in project questdb by bluestreak01.

the class Files method readStringFromFile.

public static String readStringFromFile(File file) throws JournalException {
    try {
        try (FileInputStream fis = new FileInputStream(file)) {
            byte[] buffer = new byte[(int) fis.getChannel().size()];
            int totalRead = 0;
            int read;
            while (totalRead < buffer.length && (read = fis.read(buffer, totalRead, buffer.length - totalRead)) > 0) {
                totalRead += read;
            }
            return new String(buffer, UTF_8);
        }
    } catch (IOException e) {
        throw new JournalException("Cannot read from %s", e, file.getAbsolutePath());
    }
}
Also used : JournalException(com.questdb.std.ex.JournalException) IOException(java.io.IOException) FileInputStream(java.io.FileInputStream)

Example 53 with JournalException

use of com.questdb.std.ex.JournalException in project questdb by bluestreak01.

the class JournalClient method subscribeOne.

private void subscribeOne(int index, SubscriptionHolder holder, String name, boolean newSubscription) {
    if (newSubscription) {
        SubscriptionHolder sub = new SubscriptionHolder();
        sub.local = holder.local;
        sub.remote = holder.remote;
        sub.listener = holder.listener;
        sub.writer = holder.writer;
        subscriptions.add(sub);
    }
    JournalWriter<?> writer = writers.getQuiet(index);
    try {
        commandProducer.write(channel, Command.ADD_KEY_CMD);
        setKeyRequestProducer.write(channel, new IndexedJournalKey(index, holder.remote));
        checkAck();
        // todo: do we really have to use file here?
        JournalMetadata<?> metadata;
        File file = Files.makeTempFile();
        try {
            try (HugeBufferConsumer h = new HugeBufferConsumer(file)) {
                h.read(channel);
                metadata = new JournalMetadata(h.getHb(), name);
            } catch (JournalException e) {
                throw new JournalNetworkException(e);
            }
        } finally {
            Files.delete(file);
        }
        boolean validate = true;
        if (writer == null) {
            if (holder.writer == null) {
                try {
                    writer = factory.writer(metadata);
                } catch (JournalException e) {
                    LOG.error().$("Failed to create writer: ").$(e).$();
                    unsubscribe(index, null, holder, JournalEvents.EVT_JNL_INCOMPATIBLE);
                    return;
                }
                writersToClose.add(writer);
                validate = false;
            } else {
                writer = holder.writer;
            }
            writer.disableCommitOnClose();
            statusSentList.extendAndSet(index, 0);
            deltaConsumers.extendAndSet(index, new JournalDeltaConsumer(writer));
            writers.extendAndSet(index, writer);
            writer.setJournalListener(holder.listener);
        } else {
            statusSentList.setQuick(index, 0);
        }
        if (validate && !metadata.isCompatible(writer.getMetadata(), false)) {
            LOG.error().$("Journal ").$(holder.local.getName()).$(" is not compatible with ").$(holder.remote.getName()).$("(remote)").$();
            unsubscribe(index, writer, holder, JournalEvents.EVT_JNL_INCOMPATIBLE);
            return;
        }
        commandProducer.write(channel, Command.DELTA_REQUEST_CMD);
        journalClientStateProducer.write(channel, new IndexedJournal(index, writer));
        checkAck();
        statusSentList.setQuick(index, 1);
        if (holder.listener != null) {
            holder.listener.onEvent(JournalEvents.EVT_JNL_SUBSCRIBED);
        }
        LOG.info().$("Subscribed ").$(name).$(" to ").$(holder.remote.getName()).$("(remote)").$();
    } catch (JournalNetworkException e) {
        LOG.error().$("Failed to subscribe ").$(name).$(" to ").$(holder.remote.getName()).$("(remote)").$();
        unsubscribe(index, writer, holder, JournalEvents.EVT_JNL_SERVER_ERROR);
    }
}
Also used : JournalMetadata(com.questdb.store.factory.configuration.JournalMetadata) IncompatibleJournalException(com.questdb.ex.IncompatibleJournalException) JournalException(com.questdb.std.ex.JournalException) JournalDeltaConsumer(com.questdb.net.ha.comsumer.JournalDeltaConsumer) IndexedJournalKey(com.questdb.net.ha.model.IndexedJournalKey) JournalNetworkException(com.questdb.std.ex.JournalNetworkException) HugeBufferConsumer(com.questdb.net.ha.comsumer.HugeBufferConsumer) IndexedJournal(com.questdb.net.ha.model.IndexedJournal) File(java.io.File)

Example 54 with JournalException

use of com.questdb.std.ex.JournalException in project questdb by bluestreak01.

the class JournalDeltaConsumer method doRead.

@Override
@SuppressWarnings("unchecked")
protected void doRead(ReadableByteChannel channel) throws JournalNetworkException {
    try {
        reset();
        journalServerStateConsumer.read(channel);
        this.state = journalServerStateConsumer.getValue();
        if (state.getTxn() == -1) {
            journal.notifyListener(JournalEvents.EVT_JNL_TRANSACTION_REFUSED);
            throw new IncompatibleJournalException("Server refused txn for %s", journal.getLocation());
        }
        if (state.getTxn() < journal.getTxn()) {
            journal.rollback(state.getTxn(), state.getTxPin());
            return;
        }
        journal.beginTx();
        createPartitions(state);
        if (state.isSymbolTables()) {
            journalSymbolTableConsumer.read(channel);
        }
        for (int i = 0, k = state.getNonLagPartitionCount(); i < k; i++) {
            JournalServerState.PartitionMetadata meta = state.getMeta(i);
            if (meta.getEmpty() == 0) {
                PartitionDeltaConsumer partitionDeltaConsumer = getPartitionDeltaConsumer(meta.getPartitionIndex());
                partitionDeltaConsumer.read(channel);
            }
        }
        if (state.getLagPartitionName() == null && journal.hasIrregularPartition()) {
            // delete lag partition
            journal.removeIrregularPartition();
        } else if (state.getLagPartitionName() != null) {
            if (lagPartitionDeltaConsumer == null || !journal.hasIrregularPartition() || !state.getLagPartitionName().equals(journal.getIrregularPartition().getName())) {
                Partition temp = journal.createTempPartition(state.getLagPartitionName());
                lagPartitionDeltaConsumer = new PartitionDeltaConsumer(temp.open());
                journal.setIrregularPartition(temp);
            }
            lagPartitionDeltaConsumer.read(channel);
        }
    } catch (JournalException e) {
        throw new JournalNetworkException(e);
    }
}
Also used : Partition(com.questdb.store.Partition) JournalException(com.questdb.std.ex.JournalException) IncompatibleJournalException(com.questdb.ex.IncompatibleJournalException) JournalNetworkException(com.questdb.std.ex.JournalNetworkException) JournalServerState(com.questdb.net.ha.model.JournalServerState) IncompatibleJournalException(com.questdb.ex.IncompatibleJournalException)

Example 55 with JournalException

use of com.questdb.std.ex.JournalException in project questdb by bluestreak01.

the class AllRowSource method prepareCursor.

@Override
public RowCursor prepareCursor(PartitionSlice slice) {
    try {
        this.lo = slice.lo;
        this.hi = slice.calcHi ? slice.partition.open().size() - 1 : slice.hi;
        return this;
    } catch (JournalException e) {
        throw new JournalRuntimeException(e);
    }
}
Also used : JournalException(com.questdb.std.ex.JournalException) JournalRuntimeException(com.questdb.common.JournalRuntimeException)

Aggregations

JournalException (com.questdb.std.ex.JournalException)63 JournalRuntimeException (com.questdb.common.JournalRuntimeException)29 AbstractTest (com.questdb.test.tools.AbstractTest)14 Test (org.junit.Test)13 KVIndex (com.questdb.store.KVIndex)12 Partition (com.questdb.store.Partition)9 Quote (com.questdb.model.Quote)8 IndexCursor (com.questdb.store.IndexCursor)8 File (java.io.File)8 CountDownLatch (java.util.concurrent.CountDownLatch)7 IncompatibleJournalException (com.questdb.ex.IncompatibleJournalException)5 JournalStructure (com.questdb.store.factory.configuration.JournalStructure)5 ObjList (com.questdb.std.ObjList)4 JournalWriter (com.questdb.store.JournalWriter)4 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)4 ClientConfig (com.questdb.net.ha.config.ClientConfig)3 FixedColumn (com.questdb.store.FixedColumn)3 Factory (com.questdb.store.factory.Factory)3 ColumnMetadata (com.questdb.store.factory.configuration.ColumnMetadata)3 JournalLockedException (com.questdb.ex.JournalLockedException)2