Search in sources :

Example 6 with Message

use of herddb.network.Message in project herddb by diennea.

the class DataMessageEncoder method write.

@Override
public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) {
    Message m = (Message) msg;
    ByteBuf encoded = ctx.alloc().buffer();
    MessageUtils.encodeMessage(encoded, m);
    ctx.writeAndFlush(encoded, promise);
}
Also used : Message(herddb.network.Message) ByteBuf(io.netty.buffer.ByteBuf)

Example 7 with Message

use of herddb.network.Message in project herddb by diennea.

the class RoutedClientSideConnection method executeGet.

GetResult executeGet(String tableSpace, String query, long tx, List<Object> params) throws HDBException, ClientSideMetadataProviderException {
    Channel _channel = ensureOpen();
    try {
        Message message = Message.EXECUTE_STATEMENT(clientId, tableSpace, query, tx, false, params);
        Message reply = _channel.sendMessageWithReply(message, timeout);
        if (reply.type == Message.TYPE_ERROR) {
            boolean notLeader = reply.parameters.get("notLeader") != null;
            if (notLeader) {
                this.connection.requestMetadataRefresh();
                throw new RetryRequestException(reply + "");
            }
            throw new HDBException(reply);
        }
        long found = (Long) reply.parameters.get("updateCount");
        long transactionId = (Long) reply.parameters.get("tx");
        if (found <= 0) {
            return new GetResult(null, transactionId);
        } else {
            return new GetResult((Map<String, Object>) reply.parameters.get("data"), transactionId);
        }
    } catch (InterruptedException | TimeoutException err) {
        throw new HDBException(err);
    }
}
Also used : Message(herddb.network.Message) Channel(herddb.network.Channel) RetryRequestException(herddb.client.impl.RetryRequestException) AtomicLong(java.util.concurrent.atomic.AtomicLong) TimeoutException(java.util.concurrent.TimeoutException)

Example 8 with Message

use of herddb.network.Message in project herddb by diennea.

the class RoutedClientSideConnection method executeScan.

ScanResultSet executeScan(String tableSpace, String query, List<Object> params, long tx, int maxRows, int fetchSize) throws HDBException, ClientSideMetadataProviderException {
    Channel _channel = ensureOpen();
    try {
        String scannerId = this.clientId + ":" + SCANNERID_GENERATOR.incrementAndGet();
        Message message = Message.OPEN_SCANNER(clientId, tableSpace, query, scannerId, tx, params, fetchSize, maxRows);
        LOGGER.log(Level.FINEST, "open scanner {0} for query {1}, params {2}", new Object[] { scannerId, query, params });
        Message reply = _channel.sendMessageWithReply(message, timeout);
        if (reply.type == Message.TYPE_ERROR) {
            boolean notLeader = reply.parameters.get("notLeader") != null;
            if (notLeader) {
                this.connection.requestMetadataRefresh();
                throw new RetryRequestException(reply + "");
            }
            throw new HDBException(reply);
        }
        TuplesList data = (TuplesList) reply.parameters.get("data");
        List<DataAccessor> initialFetchBuffer = data.tuples;
        String[] columnNames = data.columnNames;
        boolean last = (Boolean) reply.parameters.get("last");
        long transactionId = (Long) reply.parameters.get("tx");
        // LOGGER.log(Level.SEVERE, "received first " + initialFetchBuffer.size() + " records for query " + query);
        ScanResultSetImpl impl = new ScanResultSetImpl(scannerId, columnNames, initialFetchBuffer, fetchSize, last, transactionId);
        return impl;
    } catch (InterruptedException | TimeoutException err) {
        throw new HDBException(err);
    }
}
Also used : Message(herddb.network.Message) DataAccessor(herddb.utils.DataAccessor) Channel(herddb.network.Channel) RetryRequestException(herddb.client.impl.RetryRequestException) AtomicLong(java.util.concurrent.atomic.AtomicLong) TuplesList(herddb.utils.TuplesList) TimeoutException(java.util.concurrent.TimeoutException)

Example 9 with Message

use of herddb.network.Message in project herddb by diennea.

the class TableSpaceManager method sendTransactionsDump.

private void sendTransactionsDump(List<Transaction> batch, Channel _channel, String dumpId, final int timeout, Message response_to_start) throws TimeoutException, InterruptedException {
    if (batch.isEmpty()) {
        return;
    }
    Map<String, Object> transactionsData = new HashMap<>();
    transactionsData.put("command", "transactions");
    List<byte[]> encodedTransactions = batch.stream().map(tr -> {
        return tr.serialize();
    }).collect(Collectors.toList());
    transactionsData.put("transactions", encodedTransactions);
    Message response_to_transactionsData = _channel.sendMessageWithReply(Message.TABLESPACE_DUMP_DATA(null, tableSpaceName, dumpId, transactionsData), timeout);
    if (response_to_transactionsData.type != Message.TYPE_ACK) {
        LOGGER.log(Level.SEVERE, "error response at transactionsData command: " + response_to_start.parameters);
    }
    batch.clear();
}
Also used : HDBException(herddb.client.HDBException) SystablesTableManager(herddb.core.system.SystablesTableManager) CommitTransactionStatement(herddb.model.commands.CommitTransactionStatement) TableCheckpoint(herddb.core.AbstractTableManager.TableCheckpoint) Table(herddb.model.Table) ClientConfiguration(herddb.client.ClientConfiguration) IndexAlreadyExistsException(herddb.model.IndexAlreadyExistsException) Map(java.util.Map) DDLStatementExecutionResult(herddb.model.DDLStatementExecutionResult) LogNotAvailableException(herddb.log.LogNotAvailableException) CommitLogResult(herddb.log.CommitLogResult) ClientSideMetadataProviderException(herddb.client.ClientSideMetadataProviderException) Set(java.util.Set) LogSequenceNumber(herddb.log.LogSequenceNumber) ScanStatement(herddb.model.commands.ScanStatement) HDBClient(herddb.client.HDBClient) DropTableStatement(herddb.model.commands.DropTableStatement) CopyOnWriteArrayList(java.util.concurrent.CopyOnWriteArrayList) Bytes(herddb.utils.Bytes) LogEntry(herddb.log.LogEntry) ReentrantReadWriteLock(java.util.concurrent.locks.ReentrantReadWriteLock) SysnodesTableManager(herddb.core.system.SysnodesTableManager) ArrayList(java.util.ArrayList) CreateIndexStatement(herddb.model.commands.CreateIndexStatement) TransactionContext(herddb.model.TransactionContext) Transaction(herddb.model.Transaction) BRINIndexManager(herddb.index.brin.BRINIndexManager) BiConsumer(java.util.function.BiConsumer) SystablestatsTableManager(herddb.core.system.SystablestatsTableManager) CommitLogListener(herddb.log.CommitLogListener) IndexDoesNotExistException(herddb.model.IndexDoesNotExistException) TableSpaceManagerStats(herddb.core.stats.TableSpaceManagerStats) LogEntryType(herddb.log.LogEntryType) LogEntryFactory(herddb.log.LogEntryFactory) DataStorageManager(herddb.storage.DataStorageManager) DropIndexStatement(herddb.model.commands.DropIndexStatement) AtomicLong(java.util.concurrent.atomic.AtomicLong) FullRecoveryNeededException(herddb.log.FullRecoveryNeededException) ServerHostData(herddb.network.ServerHostData) TableAlreadyExistsException(herddb.model.TableAlreadyExistsException) RollbackTransactionStatement(herddb.model.commands.RollbackTransactionStatement) TimeoutException(java.util.concurrent.TimeoutException) CreateTableStatement(herddb.model.commands.CreateTableStatement) JMXUtils(herddb.jmx.JMXUtils) TransactionResult(herddb.model.TransactionResult) MetadataStorageManager(herddb.metadata.MetadataStorageManager) KeyValue(herddb.network.KeyValue) Channel(herddb.network.Channel) SendResultCallback(herddb.network.SendResultCallback) ServerConfiguration(herddb.server.ServerConfiguration) DataStorageManagerException(herddb.storage.DataStorageManagerException) Index(herddb.model.Index) TableDoesNotExistException(herddb.model.TableDoesNotExistException) AlterTableStatement(herddb.model.commands.AlterTableStatement) DataScanner(herddb.model.DataScanner) DDLException(herddb.model.DDLException) StatementExecutionException(herddb.model.StatementExecutionException) Collection(java.util.Collection) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) BeginTransactionStatement(herddb.model.commands.BeginTransactionStatement) Logger(java.util.logging.Logger) TableAwareStatement(herddb.model.TableAwareStatement) EOFException(java.io.EOFException) Collectors(java.util.stream.Collectors) Objects(java.util.Objects) HDBConnection(herddb.client.HDBConnection) List(java.util.List) Message(herddb.network.Message) FullTableScanConsumer(herddb.storage.FullTableScanConsumer) SystransactionsTableManager(herddb.core.system.SystransactionsTableManager) Entry(java.util.Map.Entry) NodeMetadata(herddb.model.NodeMetadata) Optional(java.util.Optional) TableSpace(herddb.model.TableSpace) SysconfigTableManager(herddb.core.system.SysconfigTableManager) SysindexesTableManager(herddb.core.system.SysindexesTableManager) Statement(herddb.model.Statement) MetadataStorageManagerException(herddb.metadata.MetadataStorageManagerException) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) SystablespacesTableManager(herddb.core.system.SystablespacesTableManager) Level(java.util.logging.Level) HashSet(java.util.HashSet) SysclientsTableManager(herddb.core.system.SysclientsTableManager) DumpedLogEntry(herddb.backup.DumpedLogEntry) StatementExecutionResult(herddb.model.StatementExecutionResult) CommitLog(herddb.log.CommitLog) ConcurrentSkipListSet(java.util.concurrent.ConcurrentSkipListSet) ClientSideMetadataProvider(herddb.client.ClientSideMetadataProvider) SystablespacereplicastateTableManager(herddb.core.system.SystablespacereplicastateTableManager) SQLPlannedOperationStatement(herddb.model.commands.SQLPlannedOperationStatement) StatementEvaluationContext(herddb.model.StatementEvaluationContext) Comparator(java.util.Comparator) SyscolumnsTableManager(herddb.core.system.SyscolumnsTableManager) Collections(java.util.Collections) TableManagerStats(herddb.core.stats.TableManagerStats) MemoryHashIndexManager(herddb.index.MemoryHashIndexManager) Message(herddb.network.Message) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) HashMap(java.util.HashMap)

Example 10 with Message

use of herddb.network.Message in project herddb by diennea.

the class RoutedClientSideConnection method messageReceived.

@Override
@SuppressFBWarnings(value = "SF_SWITCH_NO_DEFAULT")
public void messageReceived(Message message, Channel _channel) {
    switch(message.type) {
        case Message.TYPE_TABLESPACE_DUMP_DATA:
            {
                String dumpId = (String) message.parameters.get("dumpId");
                TableSpaceDumpReceiver receiver = dumpReceivers.get(dumpId);
                LOGGER.log(Level.FINE, "receiver for {0}: {1}", new Object[] { dumpId, receiver });
                if (receiver == null) {
                    if (_channel != null) {
                        _channel.sendReplyMessage(message, Message.ERROR(clientId, new Exception("no such dump receiver " + dumpId)));
                    }
                    return;
                }
                try {
                    Map<String, Object> values = (Map<String, Object>) message.parameters.get("values");
                    String command = (String) values.get("command") + "";
                    boolean sendAck = true;
                    switch(command) {
                        case "start":
                            {
                                long ledgerId = (long) values.get("ledgerid");
                                long offset = (long) values.get("offset");
                                receiver.start(new LogSequenceNumber(ledgerId, offset));
                                break;
                            }
                        case "beginTable":
                            {
                                byte[] tableDefinition = (byte[]) values.get("table");
                                Table table = Table.deserialize(tableDefinition);
                                Long estimatedSize = (Long) values.get("estimatedSize");
                                long dumpLedgerId = (Long) values.get("dumpLedgerid");
                                long dumpOffset = (Long) values.get("dumpOffset");
                                List<byte[]> indexesDef = (List<byte[]>) values.get("indexes");
                                List<Index> indexes = indexesDef.stream().map(Index::deserialize).collect(Collectors.toList());
                                Map<String, Object> stats = new HashMap<>();
                                stats.put("estimatedSize", estimatedSize);
                                stats.put("dumpLedgerId", dumpLedgerId);
                                stats.put("dumpOffset", dumpOffset);
                                receiver.beginTable(new DumpedTableMetadata(table, new LogSequenceNumber(dumpLedgerId, dumpOffset), indexes), stats);
                                break;
                            }
                        case "endTable":
                            {
                                receiver.endTable();
                                break;
                            }
                        case "finish":
                            {
                                long ledgerId = (long) values.get("ledgerid");
                                long offset = (long) values.get("offset");
                                receiver.finish(new LogSequenceNumber(ledgerId, offset));
                                sendAck = false;
                                break;
                            }
                        case "data":
                            {
                                List<KeyValue> data = (List<KeyValue>) values.get("records");
                                List<Record> records = new ArrayList<>(data.size());
                                for (KeyValue kv : data) {
                                    records.add(new Record(new Bytes(kv.key), new Bytes(kv.value)));
                                }
                                receiver.receiveTableDataChunk(records);
                                break;
                            }
                        case "txlog":
                            {
                                List<KeyValue> data = (List<KeyValue>) values.get("records");
                                List<DumpedLogEntry> records = new ArrayList<>(data.size());
                                for (KeyValue kv : data) {
                                    records.add(new DumpedLogEntry(LogSequenceNumber.deserialize(kv.key), kv.value));
                                }
                                receiver.receiveTransactionLogChunk(records);
                                break;
                            }
                        case "transactions":
                            {
                                String tableSpace = (String) values.get("tableSpace");
                                List<byte[]> data = (List<byte[]>) values.get("transactions");
                                List<Transaction> transactions = data.stream().map(array -> {
                                    return Transaction.deserialize(tableSpace, array);
                                }).collect(Collectors.toList());
                                receiver.receiveTransactionsAtDump(transactions);
                                break;
                            }
                        default:
                            throw new DataStorageManagerException("invalid dump command:" + command);
                    }
                    if (_channel != null && sendAck) {
                        _channel.sendReplyMessage(message, Message.ACK(clientId));
                    }
                } catch (DataStorageManagerException error) {
                    LOGGER.log(Level.SEVERE, "error while handling dump data", error);
                    if (_channel != null) {
                        _channel.sendReplyMessage(message, Message.ERROR(clientId, error));
                    }
                }
            }
            break;
    }
}
Also used : Bytes(herddb.utils.Bytes) RetryRequestException(herddb.client.impl.RetryRequestException) TimeoutException(java.util.concurrent.TimeoutException) Table(herddb.model.Table) HashMap(java.util.HashMap) ReentrantReadWriteLock(java.util.concurrent.locks.ReentrantReadWriteLock) KeyValue(herddb.network.KeyValue) ArrayList(java.util.ArrayList) Level(java.util.logging.Level) Channel(herddb.network.Channel) ChannelEventListener(herddb.network.ChannelEventListener) Transaction(herddb.model.Transaction) Map(java.util.Map) DataStorageManagerException(herddb.storage.DataStorageManagerException) DumpedLogEntry(herddb.backup.DumpedLogEntry) Index(herddb.model.Index) DataAccessor(herddb.utils.DataAccessor) Record(herddb.model.Record) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) LogSequenceNumber(herddb.log.LogSequenceNumber) Logger(java.util.logging.Logger) Collectors(java.util.stream.Collectors) AtomicLong(java.util.concurrent.atomic.AtomicLong) List(java.util.List) DumpedTableMetadata(herddb.backup.DumpedTableMetadata) Message(herddb.network.Message) SaslNettyClient(herddb.security.sasl.SaslNettyClient) BackupFileConstants(herddb.backup.BackupFileConstants) SaslUtils(herddb.security.sasl.SaslUtils) TuplesList(herddb.utils.TuplesList) SuppressFBWarnings(edu.umd.cs.findbugs.annotations.SuppressFBWarnings) ServerHostData(herddb.network.ServerHostData) DataStorageManagerException(herddb.storage.DataStorageManagerException) KeyValue(herddb.network.KeyValue) DumpedLogEntry(herddb.backup.DumpedLogEntry) HashMap(java.util.HashMap) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) ArrayList(java.util.ArrayList) Index(herddb.model.Index) DumpedTableMetadata(herddb.backup.DumpedTableMetadata) Bytes(herddb.utils.Bytes) ArrayList(java.util.ArrayList) List(java.util.List) TuplesList(herddb.utils.TuplesList) Record(herddb.model.Record) Table(herddb.model.Table) LogSequenceNumber(herddb.log.LogSequenceNumber) RetryRequestException(herddb.client.impl.RetryRequestException) TimeoutException(java.util.concurrent.TimeoutException) DataStorageManagerException(herddb.storage.DataStorageManagerException) Transaction(herddb.model.Transaction) AtomicLong(java.util.concurrent.atomic.AtomicLong) HashMap(java.util.HashMap) Map(java.util.Map) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) SuppressFBWarnings(edu.umd.cs.findbugs.annotations.SuppressFBWarnings)

Aggregations

Message (herddb.network.Message)22 StatementExecutionException (herddb.model.StatementExecutionException)10 NotLeaderException (herddb.model.NotLeaderException)9 ArrayList (java.util.ArrayList)9 TuplesList (herddb.utils.TuplesList)8 HerdDBInternalException (herddb.core.HerdDBInternalException)7 DataScannerException (herddb.model.DataScannerException)7 DuplicatePrimaryKeyException (herddb.model.DuplicatePrimaryKeyException)7 List (java.util.List)7 AtomicLong (java.util.concurrent.atomic.AtomicLong)7 HashMap (java.util.HashMap)6 Table (herddb.model.Table)5 Record (herddb.model.Record)4 TransactionContext (herddb.model.TransactionContext)4 SQLPlannedOperationStatement (herddb.model.commands.SQLPlannedOperationStatement)4 ScanStatement (herddb.model.commands.ScanStatement)4 Channel (herddb.network.Channel)4 KeyValue (herddb.network.KeyValue)4 DataAccessor (herddb.utils.DataAccessor)4 Map (java.util.Map)4