Search in sources :

Example 16 with TableDoesNotExistException

use of herddb.model.TableDoesNotExistException in project herddb by diennea.

the class AutoTransactionTest method testAutoTransactionOnErrorScan.

@Test
public void testAutoTransactionOnErrorScan() throws Exception {
    int i = 1;
    Map<String, Object> data = new HashMap<>();
    Bytes key = Bytes.from_string("key_" + i);
    data.put("id", "key_" + i);
    data.put("number", i);
    Record record = RecordSerializer.toRecord(data, table);
    InsertStatement st = new InsertStatement(tableSpace, tableName, record);
    DMLStatementExecutionResult executeUpdate = manager.executeUpdate(st, StatementEvaluationContext.DEFAULT_EVALUATION_CONTEXT(), TransactionContext.NO_TRANSACTION);
    long tx;
    try (DataScanner scan = manager.scan(new ScanStatement(tableSpace, "no_table", Projection.IDENTITY(table.columnNames, table.getColumns()), null, null, null), StatementEvaluationContext.DEFAULT_EVALUATION_CONTEXT(), TransactionContext.AUTOTRANSACTION_TRANSACTION)) {
        fail();
    } catch (TableDoesNotExistException ok) {
    }
    assertTrue(manager.getTableSpaceManager(tableSpace).getOpenTransactions().isEmpty());
}
Also used : HashMap(java.util.HashMap) InsertStatement(herddb.model.commands.InsertStatement) TableDoesNotExistException(herddb.model.TableDoesNotExistException) Bytes(herddb.utils.Bytes) DataScanner(herddb.model.DataScanner) DMLStatementExecutionResult(herddb.model.DMLStatementExecutionResult) Record(herddb.model.Record) ScanStatement(herddb.model.commands.ScanStatement) Test(org.junit.Test)

Example 17 with TableDoesNotExistException

use of herddb.model.TableDoesNotExistException in project herddb by diennea.

the class JSQLParserPlanner method getTableSchema.

private OpSchema getTableSchema(String defaultTableSpace, net.sf.jsqlparser.schema.Table table) {
    String tableSpace = table.getSchemaName();
    if (tableSpace == null) {
        tableSpace = defaultTableSpace;
    }
    tableSpace = fixMySqlBackTicks(tableSpace);
    TableSpaceManager tableSpaceManager = getTableSpaceManager(tableSpace);
    if (tableSpaceManager == null) {
        clearCache();
        throw new StatementExecutionException("tablespace " + defaultTableSpace + " is not available");
    }
    String tableName = fixMySqlBackTicks(table.getName().toLowerCase());
    AbstractTableManager tableManager = tableSpaceManager.getTableManager(tableName);
    if (tableManager == null) {
        throw new TableDoesNotExistException("no table " + tableName + " here for " + tableSpace);
    }
    Table tableImpl = tableManager.getTable();
    String aliasTable = tableName;
    if (table.getAlias() != null) {
        aliasTable = fixMySqlBackTicks(table.getAlias().getName());
        checkSupported(table.getAlias().getAliasColumns() == null);
    }
    ColumnRef[] refs = new ColumnRef[tableImpl.columns.length];
    for (int i = 0; i < refs.length; i++) {
        refs[i] = new ColumnRef(aliasTable, tableImpl.columns[i]);
    }
    return new OpSchema(tableSpace, tableName, aliasTable, tableImpl.columnNames, refs);
}
Also used : TableDoesNotExistException(herddb.model.TableDoesNotExistException) Table(herddb.model.Table) ShowCreateTableCalculator.calculateShowCreateTable(herddb.sql.functions.ShowCreateTableCalculator.calculateShowCreateTable) CreateTable(net.sf.jsqlparser.statement.create.table.CreateTable) AbstractTableManager(herddb.core.AbstractTableManager) TableSpaceManager(herddb.core.TableSpaceManager) ColumnRef(herddb.sql.expressions.ColumnRef) StatementExecutionException(herddb.model.StatementExecutionException) OpSchema(herddb.sql.expressions.OpSchema)

Example 18 with TableDoesNotExistException

use of herddb.model.TableDoesNotExistException in project herddb by diennea.

the class ShowCreateTableCalculator method calculateShowCreateTable.

public static TranslatedQuery calculateShowCreateTable(String query, String defaultTablespace, List<Object> parameters, DBManager manager) {
    String[] items = { "SHOW", "CREATE", "TABLE" };
    if (Arrays.stream(items).allMatch(query::contains)) {
        query = query.substring(Arrays.stream(items).collect(Collectors.joining(" ")).length()).trim();
        String tableSpace = defaultTablespace;
        String tableName;
        boolean showCreateIndex = query.contains("WITH INDEXES");
        if (showCreateIndex) {
            query = query.substring(0, query.indexOf("WITH INDEXES"));
        }
        if (query.contains(".")) {
            String[] tokens = query.split("\\.");
            tableSpace = tokens[0].trim();
            tableName = tokens[1].trim();
        } else {
            tableName = query.trim();
        }
        tableName = tableName.toLowerCase();
        TableSpaceManager tableSpaceManager = manager.getTableSpaceManager(tableSpace);
        if (tableSpaceManager == null) {
            throw new TableSpaceDoesNotExistException(String.format("Tablespace %s does not exist.", tableSpace));
        }
        AbstractTableManager tableManager = tableSpaceManager.getTableManager(tableName);
        if (tableManager == null || tableManager.getCreatedInTransaction() > 0) {
            throw new TableDoesNotExistException(String.format("Table %s does not exist.", tableName));
        }
        String showCreateResult = calculate(showCreateIndex, tableName, tableSpace, tableManager);
        ValuesOp values = new ValuesOp(manager.getNodeId(), new String[] { "tabledef" }, new Column[] { column("tabledef", ColumnTypes.STRING) }, Arrays.asList(Arrays.asList(new ConstantExpression(showCreateResult, ColumnTypes.NOTNULL_STRING))));
        ExecutionPlan executionPlan = ExecutionPlan.simple(new SQLPlannedOperationStatement(values), values);
        return new TranslatedQuery(executionPlan, new SQLStatementEvaluationContext(query, parameters, false, false));
    } else {
        throw new StatementExecutionException(String.format("Incorrect Syntax for SHOW CREATE TABLE tablespace.tablename"));
    }
}
Also used : TranslatedQuery(herddb.sql.TranslatedQuery) TableSpaceDoesNotExistException(herddb.model.TableSpaceDoesNotExistException) ConstantExpression(herddb.sql.expressions.ConstantExpression) SQLStatementEvaluationContext(herddb.sql.SQLStatementEvaluationContext) ValuesOp(herddb.model.planner.ValuesOp) StatementExecutionException(herddb.model.StatementExecutionException) SQLPlannedOperationStatement(herddb.model.commands.SQLPlannedOperationStatement) TableDoesNotExistException(herddb.model.TableDoesNotExistException) ExecutionPlan(herddb.model.ExecutionPlan) AbstractTableManager(herddb.core.AbstractTableManager) TableSpaceManager(herddb.core.TableSpaceManager)

Example 19 with TableDoesNotExistException

use of herddb.model.TableDoesNotExistException in project herddb by diennea.

the class TableSpaceManager method createAndWriteTableCheksum.

// this method return a tableCheckSum object contain scan values (record numbers , table digest,digestType, next autoincrement value, table name, tablespacename, query used for table scan )
public TableChecksum createAndWriteTableCheksum(TableSpaceManager tableSpaceManager, String tableSpaceName, String tableName, StatementEvaluationContext context) throws IOException, DataScannerException {
    CommitLogResult pos;
    boolean lockAcquired = false;
    if (context == null) {
        context = StatementEvaluationContext.DEFAULT_EVALUATION_CONTEXT();
    }
    long lockStamp = context.getTableSpaceLock();
    LOGGER.log(Level.INFO, "Create and write table {0} checksum in tablespace ", new Object[] { tableName, tableSpaceName });
    if (lockStamp == 0) {
        lockStamp = acquireWriteLock("checkDataConsistency_" + tableName);
        context.setTableSpaceLock(lockStamp);
        lockAcquired = true;
    }
    try {
        AbstractTableManager tablemanager = tableSpaceManager.getTableManager(tableName);
        if (tableSpaceManager == null) {
            throw new TableSpaceDoesNotExistException(String.format("Tablespace %s does not exist.", tableSpaceName));
        }
        if (tablemanager == null || tablemanager.getCreatedInTransaction() > 0) {
            throw new TableDoesNotExistException(String.format("Table %s does not exist.", tablemanager));
        }
        TableChecksum scanResult = TableDataChecksum.createChecksum(tableSpaceManager.getDbmanager(), null, tableSpaceManager, tableSpaceName, tableName);
        byte[] serialize = MAPPER.writeValueAsBytes(scanResult);
        Bytes value = Bytes.from_array(serialize);
        LogEntry entry = LogEntryFactory.dataConsistency(tableName, value);
        pos = log.log(entry, false);
        apply(pos, entry, false);
        return scanResult;
    } finally {
        if (lockAcquired) {
            releaseWriteLock(context.getTableSpaceLock(), "checkDataConsistency");
            context.setTableSpaceLock(0);
        }
    }
}
Also used : TableDoesNotExistException(herddb.model.TableDoesNotExistException) Bytes(herddb.utils.Bytes) TableSpaceDoesNotExistException(herddb.model.TableSpaceDoesNotExistException) CommitLogResult(herddb.log.CommitLogResult) TableChecksum(herddb.data.consistency.TableChecksum) LogEntry(herddb.log.LogEntry) DumpedLogEntry(herddb.backup.DumpedLogEntry)

Example 20 with TableDoesNotExistException

use of herddb.model.TableDoesNotExistException in project herddb by diennea.

the class TableSpaceManager method apply.

void apply(CommitLogResult position, LogEntry entry, boolean recovery) throws DataStorageManagerException, DDLException {
    if (!position.deferred || position.sync) {
        // this will wait for the write to be acknowledged by the log
        // it can throw LogNotAvailableException
        this.actualLogSequenceNumber = position.getLogSequenceNumber();
        if (LOGGER.isLoggable(Level.FINEST)) {
            LOGGER.log(Level.FINEST, "apply {0} {1}", new Object[] { position.getLogSequenceNumber(), entry });
        }
    } else {
        if (LOGGER.isLoggable(Level.FINEST)) {
            LOGGER.log(Level.FINEST, "apply {0} {1}", new Object[] { position, entry });
        }
    }
    switch(entry.type) {
        case LogEntryType.NOOP:
            {
            // NOOP
            }
            break;
        case LogEntryType.BEGINTRANSACTION:
            {
                long id = entry.transactionId;
                Transaction transaction = new Transaction(id, tableSpaceName, position);
                transactions.put(id, transaction);
            }
            break;
        case LogEntryType.ROLLBACKTRANSACTION:
            {
                long id = entry.transactionId;
                Transaction transaction = transactions.get(id);
                if (transaction == null) {
                    throw new DataStorageManagerException("invalid transaction id " + id + ", only " + transactions.keySet());
                }
                List<AbstractIndexManager> indexManagers = new ArrayList<>(indexes.values());
                for (AbstractIndexManager indexManager : indexManagers) {
                    if (indexManager.getCreatedInTransaction() == 0 || indexManager.getCreatedInTransaction() == id) {
                        indexManager.onTransactionRollback(transaction);
                    }
                }
                List<AbstractTableManager> managers = new ArrayList<>(tables.values());
                for (AbstractTableManager manager : managers) {
                    if (manager.getCreatedInTransaction() == 0 || manager.getCreatedInTransaction() == id) {
                        Table table = manager.getTable();
                        if (transaction.isNewTable(table.name)) {
                            LOGGER.log(Level.INFO, "rollback CREATE TABLE " + table.tablespace + "." + table.name);
                            disposeTable(manager);
                            Map<String, AbstractIndexManager> indexes = indexesByTable.remove(manager.getTable().name);
                            if (indexes != null) {
                                for (AbstractIndexManager indexManager : indexes.values()) {
                                    disposeIndexManager(indexManager);
                                }
                            }
                        } else {
                            manager.onTransactionRollback(transaction);
                        }
                    }
                }
                transactions.remove(transaction.transactionId);
            }
            break;
        case LogEntryType.COMMITTRANSACTION:
            {
                long id = entry.transactionId;
                Transaction transaction = transactions.get(id);
                if (transaction == null) {
                    throw new DataStorageManagerException("invalid transaction id " + id);
                }
                LogSequenceNumber commit = position.getLogSequenceNumber();
                transaction.sync(commit);
                List<AbstractTableManager> managers = new ArrayList<>(tables.values());
                for (AbstractTableManager manager : managers) {
                    if (manager.getCreatedInTransaction() == 0 || manager.getCreatedInTransaction() == id) {
                        manager.onTransactionCommit(transaction, recovery);
                    }
                }
                List<AbstractIndexManager> indexManagers = new ArrayList<>(indexes.values());
                for (AbstractIndexManager indexManager : indexManagers) {
                    if (indexManager.getCreatedInTransaction() == 0 || indexManager.getCreatedInTransaction() == id) {
                        indexManager.onTransactionCommit(transaction, recovery);
                    }
                }
                if ((transaction.droppedTables != null && !transaction.droppedTables.isEmpty()) || (transaction.droppedIndexes != null && !transaction.droppedIndexes.isEmpty())) {
                    if (transaction.droppedTables != null) {
                        for (String dropped : transaction.droppedTables) {
                            for (AbstractTableManager manager : managers) {
                                if (manager.getTable().name.equals(dropped)) {
                                    disposeTable(manager);
                                }
                            }
                        }
                    }
                    if (transaction.droppedIndexes != null) {
                        for (String dropped : transaction.droppedIndexes) {
                            for (AbstractIndexManager manager : indexManagers) {
                                if (manager.getIndex().name.equals(dropped)) {
                                    disposeIndexManager(manager);
                                }
                            }
                        }
                    }
                }
                if ((transaction.newTables != null && !transaction.newTables.isEmpty()) || (transaction.droppedTables != null && !transaction.droppedTables.isEmpty()) || (transaction.newIndexes != null && !transaction.newIndexes.isEmpty()) || (transaction.droppedIndexes != null && !transaction.droppedIndexes.isEmpty())) {
                    writeTablesOnDataStorageManager(position, false);
                    dbmanager.getPlanner().clearCache();
                }
                transactions.remove(transaction.transactionId);
            }
            break;
        case LogEntryType.CREATE_TABLE:
            {
                Table table = Table.deserialize(entry.value.to_array());
                if (entry.transactionId > 0) {
                    long id = entry.transactionId;
                    Transaction transaction = transactions.get(id);
                    transaction.registerNewTable(table, position);
                }
                bootTable(table, entry.transactionId, null, true);
                if (entry.transactionId <= 0) {
                    writeTablesOnDataStorageManager(position, false);
                }
            }
            break;
        case LogEntryType.CREATE_INDEX:
            {
                Index index = Index.deserialize(entry.value.to_array());
                if (entry.transactionId > 0) {
                    long id = entry.transactionId;
                    Transaction transaction = transactions.get(id);
                    transaction.registerNewIndex(index, position);
                }
                AbstractTableManager tableManager = tables.get(index.table);
                if (tableManager == null) {
                    throw new RuntimeException("table " + index.table + " does not exists");
                }
                bootIndex(index, tableManager, true, entry.transactionId, true, false);
                if (entry.transactionId <= 0) {
                    writeTablesOnDataStorageManager(position, false);
                }
            }
            break;
        case LogEntryType.DROP_TABLE:
            {
                String tableName = entry.tableName;
                if (entry.transactionId > 0) {
                    long id = entry.transactionId;
                    Transaction transaction = transactions.get(id);
                    transaction.registerDropTable(tableName, position);
                } else {
                    AbstractTableManager manager = tables.get(tableName);
                    if (manager != null) {
                        disposeTable(manager);
                        Map<String, AbstractIndexManager> indexes = indexesByTable.get(tableName);
                        if (indexes != null && !indexes.isEmpty()) {
                            LOGGER.log(Level.SEVERE, "It looks like we are dropping a table " + tableName + " with these indexes " + indexes);
                        }
                    }
                }
                if (entry.transactionId <= 0) {
                    writeTablesOnDataStorageManager(position, false);
                }
            }
            break;
        case LogEntryType.DROP_INDEX:
            {
                String indexName = entry.value.to_string();
                if (entry.transactionId > 0) {
                    long id = entry.transactionId;
                    Transaction transaction = transactions.get(id);
                    transaction.registerDropIndex(indexName, position);
                } else {
                    AbstractIndexManager manager = indexes.get(indexName);
                    if (manager != null) {
                        disposeIndexManager(manager);
                    }
                }
                if (entry.transactionId <= 0) {
                    writeTablesOnDataStorageManager(position, false);
                    dbmanager.getPlanner().clearCache();
                }
            }
            break;
        case LogEntryType.ALTER_TABLE:
            {
                Table table = Table.deserialize(entry.value.to_array());
                alterTable(table, null);
                writeTablesOnDataStorageManager(position, false);
            }
            break;
        case LogEntryType.TABLE_CONSISTENCY_CHECK:
            {
                /*
                    In recovery mode, we need to skip the consistency check.
                    The tablespace may not be avaible yet and therefore calcite will not able to performed the select query.
                */
                if (recovery) {
                    LOGGER.log(Level.INFO, "skip {0} consistency check LogEntry {1}", new Object[] { tableSpaceName, entry });
                    break;
                }
                try {
                    TableChecksum check = MAPPER.readValue(entry.value.to_array(), TableChecksum.class);
                    String tableSpace = check.getTableSpaceName();
                    String query = check.getQuery();
                    String tableName = entry.tableName;
                    // In the entry type = 14, the follower will have to run the query on the transaction log
                    if (!isLeader()) {
                        AbstractTableManager tablemanager = this.getTableManager(tableName);
                        DBManager manager = this.getDbmanager();
                        if (tablemanager == null || tablemanager.getCreatedInTransaction() > 0) {
                            throw new TableDoesNotExistException(String.format("Table %s does not exist.", tablemanager));
                        }
                        /*
                            scan = true
                            allowCache = false
                            returnValues = false
                            maxRows = -1
                        */
                        TranslatedQuery translated = manager.getPlanner().translate(tableSpace, query, Collections.emptyList(), true, false, false, -1);
                        TableChecksum scanResult = TableDataChecksum.createChecksum(manager, translated, this, tableSpace, tableName);
                        long followerDigest = scanResult.getDigest();
                        long leaderDigest = check.getDigest();
                        long leaderNumRecords = check.getNumRecords();
                        long followerNumRecords = scanResult.getNumRecords();
                        // the necessary condition to pass the check is to have exactly the same digest and the number of records processed
                        if (followerDigest == leaderDigest && leaderNumRecords == followerNumRecords) {
                            LOGGER.log(Level.INFO, "Data consistency check PASS for table {0}  tablespace {1} with  Checksum {2}", new Object[] { tableName, tableSpace, followerDigest });
                        } else {
                            LOGGER.log(Level.SEVERE, "Data consistency check FAILED for table {0} in tablespace {1} with Checksum {2}", new Object[] { tableName, tableSpace, followerDigest });
                        }
                    } else {
                        long digest = check.getDigest();
                        LOGGER.log(Level.INFO, "Created checksum {0}  for table {1} in tablespace {2} on node {3}", new Object[] { digest, entry.tableName, tableSpace, this.getDbmanager().getNodeId() });
                    }
                } catch (IOException | DataScannerException ex) {
                    LOGGER.log(Level.SEVERE, "Error during table consistency check ", ex);
                }
            }
            break;
        default:
            // other entry types are not important for the tablespacemanager
            break;
    }
    if (entry.tableName != null && entry.type != LogEntryType.CREATE_TABLE && entry.type != LogEntryType.CREATE_INDEX && entry.type != LogEntryType.ALTER_TABLE && entry.type != LogEntryType.DROP_TABLE && entry.type != LogEntryType.TABLE_CONSISTENCY_CHECK) {
        AbstractTableManager tableManager = tables.get(entry.tableName);
        tableManager.apply(position, entry, recovery);
    }
}
Also used : DataStorageManagerException(herddb.storage.DataStorageManagerException) Table(herddb.model.Table) TranslatedQuery(herddb.sql.TranslatedQuery) LogSequenceNumber(herddb.log.LogSequenceNumber) Index(herddb.model.Index) TableDoesNotExistException(herddb.model.TableDoesNotExistException) Transaction(herddb.model.Transaction) CopyOnWriteArrayList(java.util.concurrent.CopyOnWriteArrayList) ArrayList(java.util.ArrayList) List(java.util.List) TableChecksum(herddb.data.consistency.TableChecksum) Map(java.util.Map) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) HashMap(java.util.HashMap)

Aggregations

TableDoesNotExistException (herddb.model.TableDoesNotExistException)26 StatementExecutionException (herddb.model.StatementExecutionException)16 Table (herddb.model.Table)11 Test (org.junit.Test)10 AbstractTableManager (herddb.core.AbstractTableManager)7 DataScanner (herddb.model.DataScanner)7 TableSpaceDoesNotExistException (herddb.model.TableSpaceDoesNotExistException)7 CreateTableSpaceStatement (herddb.model.commands.CreateTableSpaceStatement)7 DataStorageManagerException (herddb.storage.DataStorageManagerException)7 DumpedLogEntry (herddb.backup.DumpedLogEntry)6 CommitLogResult (herddb.log.CommitLogResult)6 LogEntry (herddb.log.LogEntry)6 LogNotAvailableException (herddb.log.LogNotAvailableException)6 MemoryCommitLogManager (herddb.mem.MemoryCommitLogManager)6 MemoryMetadataStorageManager (herddb.mem.MemoryMetadataStorageManager)6 DDLStatementExecutionResult (herddb.model.DDLStatementExecutionResult)6 IndexAlreadyExistsException (herddb.model.IndexAlreadyExistsException)6 IndexDoesNotExistException (herddb.model.IndexDoesNotExistException)6 TransactionContext (herddb.model.TransactionContext)6 Bytes (herddb.utils.Bytes)6