Search in sources :

Example 26 with NewMycatConnection

use of io.mycat.newquery.NewMycatConnection in project Mycat2 by MyCATApache.

the class MycatViewPlan method execute.

@Override
public Observable<VectorSchemaRoot> execute(RootContext rootContext) {
    RootAllocator rootAllocator = new RootAllocator();
    NewMycatDataContext context = (NewMycatDataContext) rootContext.getContext();
    DrdsSqlWithParams drdsSql = context.getDrdsSql();
    MycatView view = (MycatView) mycatView;
    CopyMycatRowMetaData rowMetaData = new CopyMycatRowMetaData(new CalciteRowMetaData(view.getRowType().getFieldList()));
    MycatRelDatasourceSourceInfo mycatRelDatasourceSourceInfo = new MycatRelDatasourceSourceInfo(rowMetaData, view.getSQLTemplate(DrdsSql.isForUpdate(drdsSql.getParameterizedStatement())), view);
    SqlNode sqlTemplate = mycatRelDatasourceSourceInfo.getSqlTemplate();
    List<PartitionGroup> partitionGroups = AsyncMycatDataContextImpl.getSqlMap(Collections.emptyMap(), view, drdsSql, drdsSql.getHintDataNodeFilter());
    ImmutableMultimap<String, SqlString> stringSqlStringImmutableMultimap = view.apply(-1, sqlTemplate, partitionGroups, drdsSql.getParams());
    List<Observable<VectorSchemaRoot>> observableList = new ArrayList<>();
    for (Map.Entry<String, SqlString> entry : stringSqlStringImmutableMultimap.entries()) {
        String key = entry.getKey();
        SqlString sqlString = entry.getValue();
        observableList.add(Observable.create(emitter -> {
            Future<NewMycatConnection> connectionFuture = context.getConnection(context.getContext().resolveDatasourceTargetName(key));
            Future<Observable<VectorSchemaRoot>> observableFuture = connectionFuture.map(connection -> {
                Observable<VectorSchemaRoot> observable = connection.prepareQuery(sqlString.getSql(), MycatPreparedStatementUtil.extractParams(drdsSql.getParams(), sqlString.getDynamicParameters()), MycatRelDataType.getMycatRelType(MycatViewPlan.this.schema()), rootAllocator);
                return observable.doOnComplete(() -> context.recycleConnection(context.getContext().resolveDatasourceTargetName(key), Future.succeededFuture(connection)));
            });
            observableFuture.onFailure(event -> emitter.tryOnError(event));
            observableFuture.onSuccess(event -> {
                event = event.doOnComplete(() -> emitter.onComplete());
                event = event.doOnError(throwable -> emitter.tryOnError(throwable));
                event.forEach(vectorSchemaRoot -> emitter.onNext(vectorSchemaRoot));
            });
        }));
    }
    return Observable.fromIterable(observableList).flatMap(i -> i);
}
Also used : PartitionGroup(io.mycat.PartitionGroup) Schema(org.apache.arrow.vector.types.pojo.Schema) PartitionGroup(io.mycat.PartitionGroup) PhysicalSortProperty(io.ordinate.engine.builder.PhysicalSortProperty) CalciteRowMetaData(io.mycat.calcite.resultset.CalciteRowMetaData) CalciteCompiler(io.ordinate.engine.builder.CalciteCompiler) ObservableOnSubscribe(io.reactivex.rxjava3.core.ObservableOnSubscribe) SqlNode(org.apache.calcite.sql.SqlNode) RexNode(org.apache.calcite.rex.RexNode) DrdsSqlWithParams(io.mycat.DrdsSqlWithParams) RootAllocator(org.apache.arrow.memory.RootAllocator) Map(java.util.Map) BufferAllocator(org.apache.arrow.memory.BufferAllocator) MycatMergeSort(io.mycat.calcite.physical.MycatMergeSort) SqlKind(org.apache.calcite.sql.SqlKind) NewMycatConnection(io.mycat.newquery.NewMycatConnection) MycatView(io.mycat.calcite.logical.MycatView) RexLiteral(org.apache.calcite.rex.RexLiteral) Field(org.apache.arrow.vector.types.pojo.Field) FactoryUtil(io.ordinate.engine.factory.FactoryUtil) RootContext(io.ordinate.engine.record.RootContext) Future(io.vertx.core.Future) RelFieldCollation(org.apache.calcite.rel.RelFieldCollation) Collectors(java.util.stream.Collectors) Action(io.reactivex.rxjava3.functions.Action) List(java.util.List) RelCollation(org.apache.calcite.rel.RelCollation) Sort(org.apache.calcite.rel.core.Sort) MycatPreparedStatementUtil(io.mycat.calcite.executor.MycatPreparedStatementUtil) SqlString(org.apache.calcite.sql.util.SqlString) RexCall(org.apache.calcite.rex.RexCall) VertxExecuter(io.mycat.vertx.VertxExecuter) Distribution(io.mycat.calcite.rewriter.Distribution) NewMycatDataContext(org.apache.calcite.runtime.NewMycatDataContext) NonNull(io.reactivex.rxjava3.annotations.NonNull) ArrayList(java.util.ArrayList) Observable(io.reactivex.rxjava3.core.Observable) IntFunction(io.ordinate.engine.function.IntFunction) ImmutableMultimap(com.google.common.collect.ImmutableMultimap) MycatRelDataType(io.mycat.beans.mycat.MycatRelDataType) CopyMycatRowMetaData(io.mycat.beans.mycat.CopyMycatRowMetaData) VectorSchemaRoot(org.apache.arrow.vector.VectorSchemaRoot) MycatRelDatasourceSourceInfo(io.mycat.calcite.MycatRelDatasourceSourceInfo) Consumer(io.reactivex.rxjava3.functions.Consumer) ObservableEmitter(io.reactivex.rxjava3.core.ObservableEmitter) AsyncMycatDataContextImpl(io.mycat.AsyncMycatDataContextImpl) DrdsSql(io.mycat.DrdsSql) Function(io.reactivex.rxjava3.functions.Function) Handler(io.vertx.core.Handler) Collections(java.util.Collections) VectorSchemaRoot(org.apache.arrow.vector.VectorSchemaRoot) ArrayList(java.util.ArrayList) SqlString(org.apache.calcite.sql.util.SqlString) CopyMycatRowMetaData(io.mycat.beans.mycat.CopyMycatRowMetaData) Observable(io.reactivex.rxjava3.core.Observable) DrdsSqlWithParams(io.mycat.DrdsSqlWithParams) SqlString(org.apache.calcite.sql.util.SqlString) MycatView(io.mycat.calcite.logical.MycatView) RootAllocator(org.apache.arrow.memory.RootAllocator) MycatRelDatasourceSourceInfo(io.mycat.calcite.MycatRelDatasourceSourceInfo) NewMycatDataContext(org.apache.calcite.runtime.NewMycatDataContext) Future(io.vertx.core.Future) CalciteRowMetaData(io.mycat.calcite.resultset.CalciteRowMetaData) Map(java.util.Map) SqlNode(org.apache.calcite.sql.SqlNode)

Example 27 with NewMycatConnection

use of io.mycat.newquery.NewMycatConnection in project Mycat2 by MyCATApache.

the class MycatNativeDatasourcePool method getConnection.

@Override
public Future<NewMycatConnection> getConnection() {
    return vertxPoolConnection.getConnection().map(connection -> {
        DatabaseInstanceEntry stat = DatabaseInstanceEntry.stat(targetName);
        stat.plusCon();
        stat.plusQps();
        return new VertxMycatConnectionPool(targetName, connection, vertxPoolConnection) {

            long start;

            @Override
            public void onSend() {
                start = System.currentTimeMillis();
                onActiveTimestamp(start);
            }

            @Override
            public void onRev() {
                long end = System.currentTimeMillis();
                onActiveTimestamp(end);
                InstanceMonitor.plusPrt(end - start);
            }

            @Override
            public Future<Void> close() {
                stat.decCon();
                return super.close();
            }

            @Override
            public Observable<VectorSchemaRoot> prepareQuery(String sql, List<Object> params, BufferAllocator allocator) {
                return Observable.create(emitter -> {
                    JdbcDatasourcePoolImpl jdbcDatasourcePool = new JdbcDatasourcePoolImpl(targetName);
                    Future<NewMycatConnection> connectionFuture = jdbcDatasourcePool.getConnection();
                    connectionFuture.onSuccess(event -> {
                        Observable<VectorSchemaRoot> observable = event.prepareQuery(sql, params, allocator);
                        observable.subscribe(vectorSchemaRoot -> emitter.onNext(vectorSchemaRoot), throwable -> emitter.onError(throwable), () -> emitter.onComplete());
                    });
                    connectionFuture.onFailure(event -> emitter.onError(event));
                });
            }

            @Override
            public Future<List<Object>> call(String sql) {
                JdbcDatasourcePoolImpl jdbcDatasourcePool = new JdbcDatasourcePoolImpl(targetName);
                Future<NewMycatConnection> connectionFuture = jdbcDatasourcePool.getConnection();
                return connectionFuture.flatMap(connection -> connection.call(sql));
            }
        };
    });
}
Also used : DatabaseInstanceEntry(io.mycat.monitor.DatabaseInstanceEntry) VectorSchemaRoot(org.apache.arrow.vector.VectorSchemaRoot) NewMycatConnection(io.mycat.newquery.NewMycatConnection) JdbcDatasourcePoolImpl(io.mycat.commands.JdbcDatasourcePoolImpl) List(java.util.List) BufferAllocator(org.apache.arrow.memory.BufferAllocator)

Example 28 with NewMycatConnection

use of io.mycat.newquery.NewMycatConnection in project Mycat2 by MyCATApache.

the class ReceiverImpl method proxyProcedure.

@Override
public Future<Void> proxyProcedure(String sql, String targetName) {
    targetName = dataContext.resolveDatasourceTargetName(targetName, true);
    XaSqlConnection transactionSession = (XaSqlConnection) dataContext.getTransactionSession();
    Future<NewMycatConnection> mySQLManagerConnection = transactionSession.getConnection(targetName);
    Future<List<Object>> objectFuture = mySQLManagerConnection.flatMap(newMycatConnection -> {
        Future<List<Object>> call = newMycatConnection.call(sql);
        return (Future) call;
    });
    Future<List<Object>> rowBaseIteratorFuture = objectFuture.map(objects -> objects.stream().map(o -> {
        if (o instanceof long[])
            return o;
        if (o instanceof SqlResult)
            return ((SqlResult) o).toLongs();
        if (o instanceof RowSet)
            return ((RowSet) o).toRowBaseIterator();
        throw new UnsupportedOperationException();
    }).collect(Collectors.toList()));
    return rowBaseIteratorFuture.flatMap(objectList -> {
        if (objectList instanceof List) {
            List list = (List) objectList;
            int thisStmtResultSetSize = list.size();
            int resultSetCounter = getResultSetCounter();
            resetResultSetCounter(resultSetCounter + thisStmtResultSetSize - 1);
            Future<Void> future = Future.succeededFuture();
            for (Object o : list) {
                if (o instanceof long[]) {
                    long[] r = (long[]) o;
                    future = future.flatMap(unused -> sendOk(r[0], r[1]));
                } else if (o instanceof RowBaseIterator) {
                    RowBaseIterator rs = (RowBaseIterator) o;
                    future = future.flatMap(unused -> sendResultSet(rs));
                }
                return future;
            }
        }
        throw new UnsupportedOperationException();
    });
}
Also used : VertxExecuter(io.mycat.vertx.VertxExecuter) java.util(java.util) io.mycat(io.mycat) MycatRowMetaData(io.mycat.beans.mycat.MycatRowMetaData) ResultSetWriter(io.mycat.beans.resultset.ResultSetWriter) io.vertx.core(io.vertx.core) RowSet(io.mycat.newquery.RowSet) LoggerFactory(org.slf4j.LoggerFactory) NonNull(io.reactivex.rxjava3.annotations.NonNull) SimpleBinaryWriterImpl(io.mycat.beans.resultset.SimpleBinaryWriterImpl) Function(java.util.function.Function) ExecuteType(io.mycat.ExecuteType) Observable(io.reactivex.rxjava3.core.Observable) SimpleTextWriterImpl(io.mycat.beans.resultset.SimpleTextWriterImpl) ResultSetMapping(io.mycat.vertx.ResultSetMapping) ObservableSource(io.reactivex.rxjava3.core.ObservableSource) SchemaBuilder(io.ordinate.engine.builder.SchemaBuilder) InnerType(io.ordinate.engine.schema.InnerType) Observer(io.reactivex.rxjava3.core.Observer) Logger(org.slf4j.Logger) NewMycatConnection(io.mycat.newquery.NewMycatConnection) MySQLServerSession(io.mycat.proxy.session.MySQLServerSession) VectorSchemaRoot(org.apache.arrow.vector.VectorSchemaRoot) SqlResult(io.mycat.newquery.SqlResult) VertxUtil(io.mycat.util.VertxUtil) ResultWriterUtil(io.ordinate.engine.util.ResultWriterUtil) Collectors(java.util.stream.Collectors) io.mycat.api.collector(io.mycat.api.collector) AtomicLong(java.util.concurrent.atomic.AtomicLong) Buffer(io.vertx.core.buffer.Buffer) XaSqlConnection(cn.mycat.vertx.xa.XaSqlConnection) Disposable(io.reactivex.rxjava3.disposables.Disposable) NewMycatConnection(io.mycat.newquery.NewMycatConnection) SqlResult(io.mycat.newquery.SqlResult) RowSet(io.mycat.newquery.RowSet) XaSqlConnection(cn.mycat.vertx.xa.XaSqlConnection)

Example 29 with NewMycatConnection

use of io.mycat.newquery.NewMycatConnection in project Mycat2 by MyCATApache.

the class BaseSavepointSuite method baseSavepointReleaseSavepointInTwoConnection.

@Test
@SneakyThrows
public void baseSavepointReleaseSavepointInTwoConnection(VertxTestContext testContext) {
    mySQLManager.getConnection("ds1").flatMap(connection -> {
        return connection.update("delete FROM `db1`.`travelrecord`").map(u -> connection);
    }).flatMap(c -> c.close()).toCompletionStage().toCompletableFuture().get();
    mySQLManager.getConnection("ds2").flatMap(connection -> {
        return connection.update("delete FROM `db1`.`travelrecord`").map(u -> connection);
    }).flatMap(c -> c.close()).toCompletionStage().toCompletableFuture().get();
    XaSqlConnection baseXaSqlConnection = factory.apply(mySQLManager, xaLog);
    Assert.assertTrue(baseXaSqlConnection instanceof SavepointSqlConnection);
    SavepointSqlConnection savepointSqlConnection = (SavepointSqlConnection) baseXaSqlConnection;
    baseXaSqlConnection.begin().onComplete(new Handler<AsyncResult<Void>>() {

        @Override
        @SneakyThrows
        public void handle(AsyncResult<Void> event) {
            NewMycatConnection ds1 = savepointSqlConnection.getConnection("ds1").toCompletionStage().toCompletableFuture().get();
            ds1.insert("insert into `db1`.`travelrecord` (`id`) values ('2')").toCompletionStage().toCompletableFuture().get();
            NewMycatConnection ds2 = savepointSqlConnection.getConnection("ds2").toCompletionStage().toCompletableFuture().get();
            ds2.insert("insert into `db1`.`travelrecord` (`id`) values ('2')").toCompletionStage().toCompletableFuture().get();
            savepointSqlConnection.createSavepoint("sss").toCompletionStage().toCompletableFuture().get();
            RowSet objects = ds1.query("select * from `db1`.`travelrecord` where id = 2").toCompletionStage().toCompletableFuture().get();
            Assert.assertTrue(objects.size() > 0);
            RowSet objects2 = ds2.query("select * from `db1`.`travelrecord` where id = 2").toCompletionStage().toCompletableFuture().get();
            Assert.assertTrue(objects2.size() > 0);
            Assert.assertEquals("[sss]", savepointSqlConnection.getExistedSavepoints().toString());
            savepointSqlConnection.releaseSavepoint("sss").toCompletionStage().toCompletableFuture().get();
            objects = ds1.query("select * from `db1`.`travelrecord` where id = 2").toCompletionStage().toCompletableFuture().get();
            Assert.assertTrue(objects.size() > 0);
            objects2 = ds2.query("select * from `db1`.`travelrecord` where id = 2").toCompletionStage().toCompletableFuture().get();
            Assert.assertTrue(objects2.size() > 0);
            Assert.assertEquals("[]", savepointSqlConnection.getExistedSavepoints().toString());
            Assert.assertTrue(savepointSqlConnection.isInTransaction());
            ds1 = savepointSqlConnection.getConnection("ds1").toCompletionStage().toCompletableFuture().get();
            objects = ds1.query("select * from `db1`.`travelrecord` where id = 2").toCompletionStage().toCompletableFuture().get();
            Assert.assertTrue(objects.size() > 0);
            ds2 = savepointSqlConnection.getConnection("ds2").toCompletionStage().toCompletableFuture().get();
            objects = ds2.query("select * from `db1`.`travelrecord` where id = 2").toCompletionStage().toCompletableFuture().get();
            Assert.assertTrue(objects.size() > 0);
            savepointSqlConnection.commit().toCompletionStage().toCompletableFuture().get();
            Assert.assertEquals("[]", savepointSqlConnection.getExistedSavepoints().toString());
            Assert.assertFalse(savepointSqlConnection.isInTransaction());
            ds1 = savepointSqlConnection.getConnection("ds1").toCompletionStage().toCompletableFuture().get();
            objects = ds1.query("select * from `db1`.`travelrecord` where id = 2").toCompletionStage().toCompletableFuture().get();
            Assert.assertTrue(objects.size() > 0);
            ds2 = savepointSqlConnection.getConnection("ds1").toCompletionStage().toCompletableFuture().get();
            objects = ds2.query("select * from `db1`.`travelrecord` where id = 2").toCompletionStage().toCompletableFuture().get();
            Assert.assertTrue(objects.size() > 0);
            savepointSqlConnection.close();
            testContext.completeNow();
        }
    });
}
Also used : VertxTestContext(io.vertx.junit5.VertxTestContext) NewMycatConnection(io.mycat.newquery.NewMycatConnection) SneakyThrows(lombok.SneakyThrows) RowSet(io.mycat.newquery.RowSet) BiFunction(java.util.function.BiFunction) XaLog(cn.mycat.vertx.xa.XaLog) Future(io.vertx.core.Future) Test(org.junit.jupiter.api.Test) XaSqlConnection(cn.mycat.vertx.xa.XaSqlConnection) AsyncResult(io.vertx.core.AsyncResult) MySQLManager(cn.mycat.vertx.xa.MySQLManager) SavepointSqlConnection(cn.mycat.vertx.xa.SavepointSqlConnection) Handler(io.vertx.core.Handler) Assert(org.junit.Assert) NewMycatConnection(io.mycat.newquery.NewMycatConnection) SavepointSqlConnection(cn.mycat.vertx.xa.SavepointSqlConnection) RowSet(io.mycat.newquery.RowSet) SneakyThrows(lombok.SneakyThrows) AsyncResult(io.vertx.core.AsyncResult) XaSqlConnection(cn.mycat.vertx.xa.XaSqlConnection) Test(org.junit.jupiter.api.Test) SneakyThrows(lombok.SneakyThrows)

Example 30 with NewMycatConnection

use of io.mycat.newquery.NewMycatConnection in project Mycat2 by MyCATApache.

the class BaseSavepointSuite method baseSavepointRollbackInSingleConnection.

@Test
@SneakyThrows
public void baseSavepointRollbackInSingleConnection(VertxTestContext testContext) {
    mySQLManager.getConnection("ds1").flatMap(connection -> {
        return connection.update("delete FROM `db1`.`travelrecord`").map(u -> connection);
    }).flatMap(c -> c.close()).toCompletionStage().toCompletableFuture().get();
    XaSqlConnection baseXaSqlConnection = factory.apply(mySQLManager, xaLog);
    Assert.assertTrue(baseXaSqlConnection instanceof SavepointSqlConnection);
    SavepointSqlConnection savepointSqlConnection = (SavepointSqlConnection) baseXaSqlConnection;
    baseXaSqlConnection.begin().onComplete(new Handler<AsyncResult<Void>>() {

        @Override
        @SneakyThrows
        public void handle(AsyncResult<Void> event) {
            savepointSqlConnection.createSavepoint("sss").toCompletionStage().toCompletableFuture().get();
            NewMycatConnection ds1 = savepointSqlConnection.getConnection("ds1").toCompletionStage().toCompletableFuture().get();
            ds1.insert("insert into `db1`.`travelrecord` (`id`) values ('2')").toCompletionStage().toCompletableFuture().get();
            RowSet objects = ds1.query("select * from `db1`.`travelrecord` where id = 2").toCompletionStage().toCompletableFuture().get();
            Assert.assertTrue(objects.size() > 0);
            Assert.assertEquals("[sss]", savepointSqlConnection.getExistedSavepoints().toString());
            savepointSqlConnection.rollback().toCompletionStage().toCompletableFuture().get();
            Assert.assertEquals("[]", savepointSqlConnection.getExistedSavepoints().toString());
            ds1 = savepointSqlConnection.getConnection("ds1").toCompletionStage().toCompletableFuture().get();
            objects = ds1.query("select * from `db1`.`travelrecord` where id = 2").toCompletionStage().toCompletableFuture().get();
            Assert.assertTrue(objects.size() == 0);
            savepointSqlConnection.close();
            testContext.completeNow();
        }
    });
}
Also used : VertxTestContext(io.vertx.junit5.VertxTestContext) NewMycatConnection(io.mycat.newquery.NewMycatConnection) SneakyThrows(lombok.SneakyThrows) RowSet(io.mycat.newquery.RowSet) BiFunction(java.util.function.BiFunction) XaLog(cn.mycat.vertx.xa.XaLog) Future(io.vertx.core.Future) Test(org.junit.jupiter.api.Test) XaSqlConnection(cn.mycat.vertx.xa.XaSqlConnection) AsyncResult(io.vertx.core.AsyncResult) MySQLManager(cn.mycat.vertx.xa.MySQLManager) SavepointSqlConnection(cn.mycat.vertx.xa.SavepointSqlConnection) Handler(io.vertx.core.Handler) Assert(org.junit.Assert) NewMycatConnection(io.mycat.newquery.NewMycatConnection) SavepointSqlConnection(cn.mycat.vertx.xa.SavepointSqlConnection) RowSet(io.mycat.newquery.RowSet) SneakyThrows(lombok.SneakyThrows) AsyncResult(io.vertx.core.AsyncResult) XaSqlConnection(cn.mycat.vertx.xa.XaSqlConnection) Test(org.junit.jupiter.api.Test) SneakyThrows(lombok.SneakyThrows)

Aggregations

NewMycatConnection (io.mycat.newquery.NewMycatConnection)35 Future (io.vertx.core.Future)28 MySQLManager (cn.mycat.vertx.xa.MySQLManager)24 XaLog (cn.mycat.vertx.xa.XaLog)20 XaSqlConnection (cn.mycat.vertx.xa.XaSqlConnection)20 Handler (io.vertx.core.Handler)19 CompositeFuture (io.vertx.core.CompositeFuture)17 RowSet (io.mycat.newquery.RowSet)16 AsyncResult (io.vertx.core.AsyncResult)14 VertxTestContext (io.vertx.junit5.VertxTestContext)14 BiFunction (java.util.function.BiFunction)14 Function (java.util.function.Function)14 Test (org.junit.jupiter.api.Test)14 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)13 List (java.util.List)12 Logger (org.slf4j.Logger)12 LoggerFactory (org.slf4j.LoggerFactory)12 Collectors (java.util.stream.Collectors)10 SneakyThrows (lombok.SneakyThrows)10 SavepointSqlConnection (cn.mycat.vertx.xa.SavepointSqlConnection)9