Search in sources :

Example 1 with TaskExecuteResult

use of cn.edu.tsinghua.iginx.engine.physical.task.TaskExecuteResult in project IginX by thulab.

the class MemoryPhysicalTaskDispatcher method startDispatcher.

public void startDispatcher() {
    taskDispatcher.submit(() -> {
        while (true) {
            final MemoryPhysicalTask task = taskQueue.getTask();
            taskExecuteThreadPool.submit(() -> {
                MemoryPhysicalTask currentTask = task;
                while (currentTask != null) {
                    TaskExecuteResult result;
                    try {
                        result = currentTask.execute();
                    } catch (Exception e) {
                        logger.error("execute memory task failure: ", e);
                        result = new TaskExecuteResult(new PhysicalException(e));
                    }
                    currentTask.setResult(result);
                    if (currentTask.getFollowerTask() != null) {
                        // 链式执行可以被执行的任务
                        MemoryPhysicalTask followerTask = (MemoryPhysicalTask) currentTask.getFollowerTask();
                        if (followerTask.notifyParentReady()) {
                            currentTask = followerTask;
                        } else {
                            currentTask = null;
                        }
                    } else {
                        currentTask = null;
                    }
                }
            });
        }
    });
}
Also used : MemoryPhysicalTask(cn.edu.tsinghua.iginx.engine.physical.task.MemoryPhysicalTask) TaskExecuteResult(cn.edu.tsinghua.iginx.engine.physical.task.TaskExecuteResult) PhysicalException(cn.edu.tsinghua.iginx.engine.physical.exception.PhysicalException) PhysicalException(cn.edu.tsinghua.iginx.engine.physical.exception.PhysicalException)

Example 2 with TaskExecuteResult

use of cn.edu.tsinghua.iginx.engine.physical.task.TaskExecuteResult in project IginX by thulab.

the class IoTDBStorage method executeInsertTask.

private TaskExecuteResult executeInsertTask(String storageUnit, Insert insert) {
    DataView dataView = insert.getData();
    Exception e = null;
    switch(dataView.getRawDataType()) {
        case Row:
            e = insertRowRecords((RowDataView) dataView, storageUnit);
            break;
        case Column:
            e = insertColumnRecords((ColumnDataView) dataView, storageUnit);
            break;
        case NonAlignedRow:
            e = insertNonAlignedRowRecords((RowDataView) dataView, storageUnit);
            break;
        case NonAlignedColumn:
            e = insertNonAlignedColumnRecords((ColumnDataView) dataView, storageUnit);
            break;
    }
    if (e != null) {
        return new TaskExecuteResult(null, new PhysicalException("execute insert task in iotdb12 failure", e));
    }
    return new TaskExecuteResult(null, null);
}
Also used : DataView(cn.edu.tsinghua.iginx.engine.shared.data.write.DataView) RowDataView(cn.edu.tsinghua.iginx.engine.shared.data.write.RowDataView) ColumnDataView(cn.edu.tsinghua.iginx.engine.shared.data.write.ColumnDataView) TaskExecuteResult(cn.edu.tsinghua.iginx.engine.physical.task.TaskExecuteResult) ColumnDataView(cn.edu.tsinghua.iginx.engine.shared.data.write.ColumnDataView) PhysicalException(cn.edu.tsinghua.iginx.engine.physical.exception.PhysicalException) PhysicalException(cn.edu.tsinghua.iginx.engine.physical.exception.PhysicalException) StorageInitializationException(cn.edu.tsinghua.iginx.engine.physical.exception.StorageInitializationException) NonExecutablePhysicalTaskException(cn.edu.tsinghua.iginx.engine.physical.exception.NonExecutablePhysicalTaskException) StatementExecutionException(org.apache.iotdb.rpc.StatementExecutionException) IoTDBConnectionException(org.apache.iotdb.rpc.IoTDBConnectionException) PhysicalTaskExecuteFailureException(cn.edu.tsinghua.iginx.engine.physical.exception.PhysicalTaskExecuteFailureException) RowDataView(cn.edu.tsinghua.iginx.engine.shared.data.write.RowDataView)

Example 3 with TaskExecuteResult

use of cn.edu.tsinghua.iginx.engine.physical.task.TaskExecuteResult in project IginX by thulab.

the class IoTDBStorage method execute.

@Override
public TaskExecuteResult execute(StoragePhysicalTask task) {
    List<Operator> operators = task.getOperators();
    if (operators.size() < 1) {
        return new TaskExecuteResult(new NonExecutablePhysicalTaskException("storage physical task should have one more operators"));
    }
    Operator op = operators.get(0);
    String storageUnit = task.getStorageUnit();
    if (op.getType() == OperatorType.Project) {
        Project project = (Project) op;
        Filter filter;
        if (operators.size() == 2) {
            filter = ((Select) operators.get(1)).getFilter();
        } else {
            FragmentMeta fragment = task.getTargetFragment();
            filter = new AndFilter(Arrays.asList(new TimeFilter(Op.GE, fragment.getTimeInterval().getStartTime()), new TimeFilter(Op.L, fragment.getTimeInterval().getEndTime())));
        }
        return executeQueryTask(storageUnit, project, filter);
    } else if (op.getType() == OperatorType.Insert) {
        Insert insert = (Insert) op;
        return executeInsertTask(storageUnit, insert);
    } else if (op.getType() == OperatorType.Delete) {
        Delete delete = (Delete) op;
        return executeDeleteTask(storageUnit, delete);
    }
    return new TaskExecuteResult(new NonExecutablePhysicalTaskException("unsupported physical task"));
}
Also used : Operator(cn.edu.tsinghua.iginx.engine.shared.operator.Operator) NonExecutablePhysicalTaskException(cn.edu.tsinghua.iginx.engine.physical.exception.NonExecutablePhysicalTaskException) Delete(cn.edu.tsinghua.iginx.engine.shared.operator.Delete) Project(cn.edu.tsinghua.iginx.engine.shared.operator.Project) AndFilter(cn.edu.tsinghua.iginx.engine.shared.operator.filter.AndFilter) AndFilter(cn.edu.tsinghua.iginx.engine.shared.operator.filter.AndFilter) TimeFilter(cn.edu.tsinghua.iginx.engine.shared.operator.filter.TimeFilter) Filter(cn.edu.tsinghua.iginx.engine.shared.operator.filter.Filter) TimeFilter(cn.edu.tsinghua.iginx.engine.shared.operator.filter.TimeFilter) TaskExecuteResult(cn.edu.tsinghua.iginx.engine.physical.task.TaskExecuteResult) FragmentMeta(cn.edu.tsinghua.iginx.metadata.entity.FragmentMeta) Insert(cn.edu.tsinghua.iginx.engine.shared.operator.Insert)

Example 4 with TaskExecuteResult

use of cn.edu.tsinghua.iginx.engine.physical.task.TaskExecuteResult in project IginX by thulab.

the class OpenTSDBStorage method execute.

@Override
public TaskExecuteResult execute(StoragePhysicalTask task) {
    List<Operator> operators = task.getOperators();
    if (operators.size() != 1) {
        return new TaskExecuteResult(new NonExecutablePhysicalTaskException("unsupported physical task"));
    }
    FragmentMeta fragment = task.getTargetFragment();
    Operator op = operators.get(0);
    String storageUnit = task.getStorageUnit();
    if (op.getType() == OperatorType.Project) {
        Project project = (Project) op;
        return executeProjectTask(fragment.getTimeInterval(), storageUnit, project);
    } else if (op.getType() == OperatorType.Insert) {
        Insert insert = (Insert) op;
        return executeInsertTask(storageUnit, insert);
    } else if (op.getType() == OperatorType.Delete) {
        Delete delete = (Delete) op;
        return executeDeleteTask(storageUnit, delete);
    }
    return new TaskExecuteResult(new NonExecutablePhysicalTaskException("unsupported physical task"));
}
Also used : NonExecutablePhysicalTaskException(cn.edu.tsinghua.iginx.engine.physical.exception.NonExecutablePhysicalTaskException) TaskExecuteResult(cn.edu.tsinghua.iginx.engine.physical.task.TaskExecuteResult) FragmentMeta(cn.edu.tsinghua.iginx.metadata.entity.FragmentMeta)

Example 5 with TaskExecuteResult

use of cn.edu.tsinghua.iginx.engine.physical.task.TaskExecuteResult in project IginX by thulab.

the class OpenTSDBStorage method executeInsertTask.

private TaskExecuteResult executeInsertTask(String storageUnit, Insert insert) {
    DataView dataView = insert.getData();
    Exception e = null;
    switch(dataView.getRawDataType()) {
        case Row:
        case NonAlignedRow:
            e = insertRowRecords((RowDataView) dataView, storageUnit);
            break;
        case Column:
        case NonAlignedColumn:
            e = insertColRecords((ColumnDataView) dataView, storageUnit);
    }
    if (e != null) {
        return new TaskExecuteResult(null, new PhysicalException("execute insert task in opentsdb failure", e));
    }
    return new TaskExecuteResult(null, null);
}
Also used : RowDataView(cn.edu.tsinghua.iginx.engine.shared.data.write.RowDataView) ColumnDataView(cn.edu.tsinghua.iginx.engine.shared.data.write.ColumnDataView) DataView(cn.edu.tsinghua.iginx.engine.shared.data.write.DataView) TaskExecuteResult(cn.edu.tsinghua.iginx.engine.physical.task.TaskExecuteResult) ColumnDataView(cn.edu.tsinghua.iginx.engine.shared.data.write.ColumnDataView) PhysicalException(cn.edu.tsinghua.iginx.engine.physical.exception.PhysicalException) PhysicalException(cn.edu.tsinghua.iginx.engine.physical.exception.PhysicalException) PhysicalTaskExecuteFailureException(cn.edu.tsinghua.iginx.engine.physical.exception.PhysicalTaskExecuteFailureException) StorageInitializationException(cn.edu.tsinghua.iginx.engine.physical.exception.StorageInitializationException) NonExecutablePhysicalTaskException(cn.edu.tsinghua.iginx.engine.physical.exception.NonExecutablePhysicalTaskException) IOException(java.io.IOException) IOReactorException(org.apache.http.nio.reactor.IOReactorException) RowDataView(cn.edu.tsinghua.iginx.engine.shared.data.write.RowDataView)

Aggregations

TaskExecuteResult (cn.edu.tsinghua.iginx.engine.physical.task.TaskExecuteResult)17 NonExecutablePhysicalTaskException (cn.edu.tsinghua.iginx.engine.physical.exception.NonExecutablePhysicalTaskException)11 PhysicalTaskExecuteFailureException (cn.edu.tsinghua.iginx.engine.physical.exception.PhysicalTaskExecuteFailureException)10 PhysicalException (cn.edu.tsinghua.iginx.engine.physical.exception.PhysicalException)9 StorageInitializationException (cn.edu.tsinghua.iginx.engine.physical.exception.StorageInitializationException)7 DataView (cn.edu.tsinghua.iginx.engine.shared.data.write.DataView)5 RowDataView (cn.edu.tsinghua.iginx.engine.shared.data.write.RowDataView)5 FragmentMeta (cn.edu.tsinghua.iginx.metadata.entity.FragmentMeta)5 ColumnDataView (cn.edu.tsinghua.iginx.engine.shared.data.write.ColumnDataView)4 Delete (cn.edu.tsinghua.iginx.engine.shared.operator.Delete)4 Insert (cn.edu.tsinghua.iginx.engine.shared.operator.Insert)4 Operator (cn.edu.tsinghua.iginx.engine.shared.operator.Operator)4 Project (cn.edu.tsinghua.iginx.engine.shared.operator.Project)4 TimeRange (cn.edu.tsinghua.iginx.engine.shared.TimeRange)3 SQLException (java.sql.SQLException)3 ArrayList (java.util.ArrayList)3 IStorage (cn.edu.tsinghua.iginx.engine.physical.storage.IStorage)2 Timeseries (cn.edu.tsinghua.iginx.engine.physical.storage.domain.Timeseries)2 RowStream (cn.edu.tsinghua.iginx.engine.shared.data.read.RowStream)2 InfluxDBQueryRowStream (cn.edu.tsinghua.iginx.influxdb.query.entity.InfluxDBQueryRowStream)2