Search in sources :

Example 1 with VisibleForTesting

use of io.crate.common.annotations.VisibleForTesting in project crate by crate.

the class HttpAuthUpstreamHandler method credentialsFromRequest.

@VisibleForTesting
static Tuple<String, SecureString> credentialsFromRequest(HttpRequest request, @Nullable SSLSession session, Settings settings) {
    String username = null;
    if (request.headers().contains(HttpHeaderNames.AUTHORIZATION.toString())) {
        // Prefer Http Basic Auth
        return Headers.extractCredentialsFromHttpBasicAuthHeader(request.headers().get(HttpHeaderNames.AUTHORIZATION.toString()));
    } else {
        // prefer commonName as userName over AUTH_TRUST_HTTP_DEFAULT_HEADER user
        if (session != null) {
            try {
                Certificate certificate = session.getPeerCertificates()[0];
                username = SSL.extractCN(certificate);
            } catch (ArrayIndexOutOfBoundsException | SSLPeerUnverifiedException ignored) {
            // client cert is optional
            }
        }
        if (username == null) {
            username = AuthSettings.AUTH_TRUST_HTTP_DEFAULT_HEADER.get(settings);
        }
    }
    return new Tuple<>(username, null);
}
Also used : SSLPeerUnverifiedException(javax.net.ssl.SSLPeerUnverifiedException) SecureString(org.elasticsearch.common.settings.SecureString) Tuple(io.crate.common.collections.Tuple) Certificate(java.security.cert.Certificate) VisibleForTesting(io.crate.common.annotations.VisibleForTesting)

Example 2 with VisibleForTesting

use of io.crate.common.annotations.VisibleForTesting in project crate by crate.

the class HttpAuthUpstreamHandler method sendUnauthorized.

@VisibleForTesting
static void sendUnauthorized(Channel channel, @Nullable String body) {
    HttpResponse response;
    if (body != null) {
        if (!body.endsWith("\n")) {
            body += "\n";
        }
        response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.UNAUTHORIZED, copiedBuffer(body, StandardCharsets.UTF_8));
        HttpUtil.setContentLength(response, body.length());
    } else {
        response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.UNAUTHORIZED);
    }
    // "Tell" the browser to open the credentials popup
    // It helps to avoid custom login page in AdminUI
    response.headers().set(HttpHeaderNames.WWW_AUTHENTICATE, WWW_AUTHENTICATE_REALM_MESSAGE);
    channel.writeAndFlush(response).addListener(ChannelFutureListener.CLOSE);
}
Also used : DefaultFullHttpResponse(io.netty.handler.codec.http.DefaultFullHttpResponse) DefaultFullHttpResponse(io.netty.handler.codec.http.DefaultFullHttpResponse) HttpResponse(io.netty.handler.codec.http.HttpResponse) VisibleForTesting(io.crate.common.annotations.VisibleForTesting)

Example 3 with VisibleForTesting

use of io.crate.common.annotations.VisibleForTesting in project crate by crate.

the class DocTableRelation method ensureColumnCanBeUpdated.

/**
 * @throws io.crate.exceptions.ColumnValidationException if the column cannot be updated
 */
@VisibleForTesting
void ensureColumnCanBeUpdated(ColumnIdent ci) {
    if (ci.isSystemColumn()) {
        throw new ColumnValidationException(ci.toString(), tableInfo.ident(), "Updating a system column is not supported");
    }
    for (ColumnIdent pkIdent : tableInfo.primaryKey()) {
        ensureNotUpdated(ci, pkIdent, "Updating a primary key is not supported");
    }
    if (tableInfo.clusteredBy() != null) {
        ensureNotUpdated(ci, tableInfo.clusteredBy(), "Updating a clustered-by column is not supported");
    }
    List<GeneratedReference> generatedReferences = tableInfo.generatedColumns();
    for (Reference partitionRef : tableInfo.partitionedByColumns()) {
        ensureNotUpdated(ci, partitionRef.column(), "Updating a partitioned-by column is not supported");
        if (!(partitionRef instanceof GeneratedReference)) {
            continue;
        }
        int idx = generatedReferences.indexOf(partitionRef);
        if (idx >= 0) {
            GeneratedReference generatedReference = generatedReferences.get(idx);
            for (Reference reference : generatedReference.referencedReferences()) {
                ensureNotUpdated(ci, reference.column(), "Updating a column which is referenced in a partitioned by generated column expression is not supported");
            }
        }
    }
}
Also used : ColumnIdent(io.crate.metadata.ColumnIdent) GeneratedReference(io.crate.metadata.GeneratedReference) GeneratedReference(io.crate.metadata.GeneratedReference) Reference(io.crate.metadata.Reference) ColumnValidationException(io.crate.exceptions.ColumnValidationException) VisibleForTesting(io.crate.common.annotations.VisibleForTesting)

Example 4 with VisibleForTesting

use of io.crate.common.annotations.VisibleForTesting in project crate by crate.

the class CopyToPlan method bind.

@VisibleForTesting
public static BoundCopyTo bind(AnalyzedCopyTo copyTo, CoordinatorTxnCtx txnCtx, NodeContext nodeCtx, Row parameters, SubQueryResults subQueryResults) {
    Function<? super Symbol, Object> eval = x -> SymbolEvaluator.evaluate(txnCtx, nodeCtx, x, parameters, subQueryResults);
    DocTableInfo table = (DocTableInfo) copyTo.tableInfo();
    List<String> partitions = resolvePartitions(Lists2.map(copyTo.table().partitionProperties(), x -> x.map(eval)), table);
    List<Symbol> outputs = new ArrayList<>();
    Map<ColumnIdent, Symbol> overwrites = null;
    boolean columnsDefined = false;
    final List<String> outputNames = new ArrayList<>(copyTo.columns().size());
    if (!copyTo.columns().isEmpty()) {
        // TODO: remove outputNames?
        for (Symbol symbol : copyTo.columns()) {
            assert symbol instanceof Reference : "Only references are expected here";
            RefVisitor.visitRefs(symbol, r -> outputNames.add(r.column().sqlFqn()));
            outputs.add(DocReferences.toSourceLookup(symbol));
        }
        columnsDefined = true;
    } else {
        Reference sourceRef;
        if (table.isPartitioned() && partitions.isEmpty()) {
            // table is partitioned, insert partitioned columns into the output
            overwrites = new HashMap<>();
            for (Reference reference : table.partitionedByColumns()) {
                if (!(reference instanceof GeneratedReference)) {
                    overwrites.put(reference.column(), reference);
                }
            }
            if (overwrites.size() > 0) {
                sourceRef = table.getReference(DocSysColumns.DOC);
            } else {
                sourceRef = table.getReference(DocSysColumns.RAW);
            }
        } else {
            sourceRef = table.getReference(DocSysColumns.RAW);
        }
        outputs = List.of(sourceRef);
    }
    Settings settings = genericPropertiesToSettings(copyTo.properties().map(eval));
    WriterProjection.CompressionType compressionType = settingAsEnum(WriterProjection.CompressionType.class, COMPRESSION_SETTING.get(settings));
    WriterProjection.OutputFormat outputFormat = settingAsEnum(WriterProjection.OutputFormat.class, OUTPUT_FORMAT_SETTING.get(settings));
    if (!columnsDefined && outputFormat == WriterProjection.OutputFormat.JSON_ARRAY) {
        throw new UnsupportedFeatureException("Output format not supported without specifying columns.");
    }
    WhereClause whereClause = new WhereClause(copyTo.whereClause(), partitions, Collections.emptySet());
    return new BoundCopyTo(outputs, table, whereClause, Literal.of(DataTypes.STRING.sanitizeValue(eval.apply(copyTo.uri()))), compressionType, outputFormat, outputNames.isEmpty() ? null : outputNames, columnsDefined, overwrites, settings);
}
Also used : UnsupportedFeatureException(io.crate.exceptions.UnsupportedFeatureException) GenericPropertiesConverter.genericPropertiesToSettings(io.crate.analyze.GenericPropertiesConverter.genericPropertiesToSettings) Match(io.crate.planner.optimizer.matcher.Match) BoundCopyTo(io.crate.analyze.BoundCopyTo) DependencyCarrier(io.crate.planner.DependencyCarrier) RefVisitor(io.crate.expression.symbol.RefVisitor) SymbolEvaluator(io.crate.analyze.SymbolEvaluator) TableStats(io.crate.statistics.TableStats) Settings(org.elasticsearch.common.settings.Settings) Map(java.util.Map) OUTPUT_FORMAT_SETTING(io.crate.analyze.CopyStatementSettings.OUTPUT_FORMAT_SETTING) DocSysColumns(io.crate.metadata.doc.DocSysColumns) DocTableInfo(io.crate.metadata.doc.DocTableInfo) NodeContext(io.crate.metadata.NodeContext) LogicalPlan(io.crate.planner.operators.LogicalPlan) Captures(io.crate.planner.optimizer.matcher.Captures) GeneratedReference(io.crate.metadata.GeneratedReference) Collect(io.crate.planner.operators.Collect) Set(java.util.Set) Lists2(io.crate.common.collections.Lists2) ExecutionPlan(io.crate.planner.ExecutionPlan) OptimizeCollectWhereClauseAccess(io.crate.planner.optimizer.rule.OptimizeCollectWhereClauseAccess) DocReferences(io.crate.metadata.DocReferences) List(java.util.List) Row(io.crate.data.Row) Symbol(io.crate.expression.symbol.Symbol) DataTypes(io.crate.types.DataTypes) SubQueryResults(io.crate.planner.operators.SubQueryResults) VisibleForTesting(io.crate.common.annotations.VisibleForTesting) CoordinatorTxnCtx(io.crate.metadata.CoordinatorTxnCtx) PartitionPropertiesAnalyzer(io.crate.analyze.PartitionPropertiesAnalyzer) HashMap(java.util.HashMap) Function(java.util.function.Function) ArrayList(java.util.ArrayList) Assignment(io.crate.sql.tree.Assignment) COMPRESSION_SETTING(io.crate.analyze.CopyStatementSettings.COMPRESSION_SETTING) AnalyzedCopyTo(io.crate.analyze.AnalyzedCopyTo) PartitionUnknownException(io.crate.exceptions.PartitionUnknownException) ProjectionBuilder(io.crate.execution.dsl.projection.builder.ProjectionBuilder) MergeCountProjection(io.crate.execution.dsl.projection.MergeCountProjection) WhereClause(io.crate.analyze.WhereClause) NodeOperationTree(io.crate.execution.dsl.phases.NodeOperationTree) ColumnIdent(io.crate.metadata.ColumnIdent) Reference(io.crate.metadata.Reference) NodeOperationTreeGenerator(io.crate.execution.engine.NodeOperationTreeGenerator) Merge(io.crate.planner.Merge) RowConsumer(io.crate.data.RowConsumer) WriterProjection(io.crate.execution.dsl.projection.WriterProjection) DocTableRelation(io.crate.analyze.relations.DocTableRelation) Literal(io.crate.expression.symbol.Literal) PlannerContext(io.crate.planner.PlannerContext) CopyStatementSettings.settingAsEnum(io.crate.analyze.CopyStatementSettings.settingAsEnum) Plan(io.crate.planner.Plan) Collections(java.util.Collections) DocTableInfo(io.crate.metadata.doc.DocTableInfo) GeneratedReference(io.crate.metadata.GeneratedReference) UnsupportedFeatureException(io.crate.exceptions.UnsupportedFeatureException) BoundCopyTo(io.crate.analyze.BoundCopyTo) Symbol(io.crate.expression.symbol.Symbol) GeneratedReference(io.crate.metadata.GeneratedReference) Reference(io.crate.metadata.Reference) ArrayList(java.util.ArrayList) WhereClause(io.crate.analyze.WhereClause) WriterProjection(io.crate.execution.dsl.projection.WriterProjection) ColumnIdent(io.crate.metadata.ColumnIdent) GenericPropertiesConverter.genericPropertiesToSettings(io.crate.analyze.GenericPropertiesConverter.genericPropertiesToSettings) Settings(org.elasticsearch.common.settings.Settings) VisibleForTesting(io.crate.common.annotations.VisibleForTesting)

Example 5 with VisibleForTesting

use of io.crate.common.annotations.VisibleForTesting in project crate by crate.

the class CopyToPlan method planCopyToExecution.

@VisibleForTesting
static ExecutionPlan planCopyToExecution(AnalyzedCopyTo copyTo, PlannerContext context, TableStats tableStats, ProjectionBuilder projectionBuilder, Row params, SubQueryResults subQueryResults) {
    var boundedCopyTo = bind(copyTo, context.transactionContext(), context.nodeContext(), params, subQueryResults);
    WriterProjection.OutputFormat outputFormat = boundedCopyTo.outputFormat();
    if (outputFormat == null) {
        outputFormat = boundedCopyTo.columnsDefined() ? WriterProjection.OutputFormat.JSON_ARRAY : WriterProjection.OutputFormat.JSON_OBJECT;
    }
    WriterProjection projection = ProjectionBuilder.writerProjection(boundedCopyTo.outputs(), boundedCopyTo.uri(), boundedCopyTo.compressionType(), boundedCopyTo.overwrites(), boundedCopyTo.outputNames(), outputFormat, boundedCopyTo.withClauseOptions());
    LogicalPlan collect = Collect.create(new DocTableRelation(boundedCopyTo.table()), boundedCopyTo.outputs(), boundedCopyTo.whereClause(), tableStats, context.params());
    LogicalPlan source = optimizeCollect(context, tableStats, collect);
    ExecutionPlan executionPlan = source.build(context, Set.of(), projectionBuilder, 0, 0, null, null, params, SubQueryResults.EMPTY);
    executionPlan.addProjection(projection);
    return Merge.ensureOnHandler(executionPlan, context, List.of(MergeCountProjection.INSTANCE));
}
Also used : WriterProjection(io.crate.execution.dsl.projection.WriterProjection) ExecutionPlan(io.crate.planner.ExecutionPlan) DocTableRelation(io.crate.analyze.relations.DocTableRelation) LogicalPlan(io.crate.planner.operators.LogicalPlan) VisibleForTesting(io.crate.common.annotations.VisibleForTesting)

Aggregations

VisibleForTesting (io.crate.common.annotations.VisibleForTesting)36 Row (io.crate.data.Row)12 Symbol (io.crate.expression.symbol.Symbol)12 SubQueryResults (io.crate.planner.operators.SubQueryResults)11 Settings (org.elasticsearch.common.settings.Settings)11 RowConsumer (io.crate.data.RowConsumer)10 NodeContext (io.crate.metadata.NodeContext)10 DependencyCarrier (io.crate.planner.DependencyCarrier)10 Plan (io.crate.planner.Plan)10 PlannerContext (io.crate.planner.PlannerContext)10 Function (java.util.function.Function)10 SymbolEvaluator (io.crate.analyze.SymbolEvaluator)9 Row1 (io.crate.data.Row1)9 CoordinatorTxnCtx (io.crate.metadata.CoordinatorTxnCtx)9 ArrayList (java.util.ArrayList)8 OneRowActionListener (io.crate.execution.support.OneRowActionListener)7 Map (java.util.Map)7 ColumnIdent (io.crate.metadata.ColumnIdent)6 DocTableInfo (io.crate.metadata.doc.DocTableInfo)5 Nullable (javax.annotation.Nullable)5