Search in sources :

Example 1 with SqlBaseType

use of io.confluent.ksql.schema.ksql.types.SqlBaseType in project ksql by confluentinc.

the class SqlTypeCodeGenTest method shouldGenerateWorkingCodeForAllSqlBaseTypes.

@Test
public void shouldGenerateWorkingCodeForAllSqlBaseTypes() {
    for (final SqlBaseType baseType : SqlBaseType.values()) {
        // When:
        final String code = SqlTypeCodeGen.generateCode(TypeInstances.typeInstanceFor(baseType));
        // Then:
        final Object result = CodeGenTestUtil.cookAndEval(code, SqlType.class);
        assertThat(result, is(instanceOf(SqlType.class)));
        assertThat(((SqlType) result).baseType(), is(baseType));
    }
}
Also used : SqlBaseType(io.confluent.ksql.schema.ksql.types.SqlBaseType) Test(org.junit.Test)

Example 2 with SqlBaseType

use of io.confluent.ksql.schema.ksql.types.SqlBaseType in project ksql by confluentinc.

the class TimestampExtractionPolicyFactory method create.

public static TimestampExtractionPolicy create(final KsqlConfig ksqlConfig, final LogicalSchema schema, final Optional<TimestampColumn> timestampColumn) {
    if (!timestampColumn.isPresent()) {
        return new MetadataTimestampExtractionPolicy(getDefaultTimestampExtractor(ksqlConfig));
    }
    final ColumnName col = timestampColumn.get().getColumn();
    final Optional<String> timestampFormat = timestampColumn.get().getFormat();
    final Column column = schema.findColumn(col).orElseThrow(() -> new KsqlException("The TIMESTAMP column set in the WITH clause does not exist in the schema: '" + col.toString(FormatOptions.noEscape()) + "'"));
    final SqlBaseType tsColumnType = column.type().baseType();
    if (tsColumnType == SqlBaseType.STRING) {
        final String format = timestampFormat.orElseThrow(() -> new KsqlException("A String timestamp field has been specified without" + " also specifying the " + CommonCreateConfigs.TIMESTAMP_FORMAT_PROPERTY.toLowerCase()));
        return new StringTimestampExtractionPolicy(col, format);
    }
    if (timestampFormat.isPresent()) {
        throw new KsqlException("'" + CommonCreateConfigs.TIMESTAMP_FORMAT_PROPERTY + "' set in the WITH clause can only be used " + "when the timestamp column is of type STRING.");
    }
    if (tsColumnType == SqlBaseType.BIGINT) {
        return new LongColumnTimestampExtractionPolicy(col);
    }
    if (tsColumnType == SqlBaseType.TIMESTAMP) {
        return new TimestampColumnTimestampExtractionPolicy(col);
    }
    throw new KsqlException("Timestamp column, " + col + ", should be LONG(INT64), TIMESTAMP," + " or a String with a " + CommonCreateConfigs.TIMESTAMP_FORMAT_PROPERTY.toLowerCase() + " specified.");
}
Also used : ColumnName(io.confluent.ksql.name.ColumnName) TimestampColumn(io.confluent.ksql.execution.timestamp.TimestampColumn) Column(io.confluent.ksql.schema.ksql.Column) SqlBaseType(io.confluent.ksql.schema.ksql.types.SqlBaseType) KsqlException(io.confluent.ksql.util.KsqlException)

Example 3 with SqlBaseType

use of io.confluent.ksql.schema.ksql.types.SqlBaseType in project ksql by confluentinc.

the class OperatorTest method shouldWorkUsingSameRulesAsBaseTypeUpCastRules.

@Test
public void shouldWorkUsingSameRulesAsBaseTypeUpCastRules() {
    allOperations().forEach(op -> {
        for (final SqlBaseType leftBaseType : SqlBaseType.values()) {
            // Given:
            final Map<Boolean, List<SqlBaseType>> partitioned = Arrays.stream(SqlBaseType.values()).collect(Collectors.partitioningBy(rightBaseType -> shouldBeSupported(op, leftBaseType, rightBaseType)));
            final List<SqlBaseType> shouldUpCast = partitioned.getOrDefault(true, ImmutableList.of());
            final List<SqlBaseType> shouldNotUpCast = partitioned.getOrDefault(false, ImmutableList.of());
            // Then:
            shouldUpCast.forEach(rightBaseType -> assertThat("should support " + op + " on (" + leftBaseType + ", " + rightBaseType + ")", op.resultType(getType(leftBaseType), getType(rightBaseType)), is(notNullValue())));
            shouldNotUpCast.forEach(rightBaseType -> {
                try {
                    op.resultType(getType(leftBaseType), getType(rightBaseType));
                    fail("should not support " + op + " on (" + leftBaseType + ", " + rightBaseType + ")");
                } catch (final KsqlException e) {
                // Expected
                }
            });
        }
    });
}
Also used : DIVIDE(io.confluent.ksql.schema.Operator.DIVIDE) Arrays(java.util.Arrays) Assert.assertThrows(org.junit.Assert.assertThrows) SqlBaseType(io.confluent.ksql.schema.ksql.types.SqlBaseType) BOOLEAN(io.confluent.ksql.schema.ksql.types.SqlTypes.BOOLEAN) INTEGER(io.confluent.ksql.schema.ksql.types.SqlTypes.INTEGER) ADD(io.confluent.ksql.schema.Operator.ADD) ImmutableList(com.google.common.collect.ImmutableList) DOUBLE(io.confluent.ksql.schema.ksql.types.SqlTypes.DOUBLE) Map(java.util.Map) Assert.fail(org.junit.Assert.fail) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) SqlType(io.confluent.ksql.schema.ksql.types.SqlType) TIME(io.confluent.ksql.schema.ksql.types.SqlTypes.TIME) Matchers.notNullValue(org.hamcrest.Matchers.notNullValue) ImmutableMap(com.google.common.collect.ImmutableMap) Test(org.junit.Test) Collectors(java.util.stream.Collectors) SUBTRACT(io.confluent.ksql.schema.Operator.SUBTRACT) TIMESTAMP(io.confluent.ksql.schema.ksql.types.SqlTypes.TIMESTAMP) BinaryOperator(java.util.function.BinaryOperator) BYTES(io.confluent.ksql.schema.ksql.types.SqlTypes.BYTES) List(java.util.List) STRING(io.confluent.ksql.schema.ksql.types.SqlTypes.STRING) SqlDecimal(io.confluent.ksql.schema.ksql.types.SqlDecimal) KsqlException(io.confluent.ksql.util.KsqlException) Matchers.is(org.hamcrest.Matchers.is) MULTIPLY(io.confluent.ksql.schema.Operator.MULTIPLY) BIGINT(io.confluent.ksql.schema.ksql.types.SqlTypes.BIGINT) DATE(io.confluent.ksql.schema.ksql.types.SqlTypes.DATE) MODULUS(io.confluent.ksql.schema.Operator.MODULUS) SqlTypes(io.confluent.ksql.schema.ksql.types.SqlTypes) Assert.assertEquals(org.junit.Assert.assertEquals) SqlBaseType(io.confluent.ksql.schema.ksql.types.SqlBaseType) ImmutableList(com.google.common.collect.ImmutableList) List(java.util.List) KsqlException(io.confluent.ksql.util.KsqlException) Test(org.junit.Test)

Example 4 with SqlBaseType

use of io.confluent.ksql.schema.ksql.types.SqlBaseType in project ksql by confluentinc.

the class KsqlDelimitedDeserializer method buildParsers.

private static List<Parser> buildParsers(final PersistenceSchema schema) {
    final List<Parser> parsers = new ArrayList<>(schema.columns().size());
    for (final SimpleColumn column : schema.columns()) {
        final SqlBaseType baseType = column.type().baseType();
        final ParserFactory parserFactory = PARSERS.get(baseType);
        if (parserFactory == null) {
            throw new KsqlException("The '" + FormatFactory.DELIMITED.name() + "' format does not support type '" + baseType + "', column: " + column.name());
        }
        parsers.add(parserFactory.build(column.type()));
    }
    return parsers;
}
Also used : SqlBaseType(io.confluent.ksql.schema.ksql.types.SqlBaseType) ArrayList(java.util.ArrayList) SimpleColumn(io.confluent.ksql.schema.ksql.SimpleColumn) KsqlException(io.confluent.ksql.util.KsqlException) CSVParser(org.apache.commons.csv.CSVParser)

Example 5 with SqlBaseType

use of io.confluent.ksql.schema.ksql.types.SqlBaseType in project ksql by confluentinc.

the class ComparisonInterpreter method doCompareTo.

// CHECKSTYLE_RULES.OFF: CyclomaticComplexity
private static Optional<ComparisonFunction> doCompareTo(final Term left, final Term right) {
    // CHECKSTYLE_RULES.ON: CyclomaticComplexity
    final SqlBaseType leftType = left.getSqlType().baseType();
    final SqlBaseType rightType = right.getSqlType().baseType();
    if (either(leftType, rightType, SqlBaseType.DECIMAL)) {
        final ComparableCastFunction<BigDecimal> castLeft = castToBigDecimalFunction(left.getSqlType());
        final ComparableCastFunction<BigDecimal> castRight = castToBigDecimalFunction(right.getSqlType());
        return Optional.of((o1, o2) -> castLeft.cast(o1).compareTo(castRight.cast(o2)));
    } else if (either(leftType, rightType, SqlBaseType.TIMESTAMP)) {
        final ComparableCastFunction<Date> castLeft = castToTimestampFunction(left.getSqlType());
        final ComparableCastFunction<Date> castRight = castToTimestampFunction(right.getSqlType());
        return Optional.of((o1, o2) -> castLeft.cast(o1).compareTo(castRight.cast(o2)));
    } else if (either(leftType, rightType, SqlBaseType.DATE)) {
        final ComparableCastFunction<Date> castLeft = castToDateFunction(left.getSqlType());
        final ComparableCastFunction<Date> castRight = castToDateFunction(right.getSqlType());
        return Optional.of((o1, o2) -> castLeft.cast(o1).compareTo(castRight.cast(o2)));
    } else if (either(leftType, rightType, SqlBaseType.TIME)) {
        final ComparableCastFunction<Date> castLeft = castToTimeFunction(left.getSqlType());
        final ComparableCastFunction<Date> castRight = castToTimeFunction(right.getSqlType());
        return Optional.of((o1, o2) -> castLeft.cast(o1).compareTo(castRight.cast(o2)));
    } else if (leftType == SqlBaseType.STRING) {
        return Optional.of((o1, o2) -> o1.toString().compareTo(o2.toString()));
    } else if (leftType == SqlBaseType.BYTES && rightType == SqlBaseType.BYTES) {
        final ComparableCastFunction<ByteBuffer> castLeft = castToBytesFunction(left.getSqlType());
        final ComparableCastFunction<ByteBuffer> castRight = castToBytesFunction(right.getSqlType());
        return Optional.of((o1, o2) -> castLeft.cast(o1).compareTo(castRight.cast(o2)));
    } else if (either(leftType, rightType, SqlBaseType.DOUBLE)) {
        final ComparableCastFunction<Double> castLeft = castToDoubleFunction(left.getSqlType());
        final ComparableCastFunction<Double> castRight = castToDoubleFunction(right.getSqlType());
        return Optional.of((o1, o2) -> castLeft.cast(o1).compareTo(castRight.cast(o2)));
    } else if (either(leftType, rightType, SqlBaseType.BIGINT)) {
        final ComparableCastFunction<Long> castLeft = castToLongFunction(left.getSqlType());
        final ComparableCastFunction<Long> castRight = castToLongFunction(right.getSqlType());
        return Optional.of((o1, o2) -> castLeft.cast(o1).compareTo(castRight.cast(o2)));
    } else if (either(leftType, rightType, SqlBaseType.INTEGER)) {
        final ComparableCastFunction<Integer> castLeft = castToIntegerFunction(left.getSqlType());
        final ComparableCastFunction<Integer> castRight = castToIntegerFunction(right.getSqlType());
        return Optional.of((o1, o2) -> castLeft.cast(o1).compareTo(castRight.cast(o2)));
    }
    return Optional.empty();
}
Also used : CastInterpreter.castToLongFunction(io.confluent.ksql.execution.interpreter.CastInterpreter.castToLongFunction) CastInterpreter.castToBytesFunction(io.confluent.ksql.execution.interpreter.CastInterpreter.castToBytesFunction) EqualsTerm(io.confluent.ksql.execution.interpreter.terms.ComparisonTerm.EqualsTerm) ComparisonCheckFunction(io.confluent.ksql.execution.interpreter.terms.ComparisonTerm.ComparisonCheckFunction) CastInterpreter.castToDoubleFunction(io.confluent.ksql.execution.interpreter.CastInterpreter.castToDoubleFunction) Date(java.util.Date) CastInterpreter.castToIntegerFunction(io.confluent.ksql.execution.interpreter.CastInterpreter.castToIntegerFunction) CastInterpreter.castToBigDecimalFunction(io.confluent.ksql.execution.interpreter.CastInterpreter.castToBigDecimalFunction) SqlBaseType(io.confluent.ksql.schema.ksql.types.SqlBaseType) ByteBuffer(java.nio.ByteBuffer) CastInterpreter.castToDateFunction(io.confluent.ksql.execution.interpreter.CastInterpreter.castToDateFunction) ComparisonFunction(io.confluent.ksql.execution.interpreter.terms.ComparisonTerm.ComparisonFunction) BigDecimal(java.math.BigDecimal) CastInterpreter.castToTimeFunction(io.confluent.ksql.execution.interpreter.CastInterpreter.castToTimeFunction) ComparableCastFunction(io.confluent.ksql.execution.interpreter.terms.CastTerm.ComparableCastFunction) Term(io.confluent.ksql.execution.interpreter.terms.Term) CompareToTerm(io.confluent.ksql.execution.interpreter.terms.ComparisonTerm.CompareToTerm) ComparisonExpression(io.confluent.ksql.execution.expression.tree.ComparisonExpression) ComparisonNullCheckFunction(io.confluent.ksql.execution.interpreter.terms.ComparisonTerm.ComparisonNullCheckFunction) CastInterpreter.castToTimestampFunction(io.confluent.ksql.execution.interpreter.CastInterpreter.castToTimestampFunction) KsqlException(io.confluent.ksql.util.KsqlException) Optional(java.util.Optional) EqualsFunction(io.confluent.ksql.execution.interpreter.terms.ComparisonTerm.EqualsFunction) EqualsCheckFunction(io.confluent.ksql.execution.interpreter.terms.ComparisonTerm.EqualsCheckFunction) ComparableCastFunction(io.confluent.ksql.execution.interpreter.terms.CastTerm.ComparableCastFunction) SqlBaseType(io.confluent.ksql.schema.ksql.types.SqlBaseType) BigDecimal(java.math.BigDecimal) Date(java.util.Date)

Aggregations

SqlBaseType (io.confluent.ksql.schema.ksql.types.SqlBaseType)11 KsqlException (io.confluent.ksql.util.KsqlException)6 Test (org.junit.Test)6 SqlType (io.confluent.ksql.schema.ksql.types.SqlType)4 SqlTypes (io.confluent.ksql.schema.ksql.types.SqlTypes)3 BigDecimal (java.math.BigDecimal)3 ImmutableList (com.google.common.collect.ImmutableList)2 ImmutableMap (com.google.common.collect.ImmutableMap)2 CastInterpreter.castToDoubleFunction (io.confluent.ksql.execution.interpreter.CastInterpreter.castToDoubleFunction)2 CastInterpreter.castToIntegerFunction (io.confluent.ksql.execution.interpreter.CastInterpreter.castToIntegerFunction)2 CastInterpreter.castToLongFunction (io.confluent.ksql.execution.interpreter.CastInterpreter.castToLongFunction)2 ComparableCastFunction (io.confluent.ksql.execution.interpreter.terms.CastTerm.ComparableCastFunction)2 Term (io.confluent.ksql.execution.interpreter.terms.Term)2 ByteBuffer (java.nio.ByteBuffer)2 List (java.util.List)2 Map (java.util.Map)2 Optional (java.util.Optional)2 Collectors (java.util.stream.Collectors)2 MatcherAssert.assertThat (org.hamcrest.MatcherAssert.assertThat)2 Matchers.is (org.hamcrest.Matchers.is)2