Search in sources :

Example 1 with TupleType

use of org.apache.cassandra.db.marshal.TupleType in project cassandra by apache.

the class SSTableHeaderFixTest method freezeUdt.

private AbstractType<?> freezeUdt(AbstractType<?> type) {
    if (type instanceof CollectionType) {
        if (type.getClass() == ListType.class) {
            ListType<?> cHeader = (ListType<?>) type;
            return ListType.getInstance(freezeUdt(cHeader.getElementsType()), cHeader.isMultiCell());
        } else if (type.getClass() == SetType.class) {
            SetType<?> cHeader = (SetType<?>) type;
            return SetType.getInstance(freezeUdt(cHeader.getElementsType()), cHeader.isMultiCell());
        } else if (type.getClass() == MapType.class) {
            MapType<?, ?> cHeader = (MapType<?, ?>) type;
            return MapType.getInstance(freezeUdt(cHeader.getKeysType()), freezeUdt(cHeader.getValuesType()), cHeader.isMultiCell());
        }
    } else if (type instanceof AbstractCompositeType) {
        if (type.getClass() == CompositeType.class) {
            CompositeType cHeader = (CompositeType) type;
            return CompositeType.getInstance(cHeader.types.stream().map(this::freezeUdt).collect(Collectors.toList()));
        }
    } else if (type instanceof TupleType) {
        if (type.getClass() == UserType.class) {
            UserType cHeader = (UserType) type;
            cHeader = cHeader.freeze();
            return new UserType(cHeader.keyspace, cHeader.name, cHeader.fieldNames(), cHeader.allTypes().stream().map(this::freezeUdt).collect(Collectors.toList()), cHeader.isMultiCell());
        }
    }
    return type;
}
Also used : SetType(org.apache.cassandra.db.marshal.SetType) CollectionType(org.apache.cassandra.db.marshal.CollectionType) ListType(org.apache.cassandra.db.marshal.ListType) TupleType(org.apache.cassandra.db.marshal.TupleType) UserType(org.apache.cassandra.db.marshal.UserType) MapType(org.apache.cassandra.db.marshal.MapType) AbstractCompositeType(org.apache.cassandra.db.marshal.AbstractCompositeType) CompositeType(org.apache.cassandra.db.marshal.CompositeType) AbstractCompositeType(org.apache.cassandra.db.marshal.AbstractCompositeType)

Example 2 with TupleType

use of org.apache.cassandra.db.marshal.TupleType in project cassandra by apache.

the class SSTableHeaderFixTest method assertFrozenUdt.

private void assertFrozenUdt(String name, AbstractType<?> type, boolean frozen, boolean checkInner) {
    if (type instanceof CompositeType) {
        if (checkInner)
            for (AbstractType<?> component : ((CompositeType) type).types) assertFrozenUdt(name, component, frozen, true);
    } else if (type instanceof CollectionType) {
        if (checkInner) {
            if (type instanceof MapType) {
                MapType map = (MapType) type;
                // only descend for non-frozen types (checking frozen in frozen is just stupid)
                if (map.isMultiCell()) {
                    assertFrozenUdt(name + "<map-key>", map.getKeysType(), frozen, true);
                    assertFrozenUdt(name + "<map-value>", map.getValuesType(), frozen, true);
                }
            } else if (type instanceof SetType) {
                SetType set = (SetType) type;
                // only descend for non-frozen types (checking frozen in frozen is just stupid)
                if (set.isMultiCell())
                    assertFrozenUdt(name + "<set>", set.getElementsType(), frozen, true);
            } else if (type instanceof ListType) {
                ListType list = (ListType) type;
                // only descend for non-frozen types (checking frozen in frozen is just stupid)
                if (list.isMultiCell())
                    assertFrozenUdt(name + "<list>", list.getElementsType(), frozen, true);
            }
        }
    } else if (type instanceof TupleType) {
        if (checkInner) {
            TupleType tuple = (TupleType) type;
            // only descend for non-frozen types (checking frozen in frozen is just stupid)
            if (tuple.isMultiCell())
                for (AbstractType<?> component : tuple.allTypes()) assertFrozenUdt(name + "<tuple>", component, frozen, true);
        }
    }
    if (type instanceof UserType) {
        String typeString = type.toString();
        assertEquals(name + ": " + typeString, frozen, !type.isMultiCell());
        if (typeString.startsWith(UserType.class.getName() + '('))
            if (frozen)
                fail(name + ": " + typeString);
        if (typeString.startsWith(FrozenType.class.getName() + '(' + UserType.class.getName() + '('))
            if (!frozen)
                fail(name + ": " + typeString);
    }
}
Also used : SetType(org.apache.cassandra.db.marshal.SetType) CollectionType(org.apache.cassandra.db.marshal.CollectionType) ListType(org.apache.cassandra.db.marshal.ListType) AbstractType(org.apache.cassandra.db.marshal.AbstractType) TupleType(org.apache.cassandra.db.marshal.TupleType) UserType(org.apache.cassandra.db.marshal.UserType) MapType(org.apache.cassandra.db.marshal.MapType) CompositeType(org.apache.cassandra.db.marshal.CompositeType) AbstractCompositeType(org.apache.cassandra.db.marshal.AbstractCompositeType)

Example 3 with TupleType

use of org.apache.cassandra.db.marshal.TupleType in project cassandra by apache.

the class SSTableHeaderFix method fixTypeInnerTuple.

private AbstractType<?> fixTypeInnerTuple(TupleType cHeader, TupleType cSchema, boolean droppedColumnMode) {
    if (cHeader.size() != cSchema.size())
        // different number of components - bummer...
        return null;
    List<AbstractType<?>> cHeaderFixed = new ArrayList<>(cHeader.size());
    boolean anyChanged = false;
    for (int i = 0; i < cHeader.size(); i++) {
        AbstractType<?> cHeaderComp = cHeader.type(i);
        AbstractType<?> cHeaderCompFixed = fixTypeInner(cHeaderComp, cSchema.type(i), droppedColumnMode);
        if (cHeaderCompFixed == null)
            // incompatible, bummer...
            return null;
        cHeaderFixed.add(cHeaderCompFixed);
        anyChanged |= cHeaderComp != cHeaderCompFixed;
    }
    if (anyChanged || cSchema.isMultiCell() != cHeader.isMultiCell())
        // TODO this should create a non-frozen tuple type for the sake of handling a dropped, non-frozen UDT
        return new TupleType(cHeaderFixed);
    return cHeader;
}
Also used : AbstractType(org.apache.cassandra.db.marshal.AbstractType) ArrayList(java.util.ArrayList) TupleType(org.apache.cassandra.db.marshal.TupleType)

Example 4 with TupleType

use of org.apache.cassandra.db.marshal.TupleType in project cassandra by apache.

the class SSTableHeaderFixTest method variousDroppedUserTypes.

@Test
public void variousDroppedUserTypes() throws Exception {
    File dir = temporaryFolder;
    TableMetadata.Builder cols = TableMetadata.builder("ks", "cf").addPartitionKeyColumn("pk", udtPK).addClusteringColumn("ck", udtCK);
    ColSpec[] colSpecs = new ColSpec[] { // 'frozen<udt>' / live
    new ColSpec("frozen_udt_as_frozen_udt_live", makeUDT2("frozen_udt_as_frozen_udt_live", false), makeUDT2("frozen_udt_as_frozen_udt_live", false), false, false), // 'frozen<udt>' / live / as 'udt'
    new ColSpec("frozen_udt_as_unfrozen_udt_live", makeUDT2("frozen_udt_as_unfrozen_udt_live", false), makeUDT2("frozen_udt_as_unfrozen_udt_live", true), false, true), // 'frozen<udt>' / dropped
    new ColSpec("frozen_udt_as_frozen_udt_dropped", makeUDT2("frozen_udt_as_frozen_udt_dropped", true).freezeNestedMulticellTypes().freeze().expandUserTypes(), makeUDT2("frozen_udt_as_frozen_udt_dropped", false), makeUDT2("frozen_udt_as_frozen_udt_dropped", false), true, false), // 'frozen<udt>' / dropped / as 'udt'
    new ColSpec("frozen_udt_as_unfrozen_udt_dropped", makeUDT2("frozen_udt_as_unfrozen_udt_dropped", true).freezeNestedMulticellTypes().freeze().expandUserTypes(), makeUDT2("frozen_udt_as_unfrozen_udt_dropped", true), makeUDT2("frozen_udt_as_unfrozen_udt_dropped", false), true, true), // 'udt' / live
    new ColSpec("unfrozen_udt_as_unfrozen_udt_live", makeUDT2("unfrozen_udt_as_unfrozen_udt_live", true), makeUDT2("unfrozen_udt_as_unfrozen_udt_live", true), false, false), // 'frozen<tuple>' as 'TupleType(multiCell=false' (there is nothing like 'FrozenType(TupleType(')
    new ColSpec("frozen_tuple_as_frozen_tuple_live", makeTupleSimple(), makeTupleSimple(), false, false), // 'frozen<tuple>' as 'TupleType(multiCell=false' (there is nothing like 'FrozenType(TupleType(')
    new ColSpec("frozen_tuple_as_frozen_tuple_dropped", makeTupleSimple(), makeTupleSimple(), true, false) };
    Arrays.stream(colSpecs).forEach(c -> cols.addRegularColumn(c.name, // use the initial column type for the serialization header header.
    c.preFix));
    Map<String, ColSpec> colSpecMap = Arrays.stream(colSpecs).collect(Collectors.toMap(c -> c.name, c -> c));
    File sstable = buildFakeSSTable(dir, 1, cols, c -> {
        ColSpec cs = colSpecMap.get(c.name.toString());
        if (cs == null)
            return c;
        // update the column type in the schema to the "correct" one.
        return c.withNewType(cs.schema);
    });
    Arrays.stream(colSpecs).filter(c -> c.dropped).forEach(c -> {
        ColumnMetadata cd = getColDef(c.name);
        tableMetadata = tableMetadata.unbuild().removeRegularOrStaticColumn(cd.name).recordColumnDrop(cd, FBUtilities.timestampMicros()).build();
    });
    SerializationHeader.Component header = readHeader(sstable);
    for (ColSpec colSpec : colSpecs) {
        AbstractType<?> hdrType = header.getRegularColumns().get(ByteBufferUtil.bytes(colSpec.name));
        assertEquals(colSpec.name, colSpec.preFix, hdrType);
        assertEquals(colSpec.name, colSpec.preFix.isMultiCell(), hdrType.isMultiCell());
    }
    SSTableHeaderFix headerFix = builder().withPath(sstable.toPath()).build();
    headerFix.execute();
    assertFalse(headerFix.hasError());
    assertTrue(headerFix.hasChanges());
    // Verify that all columns to fix are in the updatedColumns set (paranoid, yet)
    Arrays.stream(colSpecs).filter(c -> c.mustFix).forEach(c -> assertTrue("expect " + c.name + " to be updated, but was not (" + updatedColumns + ")", updatedColumns.contains(c.name)));
    // Verify that the number of updated columns maches the expected number of columns to fix
    assertEquals(Arrays.stream(colSpecs).filter(c -> c.mustFix).count(), updatedColumns.size());
    header = readHeader(sstable);
    for (ColSpec colSpec : colSpecs) {
        AbstractType<?> hdrType = header.getRegularColumns().get(ByteBufferUtil.bytes(colSpec.name));
        assertEquals(colSpec.name, colSpec.expect, hdrType);
        assertEquals(colSpec.name, colSpec.expect.isMultiCell(), hdrType.isMultiCell());
    }
}
Also used : TableMetadata(org.apache.cassandra.schema.TableMetadata) Arrays(java.util.Arrays) IndexTarget(org.apache.cassandra.cql3.statements.schema.IndexTarget) File(org.apache.cassandra.io.util.File) AbstractType(org.apache.cassandra.db.marshal.AbstractType) ByteBuffer(java.nio.ByteBuffer) Matcher(java.util.regex.Matcher) Map(java.util.Map) After(org.junit.After) SSTableFormat(org.apache.cassandra.io.sstable.format.SSTableFormat) Assert.fail(org.junit.Assert.fail) DatabaseDescriptor(org.apache.cassandra.config.DatabaseDescriptor) CollectionType(org.apache.cassandra.db.marshal.CollectionType) CompositeType(org.apache.cassandra.db.marshal.CompositeType) SequentialWriter(org.apache.cassandra.io.util.SequentialWriter) Version(org.apache.cassandra.io.sstable.format.Version) FBUtilities(org.apache.cassandra.utils.FBUtilities) MetadataType(org.apache.cassandra.io.sstable.metadata.MetadataType) Set(java.util.Set) Collectors(java.util.stream.Collectors) Sets(com.google.common.collect.Sets) List(java.util.List) ColumnIdentifier(org.apache.cassandra.cql3.ColumnIdentifier) FloatType(org.apache.cassandra.db.marshal.FloatType) FileUtils(org.apache.cassandra.io.util.FileUtils) Assert.assertFalse(org.junit.Assert.assertFalse) FrozenType(org.apache.cassandra.db.marshal.FrozenType) TableMetadata(org.apache.cassandra.schema.TableMetadata) Pattern(java.util.regex.Pattern) IntStream(java.util.stream.IntStream) ColumnMetadata(org.apache.cassandra.schema.ColumnMetadata) SetType(org.apache.cassandra.db.marshal.SetType) FieldIdentifier(org.apache.cassandra.cql3.FieldIdentifier) Function(java.util.function.Function) HashSet(java.util.HashSet) Int32Type(org.apache.cassandra.db.marshal.Int32Type) ListType(org.apache.cassandra.db.marshal.ListType) UTF8Type(org.apache.cassandra.db.marshal.UTF8Type) AbstractCompositeType(org.apache.cassandra.db.marshal.AbstractCompositeType) TupleType(org.apache.cassandra.db.marshal.TupleType) BigFormat(org.apache.cassandra.io.sstable.format.big.BigFormat) SerializationHeader(org.apache.cassandra.db.SerializationHeader) Before(org.junit.Before) IndexMetadata(org.apache.cassandra.schema.IndexMetadata) ByteBufferUtil(org.apache.cassandra.utils.ByteBufferUtil) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) MapType(org.apache.cassandra.db.marshal.MapType) EncodingStats(org.apache.cassandra.db.rows.EncodingStats) Assert(org.junit.Assert) Collections(java.util.Collections) Assert.assertEquals(org.junit.Assert.assertEquals) UserType(org.apache.cassandra.db.marshal.UserType) ColumnMetadata(org.apache.cassandra.schema.ColumnMetadata) SerializationHeader(org.apache.cassandra.db.SerializationHeader) File(org.apache.cassandra.io.util.File) Test(org.junit.Test)

Aggregations

TupleType (org.apache.cassandra.db.marshal.TupleType)4 AbstractCompositeType (org.apache.cassandra.db.marshal.AbstractCompositeType)3 AbstractType (org.apache.cassandra.db.marshal.AbstractType)3 CollectionType (org.apache.cassandra.db.marshal.CollectionType)3 CompositeType (org.apache.cassandra.db.marshal.CompositeType)3 ListType (org.apache.cassandra.db.marshal.ListType)3 MapType (org.apache.cassandra.db.marshal.MapType)3 SetType (org.apache.cassandra.db.marshal.SetType)3 UserType (org.apache.cassandra.db.marshal.UserType)3 Sets (com.google.common.collect.Sets)1 ByteBuffer (java.nio.ByteBuffer)1 ArrayList (java.util.ArrayList)1 Arrays (java.util.Arrays)1 Collections (java.util.Collections)1 HashSet (java.util.HashSet)1 List (java.util.List)1 Map (java.util.Map)1 Set (java.util.Set)1 Function (java.util.function.Function)1 Matcher (java.util.regex.Matcher)1