Search in sources :

Example 1 with IndexColumnName

use of com.emc.storageos.db.client.impl.IndexColumnName in project coprhd-controller by CoprHD.

the class DbConsistencyCheckerHelperTest method testIsIndexExists.

@Test
public void testIsIndexExists() throws Exception {
    FileShare testData = new FileShare();
    testData.setId(URIUtil.createId(FileShare.class));
    testData.setPath("path1");
    testData.setMountPath("mountPath1");
    getDbClient().createObject(testData);
    ColumnFamily<String, IndexColumnName> indexCF = new ColumnFamily<String, IndexColumnName>("AltIdIndex", StringSerializer.get(), IndexColumnNameSerializer.get());
    Keyspace keyspace = ((DbClientImpl) getDbClient()).getLocalContext().getKeyspace();
    CompositeRangeBuilder builder = IndexColumnNameSerializer.get().buildRange();
    builder.withPrefix("FileShare").greaterThanEquals(testData.getId().toString()).lessThanEquals(testData.getId().toString());
    Rows<String, IndexColumnName> result = keyspace.prepareQuery(indexCF).getAllRows().withColumnRange(builder).execute().getResult();
    for (Row<String, IndexColumnName> row : result) {
        System.out.println(row.getColumns().getColumnByIndex(0).getName());
        assertTrue(helper.isIndexExists(keyspace, indexCF, row.getKey(), row.getColumns().getColumnByIndex(0).getName()));
    }
    ((DbClientImpl) getDbClient()).internalRemoveObjects(testData);
    for (Row<String, IndexColumnName> row : result) {
        assertFalse(helper.isIndexExists(keyspace, indexCF, row.getKey(), row.getColumns().getColumnByIndex(0).getName()));
    }
}
Also used : CompositeIndexColumnName(com.emc.storageos.db.client.impl.CompositeIndexColumnName) TimeSeriesIndexColumnName(com.emc.storageos.db.client.impl.TimeSeriesIndexColumnName) ClassNameTimeSeriesIndexColumnName(com.emc.storageos.db.client.impl.ClassNameTimeSeriesIndexColumnName) IndexColumnName(com.emc.storageos.db.client.impl.IndexColumnName) DbClientImpl(com.emc.storageos.db.client.impl.DbClientImpl) Keyspace(com.netflix.astyanax.Keyspace) CompositeRangeBuilder(com.netflix.astyanax.serializers.CompositeRangeBuilder) FileShare(com.emc.storageos.db.client.model.FileShare) ColumnFamily(com.netflix.astyanax.model.ColumnFamily) Test(org.junit.Test)

Example 2 with IndexColumnName

use of com.emc.storageos.db.client.impl.IndexColumnName in project coprhd-controller by CoprHD.

the class DbConsistencyCheckerHelperTest method testCheckIndexingCF.

@Test
public void testCheckIndexingCF() throws Exception {
    ColumnFamily<String, CompositeColumnName> cf = new ColumnFamily<String, CompositeColumnName>("FileShare", StringSerializer.get(), CompositeColumnNameSerializer.get());
    FileShare testData = new FileShare();
    testData.setId(URIUtil.createId(FileShare.class));
    testData.setPath("path1");
    testData.setMountPath("mountPath1");
    getDbClient().createObject(testData);
    Keyspace keyspace = ((DbClientImpl) getDbClient()).getLocalContext().getKeyspace();
    // delete data object
    MutationBatch mutationBatch = keyspace.prepareMutationBatch();
    mutationBatch.withRow(cf, testData.getId().toString()).delete();
    mutationBatch.execute();
    CheckResult checkResult = new CheckResult();
    ColumnFamily<String, IndexColumnName> indexCF = new ColumnFamily<String, IndexColumnName>("AltIdIndex", StringSerializer.get(), IndexColumnNameSerializer.get());
    // find inconsistency: index exits but data object is deleted
    IndexAndCf indexAndCf = new IndexAndCf(AltIdDbIndex.class, indexCF, keyspace);
    helper.checkIndexingCF(indexAndCf, false, checkResult);
    assertEquals(2, checkResult.getTotal());
    testData = new FileShare();
    testData.setId(URIUtil.createId(FileShare.class));
    testData.setPath("path2");
    testData.setMountPath("mountPath2");
    getDbClient().createObject(testData);
    // create duplicated index
    keyspace.prepareQuery(indexCF).withCql(String.format("INSERT INTO \"AltIdIndex\" (key, column1, column2, column3, column4, column5, value) VALUES ('pa', 'FileShare', '%s', '', '', now(), intasblob(10));", testData.getId().toString())).execute();
    checkResult = new CheckResult();
    helper.checkIndexingCF(indexAndCf, false, checkResult);
    assertEquals(3, checkResult.getTotal());
    keyspace.prepareQuery(indexCF).withCql("TRUNCATE \"AltIdIndex\"").execute();
    // test large columns for single row key
    for (int i = 0; i < 123; i++) {
        keyspace.prepareQuery(indexCF).withCql(String.format("INSERT INTO \"AltIdIndex\" (key, column1, column2, column3, column4, column5, value) VALUES ('sa', 'FileShare', '%s', '', '', now(), intasblob(10));", i)).execute();
    }
    checkResult = new CheckResult();
    helper.checkIndexingCF(indexAndCf, false, checkResult);
    assertEquals(123, checkResult.getTotal());
}
Also used : CompositeColumnName(com.emc.storageos.db.client.impl.CompositeColumnName) MutationBatch(com.netflix.astyanax.MutationBatch) CompositeIndexColumnName(com.emc.storageos.db.client.impl.CompositeIndexColumnName) TimeSeriesIndexColumnName(com.emc.storageos.db.client.impl.TimeSeriesIndexColumnName) ClassNameTimeSeriesIndexColumnName(com.emc.storageos.db.client.impl.ClassNameTimeSeriesIndexColumnName) IndexColumnName(com.emc.storageos.db.client.impl.IndexColumnName) Keyspace(com.netflix.astyanax.Keyspace) CheckResult(com.emc.storageos.db.client.impl.DbConsistencyCheckerHelper.CheckResult) IndexAndCf(com.emc.storageos.db.client.impl.DbConsistencyCheckerHelper.IndexAndCf) FileShare(com.emc.storageos.db.client.model.FileShare) ColumnFamily(com.netflix.astyanax.model.ColumnFamily) Test(org.junit.Test)

Example 3 with IndexColumnName

use of com.emc.storageos.db.client.impl.IndexColumnName in project coprhd-controller by CoprHD.

the class RowMutationTest method insertRecordAndIndexColumnWithError.

@Test
public void insertRecordAndIndexColumnWithError() throws ConnectionException {
    String rowKey = URIUtil.createId(Volume.class).toString();
    String volumeLabel = "volume label";
    // insert data object
    ColumnListMutation<CompositeColumnName> columnList = rowMutator.getRecordColumnList(volumeCF, rowKey);
    columnList.putColumn(new CompositeColumnName("allocatedCapacity"), 20);
    columnList.putColumn(new CompositeColumnName("label"), volumeLabel);
    // insert related index
    rowMutator.getIndexColumnList(indexCF, "vo").putColumn(new IndexColumnName("Volume", volumeLabel, volumeLabel, rowKey, rowMutator.getTimeUUID()), "");
    // insert error column
    ColumnListMutation<CompositeColumnName> no_columnList = rowMutator.getRecordColumnList(noExistCF, rowKey);
    no_columnList.putColumn(new CompositeColumnName("test"), 20);
    try {
        rowMutator.execute();
        Assert.fail();
    } catch (Exception e) {
    // expected exception
    }
    // no volume should be created
    Volume volume = (Volume) this.getDbClient().queryObject(URI.create(rowKey));
    Assert.assertNull(volume);
    // no index should be created
    CqlStatement statement = ((DbClientImpl) this.getDbClient()).getLocalContext().getKeyspace().prepareCqlStatement();
    String cql = String.format("select * from \"LabelPrefixIndex\" where key='%s' and column1='Volume' and column2='%s' and column3='%s' and column4='%s'", "vo", volumeLabel, volumeLabel, rowKey);
    CqlStatementResult result = statement.withCql(cql).execute().getResult();
    Rows<String, IndexColumnName> rows = result.getRows(indexCF);
    Assert.assertEquals(rows.size(), 0);
}
Also used : CompositeColumnName(com.emc.storageos.db.client.impl.CompositeColumnName) Volume(com.emc.storageos.db.client.model.Volume) IndexColumnName(com.emc.storageos.db.client.impl.IndexColumnName) DbClientImpl(com.emc.storageos.db.client.impl.DbClientImpl) CqlStatement(com.netflix.astyanax.cql.CqlStatement) ConnectionException(com.netflix.astyanax.connectionpool.exceptions.ConnectionException) CqlStatementResult(com.netflix.astyanax.cql.CqlStatementResult) Test(org.junit.Test)

Example 4 with IndexColumnName

use of com.emc.storageos.db.client.impl.IndexColumnName in project coprhd-controller by CoprHD.

the class RowMutationTest method setupTest.

@Before
public void setupTest() {
    volumeCF = new ColumnFamily<String, CompositeColumnName>("Volume", StringSerializer.get(), CompositeColumnNameSerializer.get());
    indexCF = new ColumnFamily<String, IndexColumnName>("LabelPrefixIndex", StringSerializer.get(), IndexColumnNameSerializer.get());
    noExistCF = new ColumnFamily<String, CompositeColumnName>("no_exits_CF", StringSerializer.get(), CompositeColumnNameSerializer.get());
    rowMutator = new RowMutator(((DbClientImpl) this.getDbClient()).getLocalContext().getKeyspace(), false);
}
Also used : CompositeColumnName(com.emc.storageos.db.client.impl.CompositeColumnName) IndexColumnName(com.emc.storageos.db.client.impl.IndexColumnName) RowMutator(com.emc.storageos.db.client.impl.RowMutator) Before(org.junit.Before)

Example 5 with IndexColumnName

use of com.emc.storageos.db.client.impl.IndexColumnName in project coprhd-controller by CoprHD.

the class UserToOrdersMigration method process.

@Override
public void process() throws MigrationCallbackException {
    log.info("Adding new index records for class: {} field: {} annotation: {} name={}", new Object[] { Order.class, Order.SUBMITTED_BY_USER_ID, ClassNameTimeSeries.class.getName(), name });
    DataObjectType doType = TypeMap.getDoType(Order.class);
    ColumnField field = doType.getColumnField(Order.SUBMITTED_BY_USER_ID);
    newIndexCF = field.getIndexCF();
    ColumnFamily<String, IndexColumnName> userToOrders = new ColumnFamily<>(SOURCE_INDEX_CF_NAME, StringSerializer.get(), IndexColumnNameSerializer.get());
    DbClientImpl client = (DbClientImpl) dbClient;
    ks = client.getKeyspace(Order.class);
    MutationBatch mutationBatch = ks.prepareMutationBatch();
    long m = 0;
    try {
        OperationResult<Rows<String, IndexColumnName>> result = ks.prepareQuery(userToOrders).getAllRows().setRowLimit(1000).withColumnRange(new RangeBuilder().setLimit(0).build()).execute();
        ColumnList<IndexColumnName> cols;
        for (Row<String, IndexColumnName> row : result.getResult()) {
            RowQuery<String, IndexColumnName> rowQuery = ks.prepareQuery(userToOrders).getKey(row.getKey()).autoPaginate(true).withColumnRange(new RangeBuilder().setLimit(5).build());
            while (!(cols = rowQuery.execute().getResult()).isEmpty()) {
                m++;
                for (Column<IndexColumnName> col : cols) {
                    String indexKey = row.getKey();
                    String orderId = col.getName().getTwo();
                    ClassNameTimeSeriesIndexColumnName newCol = new ClassNameTimeSeriesIndexColumnName(col.getName().getOne(), orderId, col.getName().getTimeUUID());
                    mutationBatch.withRow(newIndexCF, indexKey).putEmptyColumn(newCol, null);
                    if (m % 10000 == 0) {
                        mutationBatch.execute();
                    }
                }
            }
        }
        mutationBatch.execute();
    } catch (Exception e) {
        log.error("Migration to {} failed e=", newIndexCF.getName(), e);
    }
}
Also used : Order(com.emc.storageos.db.client.model.uimodels.Order) ColumnField(com.emc.storageos.db.client.impl.ColumnField) RangeBuilder(com.netflix.astyanax.util.RangeBuilder) ClassNameTimeSeriesIndexColumnName(com.emc.storageos.db.client.impl.ClassNameTimeSeriesIndexColumnName) MigrationCallbackException(com.emc.storageos.svcs.errorhandling.resources.MigrationCallbackException) ColumnFamily(com.netflix.astyanax.model.ColumnFamily) ClassNameTimeSeries(com.emc.storageos.db.client.model.ClassNameTimeSeries) ClassNameTimeSeriesIndexColumnName(com.emc.storageos.db.client.impl.ClassNameTimeSeriesIndexColumnName) IndexColumnName(com.emc.storageos.db.client.impl.IndexColumnName) MutationBatch(com.netflix.astyanax.MutationBatch) DbClientImpl(com.emc.storageos.db.client.impl.DbClientImpl) DataObjectType(com.emc.storageos.db.client.impl.DataObjectType) Rows(com.netflix.astyanax.model.Rows)

Aggregations

IndexColumnName (com.emc.storageos.db.client.impl.IndexColumnName)9 DbClientImpl (com.emc.storageos.db.client.impl.DbClientImpl)6 Test (org.junit.Test)6 CompositeColumnName (com.emc.storageos.db.client.impl.CompositeColumnName)5 ClassNameTimeSeriesIndexColumnName (com.emc.storageos.db.client.impl.ClassNameTimeSeriesIndexColumnName)4 Keyspace (com.netflix.astyanax.Keyspace)4 ColumnFamily (com.netflix.astyanax.model.ColumnFamily)4 CompositeIndexColumnName (com.emc.storageos.db.client.impl.CompositeIndexColumnName)3 TimeSeriesIndexColumnName (com.emc.storageos.db.client.impl.TimeSeriesIndexColumnName)3 FileShare (com.emc.storageos.db.client.model.FileShare)3 Volume (com.emc.storageos.db.client.model.Volume)3 CqlStatement (com.netflix.astyanax.cql.CqlStatement)3 CqlStatementResult (com.netflix.astyanax.cql.CqlStatementResult)3 ColumnField (com.emc.storageos.db.client.impl.ColumnField)2 DataObjectType (com.emc.storageos.db.client.impl.DataObjectType)2 CheckResult (com.emc.storageos.db.client.impl.DbConsistencyCheckerHelper.CheckResult)2 MutationBatch (com.netflix.astyanax.MutationBatch)2 Rows (com.netflix.astyanax.model.Rows)2 com.emc.storageos.db.client.constraint (com.emc.storageos.db.client.constraint)1 DbConsistencyCheckerHelper (com.emc.storageos.db.client.impl.DbConsistencyCheckerHelper)1