Search in sources :

Example 51 with Cell

use of org.apache.hadoop.hbase.Cell in project hbase by apache.

the class SampleRegionWALObserver method preWALWrite.

@Override
public boolean preWALWrite(ObserverContext<? extends WALCoprocessorEnvironment> env, HRegionInfo info, WALKey logKey, WALEdit logEdit) throws IOException {
    boolean bypass = false;
    // check table name matches or not.
    if (!Bytes.equals(info.getTable().toBytes(), this.tableName)) {
        return bypass;
    }
    preWALWriteCalled = true;
    // here we're going to remove one keyvalue from the WALEdit, and add
    // another one to it.
    List<Cell> cells = logEdit.getCells();
    Cell deletedCell = null;
    for (Cell cell : cells) {
        // assume only one kv from the WALEdit matches.
        byte[] family = CellUtil.cloneFamily(cell);
        byte[] qulifier = CellUtil.cloneQualifier(cell);
        if (Arrays.equals(family, ignoredFamily) && Arrays.equals(qulifier, ignoredQualifier)) {
            LOG.debug("Found the KeyValue from WALEdit which should be ignored.");
            deletedCell = cell;
        }
        if (Arrays.equals(family, changedFamily) && Arrays.equals(qulifier, changedQualifier)) {
            LOG.debug("Found the KeyValue from WALEdit which should be changed.");
            cell.getValueArray()[cell.getValueOffset()] += 1;
        }
    }
    if (null != row) {
        cells.add(new KeyValue(row, addedFamily, addedQualifier));
    }
    if (deletedCell != null) {
        LOG.debug("About to delete a KeyValue from WALEdit.");
        cells.remove(deletedCell);
    }
    return bypass;
}
Also used : KeyValue(org.apache.hadoop.hbase.KeyValue) Cell(org.apache.hadoop.hbase.Cell)

Example 52 with Cell

use of org.apache.hadoop.hbase.Cell in project hbase by apache.

the class SimpleRegionObserver method postGetOp.

@Override
public void postGetOp(final ObserverContext<RegionCoprocessorEnvironment> c, final Get get, final List<Cell> results) {
    RegionCoprocessorEnvironment e = c.getEnvironment();
    assertNotNull(e);
    assertNotNull(e.getRegion());
    assertNotNull(get);
    assertNotNull(results);
    if (e.getRegion().getTableDesc().getTableName().equals(TestRegionObserverInterface.TEST_TABLE)) {
        boolean foundA = false;
        boolean foundB = false;
        boolean foundC = false;
        for (Cell kv : results) {
            if (CellUtil.matchingFamily(kv, TestRegionObserverInterface.A)) {
                foundA = true;
            }
            if (CellUtil.matchingFamily(kv, TestRegionObserverInterface.B)) {
                foundB = true;
            }
            if (CellUtil.matchingFamily(kv, TestRegionObserverInterface.C)) {
                foundC = true;
            }
        }
        assertTrue(foundA);
        assertTrue(foundB);
        assertTrue(foundC);
    }
    ctPostGet.incrementAndGet();
}
Also used : Cell(org.apache.hadoop.hbase.Cell)

Example 53 with Cell

use of org.apache.hadoop.hbase.Cell in project hbase by apache.

the class SimpleRegionObserver method postPut.

@Override
public void postPut(final ObserverContext<RegionCoprocessorEnvironment> c, final Put put, final WALEdit edit, final Durability durability) throws IOException {
    Map<byte[], List<Cell>> familyMap = put.getFamilyCellMap();
    RegionCoprocessorEnvironment e = c.getEnvironment();
    assertNotNull(e);
    assertNotNull(e.getRegion());
    assertNotNull(familyMap);
    List<Cell> cells = familyMap.get(TestRegionObserverInterface.A);
    if (e.getRegion().getTableDesc().getTableName().equals(TestRegionObserverInterface.TEST_TABLE)) {
        assertNotNull(cells);
        assertNotNull(cells.get(0));
        // KeyValue v1 expectation.  Cast for now until we go all Cell all the time. TODO
        Cell cell = cells.get(0);
        assertTrue(Bytes.equals(cell.getQualifierArray(), cell.getQualifierOffset(), cell.getQualifierLength(), TestRegionObserverInterface.A, 0, TestRegionObserverInterface.A.length));
        cells = familyMap.get(TestRegionObserverInterface.B);
        assertNotNull(cells);
        assertNotNull(cells.get(0));
        // KeyValue v1 expectation.  Cast for now until we go all Cell all the time. TODO
        cell = cells.get(0);
        assertTrue(Bytes.equals(cell.getQualifierArray(), cell.getQualifierOffset(), cell.getQualifierLength(), TestRegionObserverInterface.B, 0, TestRegionObserverInterface.B.length));
        cells = familyMap.get(TestRegionObserverInterface.C);
        assertNotNull(cells);
        assertNotNull(cells.get(0));
        // KeyValue v1 expectation.  Cast for now until we go all Cell all the time. TODO
        cell = cells.get(0);
        assertTrue(Bytes.equals(cell.getQualifierArray(), cell.getQualifierOffset(), cell.getQualifierLength(), TestRegionObserverInterface.C, 0, TestRegionObserverInterface.C.length));
    }
    ctPostPut.incrementAndGet();
}
Also used : ImmutableList(com.google.common.collect.ImmutableList) List(java.util.List) Cell(org.apache.hadoop.hbase.Cell)

Example 54 with Cell

use of org.apache.hadoop.hbase.Cell in project hbase by apache.

the class SimpleRegionObserver method prePut.

@Override
public void prePut(final ObserverContext<RegionCoprocessorEnvironment> c, final Put put, final WALEdit edit, final Durability durability) throws IOException {
    Map<byte[], List<Cell>> familyMap = put.getFamilyCellMap();
    RegionCoprocessorEnvironment e = c.getEnvironment();
    assertNotNull(e);
    assertNotNull(e.getRegion());
    assertNotNull(familyMap);
    if (e.getRegion().getTableDesc().getTableName().equals(TestRegionObserverInterface.TEST_TABLE)) {
        List<Cell> cells = familyMap.get(TestRegionObserverInterface.A);
        assertNotNull(cells);
        assertNotNull(cells.get(0));
        Cell cell = cells.get(0);
        assertTrue(Bytes.equals(cell.getQualifierArray(), cell.getQualifierOffset(), cell.getQualifierLength(), TestRegionObserverInterface.A, 0, TestRegionObserverInterface.A.length));
        cells = familyMap.get(TestRegionObserverInterface.B);
        assertNotNull(cells);
        assertNotNull(cells.get(0));
        cell = cells.get(0);
        assertTrue(Bytes.equals(cell.getQualifierArray(), cell.getQualifierOffset(), cell.getQualifierLength(), TestRegionObserverInterface.B, 0, TestRegionObserverInterface.B.length));
        cells = familyMap.get(TestRegionObserverInterface.C);
        assertNotNull(cells);
        assertNotNull(cells.get(0));
        cell = cells.get(0);
        assertTrue(Bytes.equals(cell.getQualifierArray(), cell.getQualifierOffset(), cell.getQualifierLength(), TestRegionObserverInterface.C, 0, TestRegionObserverInterface.C.length));
    }
    ctPrePut.incrementAndGet();
}
Also used : ImmutableList(com.google.common.collect.ImmutableList) List(java.util.List) Cell(org.apache.hadoop.hbase.Cell)

Example 55 with Cell

use of org.apache.hadoop.hbase.Cell in project hbase by apache.

the class CodecPerformance method main.

public static void main(String[] args) throws IOException {
    // How many Cells to encode/decode on each cycle.
    final int count = 100000;
    // How many times to do an operation; repeat gives hotspot chance to warm up.
    final int cycles = 30;
    Cell[] cells = getCells(count);
    int size = getRoughSize(cells);
    // Multiply by 2 to ensure we don't have to grow buffer
    int initialBufferSize = 2 * size;
    // Test KeyValue codec.
    doCodec(new KeyValueCodec(), cells, cycles, count, initialBufferSize);
    doCodec(new CellCodec(), cells, cycles, count, initialBufferSize);
    doCodec(new MessageCodec(), cells, cycles, count, initialBufferSize);
}
Also used : CellCodec(org.apache.hadoop.hbase.codec.CellCodec) KeyValueCodec(org.apache.hadoop.hbase.codec.KeyValueCodec) MessageCodec(org.apache.hadoop.hbase.codec.MessageCodec) Cell(org.apache.hadoop.hbase.Cell)

Aggregations

Cell (org.apache.hadoop.hbase.Cell)862 Test (org.junit.Test)326 ArrayList (java.util.ArrayList)323 Scan (org.apache.hadoop.hbase.client.Scan)258 KeyValue (org.apache.hadoop.hbase.KeyValue)220 Result (org.apache.hadoop.hbase.client.Result)203 Put (org.apache.hadoop.hbase.client.Put)159 IOException (java.io.IOException)123 ResultScanner (org.apache.hadoop.hbase.client.ResultScanner)106 Get (org.apache.hadoop.hbase.client.Get)85 Table (org.apache.hadoop.hbase.client.Table)85 List (java.util.List)80 TableName (org.apache.hadoop.hbase.TableName)77 Delete (org.apache.hadoop.hbase.client.Delete)75 CellScanner (org.apache.hadoop.hbase.CellScanner)69 Configuration (org.apache.hadoop.conf.Configuration)62 InterruptedIOException (java.io.InterruptedIOException)48 Map (java.util.Map)45 Path (org.apache.hadoop.fs.Path)45 RegionScanner (org.apache.hadoop.hbase.regionserver.RegionScanner)45