Search in sources :

Example 11 with WriteOnly

use of co.cask.cdap.api.annotation.WriteOnly in project cdap by caskdata.

the class IndexedObjectStore method write.

/**
   * Writes to the dataset, deleting any existing secondaryKey corresponding to the key and updates the indexTable with
   * the secondaryKey that is passed.
   *
   * @param key key for storing the object
   * @param object object to be stored
   * @param secondaryKeys indices that can be used to lookup the object
   */
@WriteOnly
public void write(byte[] key, T object, byte[][] secondaryKeys) {
    writeToObjectStore(key, object);
    //Update the secondaryKeys
    //logic:
    //  - Get existing secondary keys
    //  - Compute diff between existing secondary keys and new secondary keys
    //  - Remove the secondaryKeys that are removed
    //  - Add the new keys that are added
    Row row = index.get(getPrefixedPrimaryKey(key));
    Set<byte[]> existingSecondaryKeys = new TreeSet<>(Bytes.BYTES_COMPARATOR);
    if (!row.isEmpty()) {
        existingSecondaryKeys = row.getColumns().keySet();
    }
    Set<byte[]> newSecondaryKeys = new TreeSet<>(Bytes.BYTES_COMPARATOR);
    newSecondaryKeys.addAll(Arrays.asList(secondaryKeys));
    List<byte[]> secondaryKeysDeleted = secondaryKeysToDelete(existingSecondaryKeys, newSecondaryKeys);
    if (secondaryKeysDeleted.size() > 0) {
        deleteSecondaryKeys(key, secondaryKeysDeleted.toArray(new byte[secondaryKeysDeleted.size()][]));
    }
    List<byte[]> secondaryKeysAdded = secondaryKeysToAdd(existingSecondaryKeys, newSecondaryKeys);
    //for each key store the secondaryKey. This will be used while deleting old index values.
    if (secondaryKeysAdded.size() > 0) {
        byte[][] fooValues = new byte[secondaryKeysAdded.size()][];
        Arrays.fill(fooValues, EMPTY_VALUE);
        index.put(getPrefixedPrimaryKey(key), secondaryKeysAdded.toArray(new byte[secondaryKeysAdded.size()][]), fooValues);
    }
    for (byte[] secondaryKey : secondaryKeysAdded) {
        //update the index.
        index.put(secondaryKey, key, EMPTY_VALUE);
    }
}
Also used : TreeSet(java.util.TreeSet) Row(co.cask.cdap.api.dataset.table.Row) WriteOnly(co.cask.cdap.api.annotation.WriteOnly)

Example 12 with WriteOnly

use of co.cask.cdap.api.annotation.WriteOnly in project cdap by caskdata.

the class IndexedObjectStore method write.

@WriteOnly
public void write(byte[] key, T object) {
    Row row = index.get(getPrefixedPrimaryKey(key));
    if (!row.isEmpty()) {
        Set<byte[]> columnsToDelete = row.getColumns().keySet();
        deleteSecondaryKeys(key, columnsToDelete.toArray(new byte[columnsToDelete.size()][]));
    }
    writeToObjectStore(key, object);
}
Also used : Row(co.cask.cdap.api.dataset.table.Row) WriteOnly(co.cask.cdap.api.annotation.WriteOnly)

Aggregations

WriteOnly (co.cask.cdap.api.annotation.WriteOnly)12 Row (co.cask.cdap.api.dataset.table.Row)6 Put (co.cask.cdap.api.dataset.table.Put)5 DataSetException (co.cask.cdap.api.dataset.DataSetException)4 IOException (java.io.IOException)3 Map (java.util.Map)2 TreeSet (java.util.TreeSet)2 PartitionNotFoundException (co.cask.cdap.api.dataset.PartitionNotFoundException)1 PartitionDetail (co.cask.cdap.api.dataset.lib.PartitionDetail)1 ImmutableMap (com.google.common.collect.ImmutableMap)1 HashMap (java.util.HashMap)1 NavigableMap (java.util.NavigableMap)1 TreeMap (java.util.TreeMap)1 Location (org.apache.twill.filesystem.Location)1