Search in sources :

Example 1 with WriteOnly

use of co.cask.cdap.api.annotation.WriteOnly in project cdap by caskdata.

the class AbstractTable method write.

@WriteOnly
@Override
public void write(StructuredRecord structuredRecord) throws IOException {
    if (recordPutTransformer == null) {
        throw new IllegalStateException(String.format("Table must have both '%s' and '%s' properties set in " + "order to be used as a RecordWritable.", Table.PROPERTY_SCHEMA, Table.PROPERTY_SCHEMA_ROW_FIELD));
    }
    Put put = recordPutTransformer.toPut(structuredRecord);
    put(put);
}
Also used : Put(co.cask.cdap.api.dataset.table.Put) WriteOnly(co.cask.cdap.api.annotation.WriteOnly)

Example 2 with WriteOnly

use of co.cask.cdap.api.annotation.WriteOnly in project cdap by caskdata.

the class PartitionedFileSetDataset method dropPartition.

@WriteOnly
@Override
public void dropPartition(PartitionKey key) {
    byte[] rowKey = generateRowKey(key, partitioning);
    PartitionDetail partition = getPartition(key);
    if (partition == null) {
        // silently ignore non-existing partitions
        return;
    }
    // TODO: make DDL operations transactional [CDAP-1393]
    dropPartitionFromExplore(key);
    partitionsTable.delete(rowKey);
    if (!isExternal) {
        Location partitionLocation = partition.getLocation();
        try {
            if (partitionLocation.exists()) {
                Location dstLocation = getQuarantineLocation().append(partition.getRelativePath());
                Location dstParent = Locations.getParent(dstLocation);
                // shouldn't be null, since dstLocation was created by appending to a location, so it must have a parent
                Preconditions.checkNotNull(dstParent);
                // before moving into quarantine, we need to ensure that parent location exists
                if (!dstParent.exists()) {
                    if (!dstParent.mkdirs()) {
                        throw new DataSetException(String.format("Failed to create parent directory %s", dstParent));
                    }
                }
                partitionLocation.renameTo(dstLocation);
            }
        } catch (IOException ioe) {
            throw new DataSetException(String.format("Failed to move location %s into quarantine", partitionLocation));
        }
        operationsInThisTx.add(new DropPartitionOperation(key, partition.getRelativePath()));
    }
}
Also used : DataSetException(co.cask.cdap.api.dataset.DataSetException) IOException(java.io.IOException) PartitionDetail(co.cask.cdap.api.dataset.lib.PartitionDetail) Location(org.apache.twill.filesystem.Location) WriteOnly(co.cask.cdap.api.annotation.WriteOnly)

Example 3 with WriteOnly

use of co.cask.cdap.api.annotation.WriteOnly in project cdap by caskdata.

the class IndexedTable method put.

@WriteOnly
@Override
public void put(byte[] row, byte[] column, byte[] value) {
    Put put = new Put(row);
    put.add(column, value);
    put(put);
}
Also used : Put(co.cask.cdap.api.dataset.table.Put) WriteOnly(co.cask.cdap.api.annotation.WriteOnly)

Example 4 with WriteOnly

use of co.cask.cdap.api.annotation.WriteOnly in project cdap by caskdata.

the class IndexedTable method put.

@WriteOnly
@Override
public void put(byte[] row, byte[][] columns, byte[][] values) {
    Put put = new Put(row);
    for (int i = 0; i < columns.length; i++) {
        put.add(columns[i], values[i]);
    }
    put(put);
}
Also used : Put(co.cask.cdap.api.dataset.table.Put) WriteOnly(co.cask.cdap.api.annotation.WriteOnly)

Example 5 with WriteOnly

use of co.cask.cdap.api.annotation.WriteOnly in project cdap by caskdata.

the class PartitionedFileSetDataset method addMetadata.

@WriteOnly
@Override
public void addMetadata(PartitionKey key, Map<String, String> metadata) {
    final byte[] rowKey = generateRowKey(key, partitioning);
    Row row = partitionsTable.get(rowKey);
    if (row.isEmpty()) {
        throw new PartitionNotFoundException(key, getName());
    }
    // ensure that none of the entries already exist in the metadata
    for (Map.Entry<String, String> metadataEntry : metadata.entrySet()) {
        String metadataKey = metadataEntry.getKey();
        byte[] columnKey = columnKeyFromMetadataKey(metadataKey);
        if (row.get(columnKey) != null) {
            throw new DataSetException(String.format("Entry already exists for metadata key: %s", metadataKey));
        }
    }
    Put put = new Put(rowKey);
    addMetadataToPut(metadata, put);
    partitionsTable.put(put);
}
Also used : PartitionNotFoundException(co.cask.cdap.api.dataset.PartitionNotFoundException) DataSetException(co.cask.cdap.api.dataset.DataSetException) Row(co.cask.cdap.api.dataset.table.Row) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) HashMap(java.util.HashMap) Put(co.cask.cdap.api.dataset.table.Put) WriteOnly(co.cask.cdap.api.annotation.WriteOnly)

Aggregations

WriteOnly (co.cask.cdap.api.annotation.WriteOnly)12 Row (co.cask.cdap.api.dataset.table.Row)6 Put (co.cask.cdap.api.dataset.table.Put)5 DataSetException (co.cask.cdap.api.dataset.DataSetException)4 IOException (java.io.IOException)3 Map (java.util.Map)2 TreeSet (java.util.TreeSet)2 PartitionNotFoundException (co.cask.cdap.api.dataset.PartitionNotFoundException)1 PartitionDetail (co.cask.cdap.api.dataset.lib.PartitionDetail)1 ImmutableMap (com.google.common.collect.ImmutableMap)1 HashMap (java.util.HashMap)1 NavigableMap (java.util.NavigableMap)1 TreeMap (java.util.TreeMap)1 Location (org.apache.twill.filesystem.Location)1