Search in sources :

Example 6 with Writer

use of org.apache.accumulo.core.client.impl.Writer in project accumulo by apache.

the class MetadataTableUtil method update.

public static void update(ClientContext context, ZooLock zooLock, Mutation m, KeyExtent extent) {
    Writer t = extent.isMeta() ? getRootTable(context) : getMetadataTable(context);
    update(t, zooLock, m);
}
Also used : ZooReaderWriter(org.apache.accumulo.server.zookeeper.ZooReaderWriter) Writer(org.apache.accumulo.core.client.impl.Writer) IZooReaderWriter(org.apache.accumulo.fate.zookeeper.IZooReaderWriter) BatchWriter(org.apache.accumulo.core.client.BatchWriter)

Example 7 with Writer

use of org.apache.accumulo.core.client.impl.Writer in project accumulo by apache.

the class SplitRecoveryIT method splitPartiallyAndRecover.

private void splitPartiallyAndRecover(AccumuloServerContext context, KeyExtent extent, KeyExtent high, KeyExtent low, double splitRatio, SortedMap<FileRef, DataFileValue> mapFiles, Text midRow, String location, int steps, ZooLock zl) throws Exception {
    SortedMap<FileRef, DataFileValue> lowDatafileSizes = new TreeMap<>();
    SortedMap<FileRef, DataFileValue> highDatafileSizes = new TreeMap<>();
    List<FileRef> highDatafilesToRemove = new ArrayList<>();
    MetadataTableUtil.splitDatafiles(midRow, splitRatio, new HashMap<>(), mapFiles, lowDatafileSizes, highDatafileSizes, highDatafilesToRemove);
    MetadataTableUtil.splitTablet(high, extent.getPrevEndRow(), splitRatio, context, zl);
    TServerInstance instance = new TServerInstance(location, zl.getSessionId());
    Writer writer = MetadataTableUtil.getMetadataTable(context);
    Assignment assignment = new Assignment(high, instance);
    Mutation m = new Mutation(assignment.tablet.getMetadataEntry());
    assignment.server.putFutureLocation(m);
    writer.update(m);
    if (steps >= 1) {
        Map<Long, ? extends Collection<FileRef>> bulkFiles = MetadataTableUtil.getBulkFilesLoaded(context, extent);
        MasterMetadataUtil.addNewTablet(context, low, "/lowDir", instance, lowDatafileSizes, bulkFiles, TabletTime.LOGICAL_TIME_ID + "0", -1l, -1l, zl);
    }
    if (steps >= 2) {
        MetadataTableUtil.finishSplit(high, highDatafileSizes, highDatafilesToRemove, context, zl);
    }
    TabletServer.verifyTabletInformation(context, high, instance, new TreeMap<>(), "127.0.0.1:0", zl);
    if (steps >= 1) {
        ensureTabletHasNoUnexpectedMetadataEntries(context, low, lowDatafileSizes);
        ensureTabletHasNoUnexpectedMetadataEntries(context, high, highDatafileSizes);
        Map<Long, ? extends Collection<FileRef>> lowBulkFiles = MetadataTableUtil.getBulkFilesLoaded(context, low);
        Map<Long, ? extends Collection<FileRef>> highBulkFiles = MetadataTableUtil.getBulkFilesLoaded(context, high);
        if (!lowBulkFiles.equals(highBulkFiles)) {
            throw new Exception(" " + lowBulkFiles + " != " + highBulkFiles + " " + low + " " + high);
        }
        if (lowBulkFiles.size() == 0) {
            throw new Exception(" no bulk files " + low);
        }
    } else {
        ensureTabletHasNoUnexpectedMetadataEntries(context, extent, mapFiles);
    }
}
Also used : DataFileValue(org.apache.accumulo.core.metadata.schema.DataFileValue) ArrayList(java.util.ArrayList) TreeMap(java.util.TreeMap) TServerInstance(org.apache.accumulo.server.master.state.TServerInstance) Assignment(org.apache.accumulo.server.master.state.Assignment) FileRef(org.apache.accumulo.server.fs.FileRef) Mutation(org.apache.accumulo.core.data.Mutation) ZooReaderWriter(org.apache.accumulo.server.zookeeper.ZooReaderWriter) Writer(org.apache.accumulo.core.client.impl.Writer) IZooReaderWriter(org.apache.accumulo.fate.zookeeper.IZooReaderWriter)

Aggregations

Writer (org.apache.accumulo.core.client.impl.Writer)7 Credentials (org.apache.accumulo.core.client.impl.Credentials)4 IZooReaderWriter (org.apache.accumulo.fate.zookeeper.IZooReaderWriter)4 ZooReaderWriter (org.apache.accumulo.server.zookeeper.ZooReaderWriter)4 BatchWriter (org.apache.accumulo.core.client.BatchWriter)3 Mutation (org.apache.accumulo.core.data.Mutation)3 ArrayList (java.util.ArrayList)2 ClientContext (org.apache.accumulo.core.client.impl.ClientContext)2 KeyExtent (org.apache.accumulo.core.data.impl.KeyExtent)2 Test (org.junit.Test)2 TreeMap (java.util.TreeMap)1 UUID (java.util.UUID)1 AccumuloException (org.apache.accumulo.core.client.AccumuloException)1 AccumuloSecurityException (org.apache.accumulo.core.client.AccumuloSecurityException)1 Connector (org.apache.accumulo.core.client.Connector)1 PasswordToken (org.apache.accumulo.core.client.security.tokens.PasswordToken)1 ColumnUpdate (org.apache.accumulo.core.data.ColumnUpdate)1 Value (org.apache.accumulo.core.data.Value)1 DataFileValue (org.apache.accumulo.core.metadata.schema.DataFileValue)1 ConstraintViolationException (org.apache.accumulo.core.tabletserver.thrift.ConstraintViolationException)1