Search in sources :

Example 86 with ServerContext

use of org.apache.accumulo.server.ServerContext in project accumulo by apache.

the class MergeStateIT method test.

@Test
public void test() throws Exception {
    ServerContext context = getServerContext();
    try (AccumuloClient accumuloClient = Accumulo.newClient().from(getClientProperties()).build()) {
        accumuloClient.securityOperations().grantTablePermission(accumuloClient.whoami(), MetadataTable.NAME, TablePermission.WRITE);
        BatchWriter bw = accumuloClient.createBatchWriter(MetadataTable.NAME);
        // Create a fake METADATA table with these splits
        String[] splits = { "a", "e", "j", "o", "t", "z" };
        // create metadata for a table "t" with the splits above
        TableId tableId = TableId.of("t");
        Text pr = null;
        for (String s : splits) {
            Text split = new Text(s);
            Mutation prevRow = TabletColumnFamily.createPrevRowMutation(new KeyExtent(tableId, split, pr));
            prevRow.put(CurrentLocationColumnFamily.NAME, new Text("123456"), new Value("127.0.0.1:1234"));
            ChoppedColumnFamily.CHOPPED_COLUMN.put(prevRow, new Value("junk"));
            bw.addMutation(prevRow);
            pr = split;
        }
        // Add the default tablet
        Mutation defaultTablet = TabletColumnFamily.createPrevRowMutation(new KeyExtent(tableId, null, pr));
        defaultTablet.put(CurrentLocationColumnFamily.NAME, new Text("123456"), new Value("127.0.0.1:1234"));
        bw.addMutation(defaultTablet);
        bw.close();
        // Read out the TabletLocationStates
        MockCurrentState state = new MockCurrentState(new MergeInfo(new KeyExtent(tableId, new Text("p"), new Text("e")), MergeInfo.Operation.MERGE));
        // Verify the tablet state: hosted, and count
        TabletStateStore metaDataStateStore = TabletStateStore.getStoreForLevel(DataLevel.USER, context, state);
        int count = 0;
        for (TabletLocationState tss : metaDataStateStore) {
            if (tss != null)
                count++;
        }
        // the normal case is to skip tablets in a good state
        assertEquals(0, count);
        // Create the hole
        // Split the tablet at one end of the range
        Mutation m = TabletColumnFamily.createPrevRowMutation(new KeyExtent(tableId, new Text("t"), new Text("p")));
        TabletColumnFamily.SPLIT_RATIO_COLUMN.put(m, new Value("0.5"));
        TabletColumnFamily.OLD_PREV_ROW_COLUMN.put(m, TabletColumnFamily.encodePrevEndRow(new Text("o")));
        update(accumuloClient, m);
        // do the state check
        MergeStats stats = scan(state, metaDataStateStore);
        MergeState newState = stats.nextMergeState(accumuloClient, state);
        assertEquals(MergeState.WAITING_FOR_OFFLINE, newState);
        // unassign the tablets
        try (BatchDeleter deleter = accumuloClient.createBatchDeleter(MetadataTable.NAME, Authorizations.EMPTY, 1000)) {
            deleter.fetchColumnFamily(CurrentLocationColumnFamily.NAME);
            deleter.setRanges(Collections.singletonList(new Range()));
            deleter.delete();
        }
        // now we should be ready to merge but, we have inconsistent metadata
        stats = scan(state, metaDataStateStore);
        assertEquals(MergeState.WAITING_FOR_OFFLINE, stats.nextMergeState(accumuloClient, state));
        // finish the split
        KeyExtent tablet = new KeyExtent(tableId, new Text("p"), new Text("o"));
        m = TabletColumnFamily.createPrevRowMutation(tablet);
        TabletColumnFamily.SPLIT_RATIO_COLUMN.put(m, new Value("0.5"));
        update(accumuloClient, m);
        metaDataStateStore.setLocations(Collections.singletonList(new Assignment(tablet, state.someTServer)));
        // onos... there's a new tablet online
        stats = scan(state, metaDataStateStore);
        assertEquals(MergeState.WAITING_FOR_CHOPPED, stats.nextMergeState(accumuloClient, state));
        // chop it
        m = TabletColumnFamily.createPrevRowMutation(tablet);
        ChoppedColumnFamily.CHOPPED_COLUMN.put(m, new Value("junk"));
        update(accumuloClient, m);
        stats = scan(state, metaDataStateStore);
        assertEquals(MergeState.WAITING_FOR_OFFLINE, stats.nextMergeState(accumuloClient, state));
        // take it offline
        m = TabletColumnFamily.createPrevRowMutation(tablet);
        Collection<Collection<String>> walogs = Collections.emptyList();
        metaDataStateStore.unassign(Collections.singletonList(new TabletLocationState(tablet, null, state.someTServer, null, null, walogs, false)), null);
        // now we can split
        stats = scan(state, metaDataStateStore);
        assertEquals(MergeState.MERGING, stats.nextMergeState(accumuloClient, state));
    }
}
Also used : AccumuloClient(org.apache.accumulo.core.client.AccumuloClient) TableId(org.apache.accumulo.core.data.TableId) MergeInfo(org.apache.accumulo.server.manager.state.MergeInfo) BatchDeleter(org.apache.accumulo.core.client.BatchDeleter) MergeState(org.apache.accumulo.server.manager.state.MergeState) Text(org.apache.hadoop.io.Text) TabletStateStore(org.apache.accumulo.server.manager.state.TabletStateStore) Range(org.apache.accumulo.core.data.Range) KeyExtent(org.apache.accumulo.core.dataImpl.KeyExtent) Assignment(org.apache.accumulo.server.manager.state.Assignment) ServerContext(org.apache.accumulo.server.ServerContext) Value(org.apache.accumulo.core.data.Value) MergeStats(org.apache.accumulo.manager.state.MergeStats) TabletLocationState(org.apache.accumulo.core.metadata.TabletLocationState) Collection(java.util.Collection) BatchWriter(org.apache.accumulo.core.client.BatchWriter) Mutation(org.apache.accumulo.core.data.Mutation) Test(org.junit.Test)

Example 87 with ServerContext

use of org.apache.accumulo.server.ServerContext in project accumulo by apache.

the class ReplicationProcessorTest method noPeerConfigurationThrowsAnException.

@Test
public void noPeerConfigurationThrowsAnException() {
    ServerContext context = createMock(ServerContext.class);
    var conf = new ConfigurationCopy(Map.of());
    expect(context.getConfiguration()).andReturn(conf);
    replay(context);
    ReplicationProcessor proc = new ReplicationProcessor(context);
    assertThrows(IllegalArgumentException.class, () -> proc.getPeerType("foo"));
    verify(context);
}
Also used : ConfigurationCopy(org.apache.accumulo.core.conf.ConfigurationCopy) ServerContext(org.apache.accumulo.server.ServerContext) Test(org.junit.Test)

Aggregations

ServerContext (org.apache.accumulo.server.ServerContext)87 Test (org.junit.Test)41 ZooReaderWriter (org.apache.accumulo.fate.zookeeper.ZooReaderWriter)18 AccumuloClient (org.apache.accumulo.core.client.AccumuloClient)15 AccumuloConfiguration (org.apache.accumulo.core.conf.AccumuloConfiguration)15 TServerInstance (org.apache.accumulo.core.metadata.TServerInstance)15 HostAndPort (org.apache.accumulo.core.util.HostAndPort)15 Path (org.apache.hadoop.fs.Path)15 ArrayList (java.util.ArrayList)14 KeyExtent (org.apache.accumulo.core.dataImpl.KeyExtent)14 VolumeManager (org.apache.accumulo.server.fs.VolumeManager)13 KeeperException (org.apache.zookeeper.KeeperException)13 ServerAddress (org.apache.accumulo.server.rpc.ServerAddress)12 TableId (org.apache.accumulo.core.data.TableId)11 LiveTServerSet (org.apache.accumulo.server.manager.LiveTServerSet)11 Value (org.apache.accumulo.core.data.Value)10 IOException (java.io.IOException)9 UUID (java.util.UUID)9 ConfigurationCopy (org.apache.accumulo.core.conf.ConfigurationCopy)9 Client (org.apache.accumulo.core.tabletserver.thrift.TabletClientService.Client)9