Search in sources :

Example 6 with MockInstance

use of org.apache.accumulo.core.client.mock.MockInstance in project hive by apache.

the class TestAccumuloStorageHandler method testNonExternalExistentTable.

@Test(expected = MetaException.class)
public void testNonExternalExistentTable() throws Exception {
    MockInstance inst = new MockInstance(test.getMethodName());
    Connector conn = inst.getConnector("root", new PasswordToken(""));
    String tableName = "table";
    // Create the table
    conn.tableOperations().create(tableName);
    // Define the SerDe Parameters
    Map<String, String> params = new HashMap<String, String>();
    params.put(AccumuloSerDeParameters.COLUMN_MAPPINGS, "cf:cq");
    AccumuloConnectionParameters connectionParams = Mockito.mock(AccumuloConnectionParameters.class);
    AccumuloStorageHandler storageHandler = Mockito.mock(AccumuloStorageHandler.class);
    StorageDescriptor sd = Mockito.mock(StorageDescriptor.class);
    Table table = Mockito.mock(Table.class);
    SerDeInfo serDeInfo = Mockito.mock(SerDeInfo.class);
    // Call the real preCreateTable method
    Mockito.doCallRealMethod().when(storageHandler).preCreateTable(table);
    // Return our known table name
    Mockito.when(storageHandler.getTableName(table)).thenReturn(tableName);
    // Is not an EXTERNAL table
    Mockito.when(storageHandler.isExternalTable(table)).thenReturn(false);
    // Return the mocked StorageDescriptor
    Mockito.when(table.getSd()).thenReturn(sd);
    // No location expected with AccumuloStorageHandler
    Mockito.when(sd.getLocation()).thenReturn(null);
    // Return mocked SerDeInfo
    Mockito.when(sd.getSerdeInfo()).thenReturn(serDeInfo);
    // Custom parameters
    Mockito.when(serDeInfo.getParameters()).thenReturn(params);
    // Return the MockInstance's Connector
    Mockito.when(connectionParams.getConnector()).thenReturn(conn);
    storageHandler.connectionParams = connectionParams;
    storageHandler.preCreateTable(table);
}
Also used : Connector(org.apache.accumulo.core.client.Connector) PasswordToken(org.apache.accumulo.core.client.security.tokens.PasswordToken) Table(org.apache.hadoop.hive.metastore.api.Table) MockInstance(org.apache.accumulo.core.client.mock.MockInstance) HashMap(java.util.HashMap) SerDeInfo(org.apache.hadoop.hive.metastore.api.SerDeInfo) StorageDescriptor(org.apache.hadoop.hive.metastore.api.StorageDescriptor) Test(org.junit.Test)

Example 7 with MockInstance

use of org.apache.accumulo.core.client.mock.MockInstance in project hive by apache.

the class TestAccumuloStorageHandler method testRollbackCreateTableOnNonExistentTable.

@Test()
public void testRollbackCreateTableOnNonExistentTable() throws Exception {
    MockInstance inst = new MockInstance(test.getMethodName());
    Connector conn = inst.getConnector("root", new PasswordToken(""));
    AccumuloStorageHandler storageHandler = Mockito.mock(AccumuloStorageHandler.class);
    String tableName = "table";
    AccumuloConnectionParameters connectionParams = Mockito.mock(AccumuloConnectionParameters.class);
    Table table = Mockito.mock(Table.class);
    // Call the real preCreateTable method
    Mockito.doCallRealMethod().when(storageHandler).rollbackCreateTable(table);
    // Return our known table name
    Mockito.when(storageHandler.getTableName(table)).thenReturn(tableName);
    // Is not an EXTERNAL table
    Mockito.when(storageHandler.isExternalTable(table)).thenReturn(false);
    // Return the MockInstance's Connector
    Mockito.when(connectionParams.getConnector()).thenReturn(conn);
    storageHandler.connectionParams = connectionParams;
    storageHandler.rollbackCreateTable(table);
}
Also used : Connector(org.apache.accumulo.core.client.Connector) PasswordToken(org.apache.accumulo.core.client.security.tokens.PasswordToken) Table(org.apache.hadoop.hive.metastore.api.Table) MockInstance(org.apache.accumulo.core.client.mock.MockInstance) Test(org.junit.Test)

Example 8 with MockInstance

use of org.apache.accumulo.core.client.mock.MockInstance in project hive by apache.

the class TestAccumuloStorageHandler method testRollbackCreateTableDoesntDeleteExternalExistentTable.

@Test()
public void testRollbackCreateTableDoesntDeleteExternalExistentTable() throws Exception {
    MockInstance inst = new MockInstance(test.getMethodName());
    Connector conn = inst.getConnector("root", new PasswordToken(""));
    AccumuloStorageHandler storageHandler = Mockito.mock(AccumuloStorageHandler.class);
    String tableName = "table";
    // Create the table
    conn.tableOperations().create(tableName);
    AccumuloConnectionParameters connectionParams = Mockito.mock(AccumuloConnectionParameters.class);
    Table table = Mockito.mock(Table.class);
    // Call the real preCreateTable method
    Mockito.doCallRealMethod().when(storageHandler).rollbackCreateTable(table);
    Mockito.doCallRealMethod().when(storageHandler).commitDropTable(table, true);
    // Return our known table name
    Mockito.when(storageHandler.getTableName(table)).thenReturn(tableName);
    // Is not an EXTERNAL table
    Mockito.when(storageHandler.isExternalTable(table)).thenReturn(true);
    // Return the MockInstance's Connector
    Mockito.when(connectionParams.getConnector()).thenReturn(conn);
    storageHandler.connectionParams = connectionParams;
    storageHandler.rollbackCreateTable(table);
    Assert.assertTrue(conn.tableOperations().exists(tableName));
}
Also used : Connector(org.apache.accumulo.core.client.Connector) PasswordToken(org.apache.accumulo.core.client.security.tokens.PasswordToken) Table(org.apache.hadoop.hive.metastore.api.Table) MockInstance(org.apache.accumulo.core.client.mock.MockInstance) Test(org.junit.Test)

Example 9 with MockInstance

use of org.apache.accumulo.core.client.mock.MockInstance in project presto by prestodb.

the class TestIndexer method testMutationIndexWithVisibilities.

@Test
public void testMutationIndexWithVisibilities() throws Exception {
    Instance inst = new MockInstance();
    Connector conn = inst.getConnector("root", new PasswordToken(""));
    conn.tableOperations().create(table.getFullTableName());
    conn.tableOperations().create(table.getIndexTableName());
    conn.tableOperations().create(table.getMetricsTableName());
    for (IteratorSetting s : Indexer.getMetricIterators(table)) {
        conn.tableOperations().attachIterator(table.getMetricsTableName(), s);
    }
    Indexer indexer = new Indexer(conn, new Authorizations(), table, new BatchWriterConfig());
    indexer.index(m1);
    indexer.index(m1v);
    indexer.flush();
    Scanner scan = conn.createScanner(table.getIndexTableName(), new Authorizations("private", "moreprivate"));
    scan.setRange(new Range());
    Iterator<Entry<Key, Value>> iter = scan.iterator();
    assertKeyValuePair(iter.next(), AGE_VALUE, "cf_age", "row1", "");
    assertKeyValuePair(iter.next(), AGE_VALUE, "cf_age", "row1", "private", "");
    assertKeyValuePair(iter.next(), bytes("abc"), "cf_arr", "row1", "");
    assertKeyValuePair(iter.next(), bytes("abc"), "cf_arr", "row1", "moreprivate", "");
    assertKeyValuePair(iter.next(), M1_FNAME_VALUE, "cf_firstname", "row1", "");
    assertKeyValuePair(iter.next(), M1_FNAME_VALUE, "cf_firstname", "row1", "private", "");
    assertKeyValuePair(iter.next(), bytes("def"), "cf_arr", "row1", "");
    assertKeyValuePair(iter.next(), bytes("def"), "cf_arr", "row1", "moreprivate", "");
    assertKeyValuePair(iter.next(), bytes("ghi"), "cf_arr", "row1", "");
    assertKeyValuePair(iter.next(), bytes("ghi"), "cf_arr", "row1", "moreprivate", "");
    assertFalse(iter.hasNext());
    scan.close();
    scan = conn.createScanner(table.getMetricsTableName(), new Authorizations("private", "moreprivate"));
    scan.setRange(new Range());
    iter = scan.iterator();
    assertKeyValuePair(iter.next(), AGE_VALUE, "cf_age", "___card___", "1");
    assertKeyValuePair(iter.next(), AGE_VALUE, "cf_age", "___card___", "private", "1");
    assertKeyValuePair(iter.next(), Indexer.METRICS_TABLE_ROW_ID.array(), "___rows___", "___card___", "2");
    assertKeyValuePair(iter.next(), Indexer.METRICS_TABLE_ROW_ID.array(), "___rows___", "___first_row___", "row1");
    assertKeyValuePair(iter.next(), Indexer.METRICS_TABLE_ROW_ID.array(), "___rows___", "___last_row___", "row1");
    assertKeyValuePair(iter.next(), bytes("abc"), "cf_arr", "___card___", "1");
    assertKeyValuePair(iter.next(), bytes("abc"), "cf_arr", "___card___", "moreprivate", "1");
    assertKeyValuePair(iter.next(), M1_FNAME_VALUE, "cf_firstname", "___card___", "1");
    assertKeyValuePair(iter.next(), M1_FNAME_VALUE, "cf_firstname", "___card___", "private", "1");
    assertKeyValuePair(iter.next(), bytes("def"), "cf_arr", "___card___", "1");
    assertKeyValuePair(iter.next(), bytes("def"), "cf_arr", "___card___", "moreprivate", "1");
    assertKeyValuePair(iter.next(), bytes("ghi"), "cf_arr", "___card___", "1");
    assertKeyValuePair(iter.next(), bytes("ghi"), "cf_arr", "___card___", "moreprivate", "1");
    assertFalse(iter.hasNext());
    scan.close();
    indexer.index(m2);
    indexer.index(m2v);
    indexer.close();
    scan = conn.createScanner(table.getIndexTableName(), new Authorizations("private", "moreprivate"));
    scan.setRange(new Range());
    iter = scan.iterator();
    assertKeyValuePair(iter.next(), AGE_VALUE, "cf_age", "row1", "");
    assertKeyValuePair(iter.next(), AGE_VALUE, "cf_age", "row1", "private", "");
    assertKeyValuePair(iter.next(), AGE_VALUE, "cf_age", "row2", "");
    assertKeyValuePair(iter.next(), AGE_VALUE, "cf_age", "row2", "private", "");
    assertKeyValuePair(iter.next(), bytes("abc"), "cf_arr", "row1", "");
    assertKeyValuePair(iter.next(), bytes("abc"), "cf_arr", "row1", "moreprivate", "");
    assertKeyValuePair(iter.next(), bytes("abc"), "cf_arr", "row2", "");
    assertKeyValuePair(iter.next(), bytes("abc"), "cf_arr", "row2", "moreprivate", "");
    assertKeyValuePair(iter.next(), M1_FNAME_VALUE, "cf_firstname", "row1", "");
    assertKeyValuePair(iter.next(), M1_FNAME_VALUE, "cf_firstname", "row1", "private", "");
    assertKeyValuePair(iter.next(), M2_FNAME_VALUE, "cf_firstname", "row2", "");
    assertKeyValuePair(iter.next(), M2_FNAME_VALUE, "cf_firstname", "row2", "moreprivate", "");
    assertKeyValuePair(iter.next(), bytes("def"), "cf_arr", "row1", "");
    assertKeyValuePair(iter.next(), bytes("def"), "cf_arr", "row1", "moreprivate", "");
    assertKeyValuePair(iter.next(), bytes("ghi"), "cf_arr", "row1", "");
    assertKeyValuePair(iter.next(), bytes("ghi"), "cf_arr", "row1", "moreprivate", "");
    assertKeyValuePair(iter.next(), bytes("ghi"), "cf_arr", "row2", "");
    assertKeyValuePair(iter.next(), bytes("ghi"), "cf_arr", "row2", "moreprivate", "");
    assertKeyValuePair(iter.next(), bytes("mno"), "cf_arr", "row2", "");
    assertKeyValuePair(iter.next(), bytes("mno"), "cf_arr", "row2", "moreprivate", "");
    assertFalse(iter.hasNext());
    scan.close();
    scan = conn.createScanner(table.getMetricsTableName(), new Authorizations("private", "moreprivate"));
    scan.setRange(new Range());
    iter = scan.iterator();
    assertKeyValuePair(iter.next(), AGE_VALUE, "cf_age", "___card___", "2");
    assertKeyValuePair(iter.next(), AGE_VALUE, "cf_age", "___card___", "private", "2");
    assertKeyValuePair(iter.next(), Indexer.METRICS_TABLE_ROW_ID.array(), "___rows___", "___card___", "4");
    assertKeyValuePair(iter.next(), Indexer.METRICS_TABLE_ROW_ID.array(), "___rows___", "___first_row___", "row1");
    assertKeyValuePair(iter.next(), Indexer.METRICS_TABLE_ROW_ID.array(), "___rows___", "___last_row___", "row2");
    assertKeyValuePair(iter.next(), bytes("abc"), "cf_arr", "___card___", "2");
    assertKeyValuePair(iter.next(), bytes("abc"), "cf_arr", "___card___", "moreprivate", "2");
    assertKeyValuePair(iter.next(), M1_FNAME_VALUE, "cf_firstname", "___card___", "1");
    assertKeyValuePair(iter.next(), M1_FNAME_VALUE, "cf_firstname", "___card___", "private", "1");
    assertKeyValuePair(iter.next(), M2_FNAME_VALUE, "cf_firstname", "___card___", "1");
    assertKeyValuePair(iter.next(), M2_FNAME_VALUE, "cf_firstname", "___card___", "moreprivate", "1");
    assertKeyValuePair(iter.next(), bytes("def"), "cf_arr", "___card___", "1");
    assertKeyValuePair(iter.next(), bytes("def"), "cf_arr", "___card___", "moreprivate", "1");
    assertKeyValuePair(iter.next(), bytes("ghi"), "cf_arr", "___card___", "2");
    assertKeyValuePair(iter.next(), bytes("ghi"), "cf_arr", "___card___", "moreprivate", "2");
    assertKeyValuePair(iter.next(), bytes("mno"), "cf_arr", "___card___", "1");
    assertKeyValuePair(iter.next(), bytes("mno"), "cf_arr", "___card___", "moreprivate", "1");
    assertFalse(iter.hasNext());
    scan.close();
}
Also used : Connector(org.apache.accumulo.core.client.Connector) Scanner(org.apache.accumulo.core.client.Scanner) PasswordToken(org.apache.accumulo.core.client.security.tokens.PasswordToken) Authorizations(org.apache.accumulo.core.security.Authorizations) Entry(java.util.Map.Entry) IteratorSetting(org.apache.accumulo.core.client.IteratorSetting) MockInstance(org.apache.accumulo.core.client.mock.MockInstance) Instance(org.apache.accumulo.core.client.Instance) MockInstance(org.apache.accumulo.core.client.mock.MockInstance) BatchWriterConfig(org.apache.accumulo.core.client.BatchWriterConfig) Range(org.apache.accumulo.core.data.Range) Test(org.testng.annotations.Test)

Example 10 with MockInstance

use of org.apache.accumulo.core.client.mock.MockInstance in project Gaffer by gchq.

the class MockAccumuloStore method initialise.

public void initialise(final Schema schema, final StoreProperties properties) throws StoreException {
    if (!(properties instanceof AccumuloProperties)) {
        throw new StoreException("Store must be initialised with AccumuloProperties");
    }
    mockAccumulo = new MockInstance(((AccumuloProperties) properties).getInstance());
    super.initialise(schema, properties);
}
Also used : MockInstance(org.apache.accumulo.core.client.mock.MockInstance) StoreException(uk.gov.gchq.gaffer.store.StoreException)

Aggregations

MockInstance (org.apache.accumulo.core.client.mock.MockInstance)21 PasswordToken (org.apache.accumulo.core.client.security.tokens.PasswordToken)20 Connector (org.apache.accumulo.core.client.Connector)18 Test (org.junit.Test)15 Authorizations (org.apache.accumulo.core.security.Authorizations)10 Table (org.apache.hadoop.hive.metastore.api.Table)9 Mutation (org.apache.accumulo.core.data.Mutation)7 Value (org.apache.accumulo.core.data.Value)7 Text (org.apache.hadoop.io.Text)7 JobConf (org.apache.hadoop.mapred.JobConf)7 Entry (java.util.Map.Entry)6 Instance (org.apache.accumulo.core.client.Instance)6 Key (org.apache.accumulo.core.data.Key)6 ByteArrayRef (org.apache.hadoop.hive.serde2.lazy.ByteArrayRef)6 HashMap (java.util.HashMap)5 BatchWriterConfig (org.apache.accumulo.core.client.BatchWriterConfig)5 Properties (java.util.Properties)4 Configuration (org.apache.hadoop.conf.Configuration)4 FileSystem (org.apache.hadoop.fs.FileSystem)4 AccumuloRowSerializer (org.apache.hadoop.hive.accumulo.serde.AccumuloRowSerializer)4