Search in sources :

Example 1 with AbstractSampleElementsForSplitPointsHandler

use of uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandler in project Gaffer by gchq.

the class SampleElementsForSplitPointsHandlerTest method shouldCalculateRequiredNumberOfSplitsFromEdges.

@Test
public void shouldCalculateRequiredNumberOfSplitsFromEdges() throws OperationException {
    // Given
    final int numSplits = 3;
    final List<Element> elements = IntStream.range(0, numSplits * 10).mapToObj(i -> new Edge(TestGroups.EDGE, "source_" + i, "dest_" + i, true)).collect(Collectors.toList());
    final AbstractSampleElementsForSplitPointsHandler<String, AccumuloStore> handler = createHandler();
    final SampleElementsForSplitPoints<String> operation = new SampleElementsForSplitPoints.Builder<String>().input(elements).numSplits(numSplits).build();
    // When
    createHandler().doOperation(operation, new Context(), store);
    // Then
    final ArgumentCaptor<GenerateSplitPointsFromSample> generateSplitPointsFromSampleCaptor = ArgumentCaptor.forClass(GenerateSplitPointsFromSample.class);
    verify(store).execute(generateSplitPointsFromSampleCaptor.capture(), any(Context.class));
    final int expectedElementCount = elements.size() * 2;
    assertExpectedNumberOfSplitPointsAndSampleSize(generateSplitPointsFromSampleCaptor, numSplits, expectedElementCount);
}
Also used : ByteEntityAccumuloElementConverter(uk.gov.gchq.gaffer.accumulostore.key.core.impl.byteEntity.ByteEntityAccumuloElementConverter) IntStream(java.util.stream.IntStream) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) BeforeEach(org.junit.jupiter.api.BeforeEach) AccumuloElementConverter(uk.gov.gchq.gaffer.accumulostore.key.AccumuloElementConverter) StoreException(uk.gov.gchq.gaffer.store.StoreException) AccumuloKeyPackage(uk.gov.gchq.gaffer.accumulostore.key.AccumuloKeyPackage) Element(uk.gov.gchq.gaffer.data.element.Element) AbstractSampleElementsForSplitPointsHandlerTest(uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandlerTest) ArgumentCaptor(org.mockito.ArgumentCaptor) BDDMockito.given(org.mockito.BDDMockito.given) AccumuloStore(uk.gov.gchq.gaffer.accumulostore.AccumuloStore) Edge(uk.gov.gchq.gaffer.data.element.Edge) TestGroups(uk.gov.gchq.gaffer.commonutil.TestGroups) SampleElementsForSplitPoints(uk.gov.gchq.gaffer.operation.impl.SampleElementsForSplitPoints) GenerateSplitPointsFromSample(uk.gov.gchq.gaffer.operation.impl.GenerateSplitPointsFromSample) AbstractSampleElementsForSplitPointsHandler(uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandler) Collectors(java.util.stream.Collectors) Mockito.verify(org.mockito.Mockito.verify) Test(org.junit.jupiter.api.Test) List(java.util.List) Context(uk.gov.gchq.gaffer.store.Context) OperationException(uk.gov.gchq.gaffer.operation.OperationException) Collections(java.util.Collections) Mockito.mock(org.mockito.Mockito.mock) Context(uk.gov.gchq.gaffer.store.Context) GenerateSplitPointsFromSample(uk.gov.gchq.gaffer.operation.impl.GenerateSplitPointsFromSample) Element(uk.gov.gchq.gaffer.data.element.Element) SampleElementsForSplitPoints(uk.gov.gchq.gaffer.operation.impl.SampleElementsForSplitPoints) AccumuloStore(uk.gov.gchq.gaffer.accumulostore.AccumuloStore) Edge(uk.gov.gchq.gaffer.data.element.Edge) AbstractSampleElementsForSplitPointsHandlerTest(uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandlerTest) Test(org.junit.jupiter.api.Test)

Example 2 with AbstractSampleElementsForSplitPointsHandler

use of uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandler in project Gaffer by gchq.

the class SampleElementsForSplitPointsHandlerTest method shouldUseTheNumberOfTabletServersToCalculateNumSplits.

@Test
public void shouldUseTheNumberOfTabletServersToCalculateNumSplits() throws OperationException {
    // Given
    final Integer numSplits = null;
    final List<Element> elements = IntStream.range(0, 30).mapToObj(i -> new Edge(TestGroups.EDGE, "source_" + i, "dest_" + i, true)).collect(Collectors.toList());
    final AbstractSampleElementsForSplitPointsHandler<String, AccumuloStore> handler = createHandler();
    final SampleElementsForSplitPoints<String> operation = new SampleElementsForSplitPoints.Builder<String>().input(elements).numSplits(numSplits).build();
    // When
    createHandler().doOperation(operation, new Context(), store);
    // Then
    final ArgumentCaptor<GenerateSplitPointsFromSample> generateSplitPointsFromSampleCaptor = ArgumentCaptor.forClass(GenerateSplitPointsFromSample.class);
    verify(store).execute(generateSplitPointsFromSampleCaptor.capture(), any(Context.class));
    final int expectedNumOfSplits = NUM_TABLET_SERVERS - 1;
    final int expectedElementCount = elements.size() * 2;
    assertExpectedNumberOfSplitPointsAndSampleSize(generateSplitPointsFromSampleCaptor, expectedNumOfSplits, expectedElementCount);
}
Also used : ByteEntityAccumuloElementConverter(uk.gov.gchq.gaffer.accumulostore.key.core.impl.byteEntity.ByteEntityAccumuloElementConverter) IntStream(java.util.stream.IntStream) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) BeforeEach(org.junit.jupiter.api.BeforeEach) AccumuloElementConverter(uk.gov.gchq.gaffer.accumulostore.key.AccumuloElementConverter) StoreException(uk.gov.gchq.gaffer.store.StoreException) AccumuloKeyPackage(uk.gov.gchq.gaffer.accumulostore.key.AccumuloKeyPackage) Element(uk.gov.gchq.gaffer.data.element.Element) AbstractSampleElementsForSplitPointsHandlerTest(uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandlerTest) ArgumentCaptor(org.mockito.ArgumentCaptor) BDDMockito.given(org.mockito.BDDMockito.given) AccumuloStore(uk.gov.gchq.gaffer.accumulostore.AccumuloStore) Edge(uk.gov.gchq.gaffer.data.element.Edge) TestGroups(uk.gov.gchq.gaffer.commonutil.TestGroups) SampleElementsForSplitPoints(uk.gov.gchq.gaffer.operation.impl.SampleElementsForSplitPoints) GenerateSplitPointsFromSample(uk.gov.gchq.gaffer.operation.impl.GenerateSplitPointsFromSample) AbstractSampleElementsForSplitPointsHandler(uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandler) Collectors(java.util.stream.Collectors) Mockito.verify(org.mockito.Mockito.verify) Test(org.junit.jupiter.api.Test) List(java.util.List) Context(uk.gov.gchq.gaffer.store.Context) OperationException(uk.gov.gchq.gaffer.operation.OperationException) Collections(java.util.Collections) Mockito.mock(org.mockito.Mockito.mock) Context(uk.gov.gchq.gaffer.store.Context) GenerateSplitPointsFromSample(uk.gov.gchq.gaffer.operation.impl.GenerateSplitPointsFromSample) Element(uk.gov.gchq.gaffer.data.element.Element) SampleElementsForSplitPoints(uk.gov.gchq.gaffer.operation.impl.SampleElementsForSplitPoints) AccumuloStore(uk.gov.gchq.gaffer.accumulostore.AccumuloStore) Edge(uk.gov.gchq.gaffer.data.element.Edge) AbstractSampleElementsForSplitPointsHandlerTest(uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandlerTest) Test(org.junit.jupiter.api.Test)

Example 3 with AbstractSampleElementsForSplitPointsHandler

use of uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandler in project Gaffer by gchq.

the class SampleElementsForSplitPointsHandlerTest method shouldUseTheNumberOfRegionsToCalculateNumSplits.

@Test
public void shouldUseTheNumberOfRegionsToCalculateNumSplits() throws OperationException {
    // Given
    final Integer numSplits = null;
    final List<Element> elements = IntStream.range(0, 30).mapToObj(i -> new Edge(TestGroups.EDGE, "source_" + i, "dest_" + i, true)).collect(Collectors.toList());
    final AbstractSampleElementsForSplitPointsHandler<String, HBaseStore> handler = createHandler();
    final SampleElementsForSplitPoints<String> operation = new SampleElementsForSplitPoints.Builder<String>().input(elements).numSplits(numSplits).build();
    // When
    createHandler().doOperation(operation, new Context(), store);
    // Then
    final ArgumentCaptor<GenerateSplitPointsFromSample> generateSplitPointsFromSampleCaptor = ArgumentCaptor.forClass(GenerateSplitPointsFromSample.class);
    verify(store).execute(generateSplitPointsFromSampleCaptor.capture(), any(Context.class));
    final int expectedNumOfSplits = NUM_TABLE_REGIONS - 1;
    final int expectedElementCount = elements.size() * 2;
    assertExpectedNumberOfSplitPointsAndSampleSize(generateSplitPointsFromSampleCaptor, expectedNumOfSplits, expectedElementCount);
}
Also used : IntStream(java.util.stream.IntStream) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) BeforeEach(org.junit.jupiter.api.BeforeEach) StoreException(uk.gov.gchq.gaffer.store.StoreException) Element(uk.gov.gchq.gaffer.data.element.Element) AbstractSampleElementsForSplitPointsHandlerTest(uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandlerTest) ArgumentCaptor(org.mockito.ArgumentCaptor) BDDMockito.given(org.mockito.BDDMockito.given) Edge(uk.gov.gchq.gaffer.data.element.Edge) TestGroups(uk.gov.gchq.gaffer.commonutil.TestGroups) SampleElementsForSplitPoints(uk.gov.gchq.gaffer.operation.impl.SampleElementsForSplitPoints) TableName(org.apache.hadoop.hbase.TableName) HBaseStore(uk.gov.gchq.gaffer.hbasestore.HBaseStore) GenerateSplitPointsFromSample(uk.gov.gchq.gaffer.operation.impl.GenerateSplitPointsFromSample) AbstractSampleElementsForSplitPointsHandler(uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandler) IOException(java.io.IOException) Collectors(java.util.stream.Collectors) HRegionInfo(org.apache.hadoop.hbase.HRegionInfo) Mockito.verify(org.mockito.Mockito.verify) Test(org.junit.jupiter.api.Test) List(java.util.List) Admin(org.apache.hadoop.hbase.client.Admin) Context(uk.gov.gchq.gaffer.store.Context) Connection(org.apache.hadoop.hbase.client.Connection) OperationException(uk.gov.gchq.gaffer.operation.OperationException) Collections(java.util.Collections) Mockito.mock(org.mockito.Mockito.mock) Context(uk.gov.gchq.gaffer.store.Context) HBaseStore(uk.gov.gchq.gaffer.hbasestore.HBaseStore) GenerateSplitPointsFromSample(uk.gov.gchq.gaffer.operation.impl.GenerateSplitPointsFromSample) Element(uk.gov.gchq.gaffer.data.element.Element) SampleElementsForSplitPoints(uk.gov.gchq.gaffer.operation.impl.SampleElementsForSplitPoints) Edge(uk.gov.gchq.gaffer.data.element.Edge) AbstractSampleElementsForSplitPointsHandlerTest(uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandlerTest) Test(org.junit.jupiter.api.Test)

Example 4 with AbstractSampleElementsForSplitPointsHandler

use of uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandler in project Gaffer by gchq.

the class SampleElementsForSplitPointsHandlerTest method shouldCalculateRequiredNumberOfSplitsFromEdges.

@Test
public void shouldCalculateRequiredNumberOfSplitsFromEdges() throws OperationException {
    // Given
    final int numSplits = 3;
    final List<Element> elements = IntStream.range(0, numSplits * 10).mapToObj(i -> new Edge(TestGroups.EDGE, "source_" + i, "dest_" + i, true)).collect(Collectors.toList());
    final AbstractSampleElementsForSplitPointsHandler<String, HBaseStore> handler = createHandler();
    final SampleElementsForSplitPoints<String> operation = new SampleElementsForSplitPoints.Builder<String>().input(elements).numSplits(numSplits).build();
    // When
    createHandler().doOperation(operation, new Context(), createStore());
    // Then
    final ArgumentCaptor<GenerateSplitPointsFromSample> generateSplitPointsFromSampleCaptor = ArgumentCaptor.forClass(GenerateSplitPointsFromSample.class);
    verify(store).execute(generateSplitPointsFromSampleCaptor.capture(), any(Context.class));
    final int expectedElementCount = elements.size() * 2;
    assertExpectedNumberOfSplitPointsAndSampleSize(generateSplitPointsFromSampleCaptor, numSplits, expectedElementCount);
}
Also used : IntStream(java.util.stream.IntStream) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) BeforeEach(org.junit.jupiter.api.BeforeEach) StoreException(uk.gov.gchq.gaffer.store.StoreException) Element(uk.gov.gchq.gaffer.data.element.Element) AbstractSampleElementsForSplitPointsHandlerTest(uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandlerTest) ArgumentCaptor(org.mockito.ArgumentCaptor) BDDMockito.given(org.mockito.BDDMockito.given) Edge(uk.gov.gchq.gaffer.data.element.Edge) TestGroups(uk.gov.gchq.gaffer.commonutil.TestGroups) SampleElementsForSplitPoints(uk.gov.gchq.gaffer.operation.impl.SampleElementsForSplitPoints) TableName(org.apache.hadoop.hbase.TableName) HBaseStore(uk.gov.gchq.gaffer.hbasestore.HBaseStore) GenerateSplitPointsFromSample(uk.gov.gchq.gaffer.operation.impl.GenerateSplitPointsFromSample) AbstractSampleElementsForSplitPointsHandler(uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandler) IOException(java.io.IOException) Collectors(java.util.stream.Collectors) HRegionInfo(org.apache.hadoop.hbase.HRegionInfo) Mockito.verify(org.mockito.Mockito.verify) Test(org.junit.jupiter.api.Test) List(java.util.List) Admin(org.apache.hadoop.hbase.client.Admin) Context(uk.gov.gchq.gaffer.store.Context) Connection(org.apache.hadoop.hbase.client.Connection) OperationException(uk.gov.gchq.gaffer.operation.OperationException) Collections(java.util.Collections) Mockito.mock(org.mockito.Mockito.mock) Context(uk.gov.gchq.gaffer.store.Context) HBaseStore(uk.gov.gchq.gaffer.hbasestore.HBaseStore) GenerateSplitPointsFromSample(uk.gov.gchq.gaffer.operation.impl.GenerateSplitPointsFromSample) Element(uk.gov.gchq.gaffer.data.element.Element) SampleElementsForSplitPoints(uk.gov.gchq.gaffer.operation.impl.SampleElementsForSplitPoints) Edge(uk.gov.gchq.gaffer.data.element.Edge) AbstractSampleElementsForSplitPointsHandlerTest(uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandlerTest) Test(org.junit.jupiter.api.Test)

Aggregations

Collections (java.util.Collections)4 List (java.util.List)4 Collectors (java.util.stream.Collectors)4 IntStream (java.util.stream.IntStream)4 BeforeEach (org.junit.jupiter.api.BeforeEach)4 Test (org.junit.jupiter.api.Test)4 ArgumentCaptor (org.mockito.ArgumentCaptor)4 ArgumentMatchers.any (org.mockito.ArgumentMatchers.any)4 BDDMockito.given (org.mockito.BDDMockito.given)4 Mockito.mock (org.mockito.Mockito.mock)4 Mockito.verify (org.mockito.Mockito.verify)4 TestGroups (uk.gov.gchq.gaffer.commonutil.TestGroups)4 Edge (uk.gov.gchq.gaffer.data.element.Edge)4 Element (uk.gov.gchq.gaffer.data.element.Element)4 OperationException (uk.gov.gchq.gaffer.operation.OperationException)4 GenerateSplitPointsFromSample (uk.gov.gchq.gaffer.operation.impl.GenerateSplitPointsFromSample)4 SampleElementsForSplitPoints (uk.gov.gchq.gaffer.operation.impl.SampleElementsForSplitPoints)4 Context (uk.gov.gchq.gaffer.store.Context)4 StoreException (uk.gov.gchq.gaffer.store.StoreException)4 AbstractSampleElementsForSplitPointsHandler (uk.gov.gchq.gaffer.store.operation.handler.AbstractSampleElementsForSplitPointsHandler)4