Search in sources :

Example 31 with DisplayData

use of org.apache.beam.sdk.transforms.display.DisplayData in project beam by apache.

the class WindowTest method testDisplayDataExcludesDefaults.

@Test
public void testDisplayDataExcludesDefaults() {
    Window<?> window = Window.into(new GlobalWindows()).triggering(DefaultTrigger.of()).withAllowedLateness(Duration.millis(BoundedWindow.TIMESTAMP_MAX_VALUE.getMillis()));
    DisplayData data = DisplayData.from(window);
    assertThat(data, not(hasDisplayItem("trigger")));
    assertThat(data, not(hasDisplayItem("allowedLateness")));
}
Also used : DisplayData(org.apache.beam.sdk.transforms.display.DisplayData) Test(org.junit.Test)

Example 32 with DisplayData

use of org.apache.beam.sdk.transforms.display.DisplayData in project beam by apache.

the class WindowTest method testAssignDisplayDataUnchanged.

@Test
public void testAssignDisplayDataUnchanged() {
    FixedWindows windowFn = FixedWindows.of(Duration.standardHours(5));
    Window<Object> original = Window.into(windowFn);
    WindowingStrategy<?, ?> updated = WindowingStrategy.globalDefault().withWindowFn(windowFn);
    DisplayData displayData = DisplayData.from(new Window.Assign<>(original, updated));
    assertThat(displayData, hasDisplayItem("windowFn", windowFn.getClass()));
    assertThat(displayData, includesDisplayDataFor("windowFn", windowFn));
    assertThat(displayData, not(hasDisplayItem("trigger")));
    assertThat(displayData, not(hasDisplayItem("accumulationMode")));
    assertThat(displayData, not(hasDisplayItem("allowedLateness")));
    assertThat(displayData, not(hasDisplayItem("closingBehavior")));
    assertThat(displayData, not(hasDisplayItem("timestampCombiner")));
}
Also used : DisplayData(org.apache.beam.sdk.transforms.display.DisplayData) Test(org.junit.Test)

Example 33 with DisplayData

use of org.apache.beam.sdk.transforms.display.DisplayData in project beam by apache.

the class DatastoreV1Test method testDeleteKeyDisplayData.

@Test
public void testDeleteKeyDisplayData() {
    DeleteKey deleteKey = DatastoreIO.v1().deleteKey().withProjectId(PROJECT_ID);
    DisplayData displayData = DisplayData.from(deleteKey);
    assertThat(displayData, hasDisplayItem("projectId", PROJECT_ID));
}
Also used : DeleteKey(org.apache.beam.sdk.io.gcp.datastore.DatastoreV1.DeleteKey) DisplayData(org.apache.beam.sdk.transforms.display.DisplayData) Test(org.junit.Test)

Example 34 with DisplayData

use of org.apache.beam.sdk.transforms.display.DisplayData in project beam by apache.

the class DatastoreV1Test method testSourcePrimitiveDisplayData.

@Test
public void testSourcePrimitiveDisplayData() {
    DisplayDataEvaluator evaluator = DisplayDataEvaluator.create();
    int numSplits = 98;
    PTransform<PBegin, PCollection<Entity>> read = DatastoreIO.v1().read().withProjectId(PROJECT_ID).withQuery(Query.newBuilder().build()).withNumQuerySplits(numSplits);
    String assertMessage = "DatastoreIO read should include the '%s' in its primitive display data";
    Set<DisplayData> displayData = evaluator.displayDataForPrimitiveSourceTransforms(read);
    assertThat(String.format(assertMessage, "project id"), displayData, hasItem(hasDisplayItem("projectId", PROJECT_ID)));
    assertThat(String.format(assertMessage, "number of query splits"), displayData, hasItem(hasDisplayItem("numQuerySplits", numSplits)));
}
Also used : PCollection(org.apache.beam.sdk.values.PCollection) DisplayDataEvaluator(org.apache.beam.sdk.transforms.display.DisplayDataEvaluator) PBegin(org.apache.beam.sdk.values.PBegin) DisplayData(org.apache.beam.sdk.transforms.display.DisplayData) Test(org.junit.Test)

Example 35 with DisplayData

use of org.apache.beam.sdk.transforms.display.DisplayData in project beam by apache.

the class DatastoreV1Test method testReadDisplayData.

@Test
public void testReadDisplayData() {
    DatastoreV1.Read read = DatastoreIO.v1().read().withProjectId(PROJECT_ID).withQuery(QUERY).withNamespace(NAMESPACE);
    DisplayData displayData = DisplayData.from(read);
    assertThat(displayData, hasDisplayItem("projectId", PROJECT_ID));
    assertThat(displayData, hasDisplayItem("query", QUERY.toString()));
    assertThat(displayData, hasDisplayItem("namespace", NAMESPACE));
}
Also used : DisplayData(org.apache.beam.sdk.transforms.display.DisplayData) Test(org.junit.Test)

Aggregations

DisplayData (org.apache.beam.sdk.transforms.display.DisplayData)117 Test (org.junit.Test)115 DisplayDataEvaluator (org.apache.beam.sdk.transforms.display.DisplayDataEvaluator)16 PCollection (org.apache.beam.sdk.values.PCollection)11 ByteString (com.google.protobuf.ByteString)9 Category (org.junit.experimental.categories.Category)9 Matchers.containsString (org.hamcrest.Matchers.containsString)6 Duration (org.joda.time.Duration)6 TableRow (com.google.api.services.bigquery.model.TableRow)4 Params (org.apache.beam.sdk.io.DefaultFilenamePolicy.Params)3 BigQueryHelpers.toJsonString (org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers.toJsonString)3 Builder (org.apache.beam.sdk.transforms.display.DisplayData.Builder)3 Instant (org.joda.time.Instant)3 TableSchema (com.google.api.services.bigquery.model.TableSchema)2 RowFilter (com.google.bigtable.v2.RowFilter)2 ArrayList (java.util.ArrayList)2 WritableByteChannelFactory (org.apache.beam.sdk.io.FileBasedSink.WritableByteChannelFactory)2 ParDo (org.apache.beam.sdk.transforms.ParDo)2 StringUtils.byteArrayToJsonString (org.apache.beam.sdk.util.StringUtils.byteArrayToJsonString)2 ImmutableSet (com.google.common.collect.ImmutableSet)1