Search in sources :

Example 31 with Value

use of com.google.api.ads.admanager.jaxws.v202108.Value in project beam by apache.

the class FirestoreV1FnBatchWriteWithDeadLetterQueueTest method enqueueingWritesValidateBytesSize.

@Override
@Test
public void enqueueingWritesValidateBytesSize() throws Exception {
    int maxBytes = 50;
    RpcQosOptions options = rpcQosOptions.toBuilder().withBatchMaxBytes(maxBytes).build();
    when(ff.getFirestoreStub(any())).thenReturn(stub);
    when(ff.getRpcQos(any())).thenReturn(FirestoreStatefulComponentFactory.INSTANCE.getRpcQos(options));
    byte[] bytes = new byte[maxBytes + 1];
    SecureRandom.getInstanceStrong().nextBytes(bytes);
    byte[] base64Bytes = Base64.getEncoder().encode(bytes);
    String base64String = Base64.getEncoder().encodeToString(bytes);
    Value largeValue = Value.newBuilder().setStringValueBytes(ByteString.copyFrom(base64Bytes)).build();
    // apply a doc transform that is too large
    Write write1 = Write.newBuilder().setTransform(DocumentTransform.newBuilder().setDocument(String.format("doc-%03d", 2)).addFieldTransforms(FieldTransform.newBuilder().setAppendMissingElements(ArrayValue.newBuilder().addValues(largeValue)))).build();
    // delete a doc that is too large
    Write write2 = Write.newBuilder().setDelete(String.format("doc-%03d_%s", 3, base64String)).build();
    // update a doc that is too large
    Write write3 = Write.newBuilder().setUpdate(Document.newBuilder().setName(String.format("doc-%03d", 4)).putAllFields(ImmutableMap.of("foo", largeValue))).build();
    BatchWriteFnWithDeadLetterQueue fn = getFn(clock, ff, options, metricsFixture.counterFactory, metricsFixture.distributionFactory);
    fn.populateDisplayData(displayDataBuilder);
    fn.setup();
    fn.startBundle(startBundleContext);
    ArgumentCaptor<WriteFailure> write1FailureCapture = ArgumentCaptor.forClass(WriteFailure.class);
    doNothing().when(processContext).outputWithTimestamp(write1FailureCapture.capture(), any());
    when(processContext.element()).thenReturn(write1);
    fn.processElement(processContext, window);
    WriteFailure failure = write1FailureCapture.getValue();
    assertNotNull(failure);
    String message = failure.getStatus().getMessage();
    assertTrue(message.contains("TRANSFORM"));
    assertTrue(message.contains("larger than configured max allowed bytes per batch"));
    ArgumentCaptor<WriteFailure> write2FailureCapture = ArgumentCaptor.forClass(WriteFailure.class);
    doNothing().when(processContext).outputWithTimestamp(write2FailureCapture.capture(), any());
    when(processContext.element()).thenReturn(write2);
    fn.processElement(processContext, window);
    WriteFailure failure2 = write2FailureCapture.getValue();
    assertNotNull(failure2);
    String message2 = failure2.getStatus().getMessage();
    assertTrue(message2.contains("DELETE"));
    assertTrue(message2.contains("larger than configured max allowed bytes per batch"));
    ArgumentCaptor<WriteFailure> write3FailureCapture = ArgumentCaptor.forClass(WriteFailure.class);
    doNothing().when(processContext).outputWithTimestamp(write3FailureCapture.capture(), any());
    when(processContext.element()).thenReturn(write3);
    fn.processElement(processContext, window);
    WriteFailure failure3 = write3FailureCapture.getValue();
    assertNotNull(failure3);
    String message3 = failure3.getStatus().getMessage();
    assertTrue(message3.contains("UPDATE"));
    assertTrue(message3.contains("larger than configured max allowed bytes per batch"));
    assertEquals(0, fn.writes.size());
}
Also used : Write(com.google.firestore.v1.Write) BatchWriteFnWithDeadLetterQueue(org.apache.beam.sdk.io.gcp.firestore.FirestoreV1WriteFn.BatchWriteFnWithDeadLetterQueue) Value(com.google.firestore.v1.Value) ArrayValue(com.google.firestore.v1.ArrayValue) WriteFailure(org.apache.beam.sdk.io.gcp.firestore.FirestoreV1.WriteFailure) ByteString(com.google.protobuf.ByteString) Test(org.junit.Test)

Example 32 with Value

use of com.google.api.ads.admanager.jaxws.v202108.Value in project beam by apache.

the class FirestoreV1FnBatchWriteWithSummaryTest method enqueueingWritesValidateBytesSize.

@Override
@Test
public void enqueueingWritesValidateBytesSize() throws Exception {
    int maxBytes = 50;
    RpcQosOptions options = rpcQosOptions.toBuilder().withBatchMaxBytes(maxBytes).build();
    when(ff.getFirestoreStub(any())).thenReturn(stub);
    when(ff.getRpcQos(any())).thenReturn(FirestoreStatefulComponentFactory.INSTANCE.getRpcQos(options));
    byte[] bytes = new byte[maxBytes + 1];
    SecureRandom.getInstanceStrong().nextBytes(bytes);
    byte[] base64Bytes = Base64.getEncoder().encode(bytes);
    String base64String = Base64.getEncoder().encodeToString(bytes);
    Value largeValue = Value.newBuilder().setStringValueBytes(ByteString.copyFrom(base64Bytes)).build();
    // apply a doc transform that is too large
    Write write1 = Write.newBuilder().setTransform(DocumentTransform.newBuilder().setDocument(String.format("doc-%03d", 2)).addFieldTransforms(FieldTransform.newBuilder().setAppendMissingElements(ArrayValue.newBuilder().addValues(largeValue)))).build();
    // delete a doc that is too large
    Write write2 = Write.newBuilder().setDelete(String.format("doc-%03d_%s", 3, base64String)).build();
    // update a doc that is too large
    Write write3 = Write.newBuilder().setUpdate(Document.newBuilder().setName(String.format("doc-%03d", 4)).putAllFields(ImmutableMap.of("foo", largeValue))).build();
    BatchWriteFnWithSummary fn = getFn(clock, ff, options, metricsFixture.counterFactory, metricsFixture.distributionFactory);
    fn.populateDisplayData(displayDataBuilder);
    fn.setup();
    fn.startBundle(startBundleContext);
    try {
        when(processContext.element()).thenReturn(write1);
        fn.processElement(processContext, window);
        fail("expected validation error");
    } catch (FailedWritesException e) {
        WriteFailure failure = e.getWriteFailures().get(0);
        assertNotNull(failure);
        String message = failure.getStatus().getMessage();
        assertTrue(message.contains("TRANSFORM"));
        assertTrue(message.contains("larger than configured max allowed bytes per batch"));
    }
    try {
        when(processContext.element()).thenReturn(write2);
        fn.processElement(processContext, window);
        fail("expected validation error");
    } catch (FailedWritesException e) {
        WriteFailure failure = e.getWriteFailures().get(0);
        assertNotNull(failure);
        String message = failure.getStatus().getMessage();
        assertTrue(message.contains("DELETE"));
        assertTrue(message.contains("larger than configured max allowed bytes per batch"));
    }
    try {
        when(processContext.element()).thenReturn(write3);
        fn.processElement(processContext, window);
        fail("expected validation error");
    } catch (FailedWritesException e) {
        WriteFailure failure = e.getWriteFailures().get(0);
        assertNotNull(failure);
        String message = failure.getStatus().getMessage();
        assertTrue(message.contains("UPDATE"));
        assertTrue(message.contains("larger than configured max allowed bytes per batch"));
    }
    assertEquals(0, fn.writes.size());
}
Also used : Write(com.google.firestore.v1.Write) FailedWritesException(org.apache.beam.sdk.io.gcp.firestore.FirestoreV1.FailedWritesException) Value(com.google.firestore.v1.Value) ArrayValue(com.google.firestore.v1.ArrayValue) WriteFailure(org.apache.beam.sdk.io.gcp.firestore.FirestoreV1.WriteFailure) ByteString(com.google.protobuf.ByteString) BatchWriteFnWithSummary(org.apache.beam.sdk.io.gcp.firestore.FirestoreV1WriteFn.BatchWriteFnWithSummary) Test(org.junit.Test)

Example 33 with Value

use of com.google.api.ads.admanager.jaxws.v202108.Value in project spring-cloud-gcp by spring-cloud.

the class FirestoreDefaultClassMapper method entityToDocument.

public <T> Document entityToDocument(T entity, String documentResourceName) {
    DocumentSnapshot documentSnapshot = INTERNAL.snapshotFromObject(NOT_USED_PATH, entity);
    Map<String, Value> valuesMap = INTERNAL.protoFromSnapshot(documentSnapshot);
    return Document.newBuilder().putAllFields(valuesMap).setName(documentResourceName).build();
}
Also used : DocumentSnapshot(com.google.cloud.firestore.DocumentSnapshot) Value(com.google.firestore.v1.Value)

Example 34 with Value

use of com.google.api.ads.admanager.jaxws.v202108.Value in project googleads-java-lib by googleads.

the class PqlTest method testCreateValue_dateSet.

@Test
public void testCreateValue_dateSet() {
    Set<Date> numberSet = new LinkedHashSet<Date>();
    numberSet.add(date1);
    Value value1 = ((SetValue) Pql.createValue(numberSet)).getValues(0);
    assertEquals("2012-12-02", DateTimes.toString(((DateValue) value1).getValue()));
}
Also used : LinkedHashSet(java.util.LinkedHashSet) DateValue(com.google.api.ads.admanager.axis.v202105.DateValue) TextValue(com.google.api.ads.admanager.axis.v202105.TextValue) BooleanValue(com.google.api.ads.admanager.axis.v202105.BooleanValue) DateTimeValue(com.google.api.ads.admanager.axis.v202105.DateTimeValue) Value(com.google.api.ads.admanager.axis.v202105.Value) DateValue(com.google.api.ads.admanager.axis.v202105.DateValue) TargetingValue(com.google.api.ads.admanager.axis.v202105.TargetingValue) NumberValue(com.google.api.ads.admanager.axis.v202105.NumberValue) SetValue(com.google.api.ads.admanager.axis.v202105.SetValue) SetValue(com.google.api.ads.admanager.axis.v202105.SetValue) Date(com.google.api.ads.admanager.axis.v202105.Date) Test(org.junit.Test)

Example 35 with Value

use of com.google.api.ads.admanager.jaxws.v202108.Value in project googleads-java-lib by googleads.

the class PqlTest method testCreateValue_textSet.

@Test
public void testCreateValue_textSet() {
    Set<String> textSet = new LinkedHashSet<String>();
    textSet.add("value1");
    Value value1 = ((SetValue) Pql.createValue(textSet)).getValues(0);
    assertEquals("value1", ((TextValue) value1).getValue());
}
Also used : LinkedHashSet(java.util.LinkedHashSet) TextValue(com.google.api.ads.admanager.axis.v202105.TextValue) BooleanValue(com.google.api.ads.admanager.axis.v202105.BooleanValue) DateTimeValue(com.google.api.ads.admanager.axis.v202105.DateTimeValue) Value(com.google.api.ads.admanager.axis.v202105.Value) DateValue(com.google.api.ads.admanager.axis.v202105.DateValue) TargetingValue(com.google.api.ads.admanager.axis.v202105.TargetingValue) NumberValue(com.google.api.ads.admanager.axis.v202105.NumberValue) SetValue(com.google.api.ads.admanager.axis.v202105.SetValue) SetValue(com.google.api.ads.admanager.axis.v202105.SetValue) Test(org.junit.Test)

Aggregations

LinkedHashSet (java.util.LinkedHashSet)56 Test (org.junit.Test)42 BooleanValue (com.google.api.ads.admanager.axis.v202105.BooleanValue)7 DateTimeValue (com.google.api.ads.admanager.axis.v202105.DateTimeValue)7 DateValue (com.google.api.ads.admanager.axis.v202105.DateValue)7 NumberValue (com.google.api.ads.admanager.axis.v202105.NumberValue)7 SetValue (com.google.api.ads.admanager.axis.v202105.SetValue)7 TargetingValue (com.google.api.ads.admanager.axis.v202105.TargetingValue)7 BooleanValue (com.google.api.ads.admanager.jaxws.v202202.BooleanValue)6 DateTimeValue (com.google.api.ads.admanager.jaxws.v202202.DateTimeValue)6 DateValue (com.google.api.ads.admanager.jaxws.v202202.DateValue)6 NumberValue (com.google.api.ads.admanager.jaxws.v202202.NumberValue)6 SetValue (com.google.api.ads.admanager.jaxws.v202202.SetValue)6 TargetingValue (com.google.api.ads.admanager.jaxws.v202202.TargetingValue)6 TextValue (com.google.api.ads.admanager.jaxws.v202202.TextValue)6 Value (com.google.api.ads.admanager.jaxws.v202202.Value)6 BooleanValue (com.google.api.ads.admanager.axis.v202108.BooleanValue)5 DateTimeValue (com.google.api.ads.admanager.axis.v202108.DateTimeValue)5 DateValue (com.google.api.ads.admanager.axis.v202108.DateValue)5 NumberValue (com.google.api.ads.admanager.axis.v202108.NumberValue)5