Search in sources :

Example 1 with CoderException

use of org.apache.beam.sdk.coders.CoderException in project beam by apache.

the class WritableCoder method decode.

@SuppressWarnings("unchecked")
@Override
public T decode(InputStream inStream) throws IOException {
    try {
        if (type == NullWritable.class) {
            // NullWritable has no default constructor
            return (T) NullWritable.get();
        }
        T t = type.newInstance();
        t.readFields(new DataInputStream(inStream));
        return t;
    } catch (InstantiationException | IllegalAccessException e) {
        throw new CoderException("unable to deserialize record", e);
    }
}
Also used : CannotProvideCoderException(org.apache.beam.sdk.coders.CannotProvideCoderException) CoderException(org.apache.beam.sdk.coders.CoderException) DataInputStream(java.io.DataInputStream)

Example 2 with CoderException

use of org.apache.beam.sdk.coders.CoderException in project beam by apache.

the class JAXBCoder method encode.

@Override
public void encode(T value, OutputStream outStream, Context context) throws CoderException, IOException {
    if (context.isWholeStream) {
        try {
            jaxbMarshaller.get().marshal(value, new CloseIgnoringOutputStream(outStream));
        } catch (JAXBException e) {
            throw new CoderException(e);
        }
    } else {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        try {
            jaxbMarshaller.get().marshal(value, baos);
        } catch (JAXBException e) {
            throw new CoderException(e);
        }
        VarInt.encode(baos.size(), outStream);
        baos.writeTo(outStream);
    }
}
Also used : JAXBException(javax.xml.bind.JAXBException) CoderException(org.apache.beam.sdk.coders.CoderException) ByteArrayOutputStream(java.io.ByteArrayOutputStream)

Example 3 with CoderException

use of org.apache.beam.sdk.coders.CoderException in project beam by apache.

the class ApexTimerInternals method fireReadyTimers.

/**
   * Fire the timers that are ready. These are the timers
   * that are registered to be triggered at a time before the current time.
   * Timer processing may register new timers, which can cause the returned
   * timestamp to be before the the current time. The caller may repeat
   * the call until such backdated timers are cleared.
   * @return minimum timestamp of registered timers.
   */
public long fireReadyTimers(long currentTime, TimerProcessor<K> timerProcessor, TimeDomain timeDomain) {
    TimerSet timers = getTimerSet(timeDomain);
    // move minTimestamp first,
    // timer additions that result from firing may modify it
    timers.minTimestamp = currentTime;
    // we keep the timers to return in a different list and launch them later
    // because we cannot prevent a trigger from registering another timer,
    // which would lead to concurrent modification exception.
    Multimap<Slice, TimerInternals.TimerData> toFire = HashMultimap.create();
    Iterator<Map.Entry<Slice, Set<Slice>>> it = timers.activeTimers.entrySet().iterator();
    while (it.hasNext()) {
        Map.Entry<Slice, Set<Slice>> keyWithTimers = it.next();
        Iterator<Slice> timerIt = keyWithTimers.getValue().iterator();
        while (timerIt.hasNext()) {
            try {
                TimerData timerData = CoderUtils.decodeFromByteArray(timers.timerDataCoder, timerIt.next().buffer);
                if (timerData.getTimestamp().isBefore(currentTime)) {
                    toFire.put(keyWithTimers.getKey(), timerData);
                    timerIt.remove();
                }
            } catch (CoderException e) {
                throw new RuntimeException(e);
            }
        }
        if (keyWithTimers.getValue().isEmpty()) {
            it.remove();
        }
    }
    // fire ready timers
    if (!toFire.isEmpty()) {
        for (Slice keyBytes : toFire.keySet()) {
            try {
                K key = CoderUtils.decodeFromByteArray(keyCoder, keyBytes.buffer);
                timerProcessor.fireTimer(key, toFire.get(keyBytes));
            } catch (CoderException e) {
                throw new RuntimeException(e);
            }
        }
    }
    return timers.minTimestamp;
}
Also used : Set(java.util.Set) HashSet(java.util.HashSet) Slice(com.datatorrent.netlet.util.Slice) CoderException(org.apache.beam.sdk.coders.CoderException) HashMap(java.util.HashMap) Map(java.util.Map)

Example 4 with CoderException

use of org.apache.beam.sdk.coders.CoderException in project beam by apache.

the class JAXBCoder method decode.

@Override
public T decode(InputStream inStream, Context context) throws CoderException, IOException {
    try {
        if (!context.isWholeStream) {
            long limit = VarInt.decodeLong(inStream);
            inStream = ByteStreams.limit(inStream, limit);
        }
        @SuppressWarnings("unchecked") T obj = (T) jaxbUnmarshaller.get().unmarshal(new CloseIgnoringInputStream(inStream));
        return obj;
    } catch (JAXBException e) {
        throw new CoderException(e);
    }
}
Also used : JAXBException(javax.xml.bind.JAXBException) CoderException(org.apache.beam.sdk.coders.CoderException)

Example 5 with CoderException

use of org.apache.beam.sdk.coders.CoderException in project beam by apache.

the class CoderUtilsTest method testCoderExceptionPropagation.

@Test
public void testCoderExceptionPropagation() throws Exception {
    @SuppressWarnings("unchecked") Coder<String> crashingCoder = mock(Coder.class);
    doThrow(new CoderException("testing exception")).when(crashingCoder).encode(anyString(), any(OutputStream.class), any(Coder.Context.class));
    expectedException.expect(CoderException.class);
    expectedException.expectMessage("testing exception");
    CoderUtils.encodeToByteArray(crashingCoder, "hello");
}
Also used : Context(org.apache.beam.sdk.coders.Coder.Context) OutputStream(java.io.OutputStream) Matchers.anyString(org.mockito.Matchers.anyString) CoderException(org.apache.beam.sdk.coders.CoderException) Test(org.junit.Test)

Aggregations

CoderException (org.apache.beam.sdk.coders.CoderException)6 JAXBException (javax.xml.bind.JAXBException)2 Test (org.junit.Test)2 Slice (com.datatorrent.netlet.util.Slice)1 ByteArrayOutputStream (java.io.ByteArrayOutputStream)1 DataInputStream (java.io.DataInputStream)1 OutputStream (java.io.OutputStream)1 HashMap (java.util.HashMap)1 HashSet (java.util.HashSet)1 Map (java.util.Map)1 Set (java.util.Set)1 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)1 Pipeline (org.apache.beam.sdk.Pipeline)1 CannotProvideCoderException (org.apache.beam.sdk.coders.CannotProvideCoderException)1 Context (org.apache.beam.sdk.coders.Coder.Context)1 TestPipeline (org.apache.beam.sdk.testing.TestPipeline)1 SerializableFunction (org.apache.beam.sdk.transforms.SerializableFunction)1 SimpleFunction (org.apache.beam.sdk.transforms.SimpleFunction)1 KV (org.apache.beam.sdk.values.KV)1 TypeDescriptor (org.apache.beam.sdk.values.TypeDescriptor)1