Examples with Read org.apache.beam.sdk.io.hcatalog.HCatalogIO.Read used on opensource projects

Example 1 with Read

use of org.apache.beam.sdk.io.hcatalog.HCatalogIO.Read in project beam by apache.

the class PartitionReaderFn method processElement.

@ProcessElement
public void processElement(ProcessContext c) throws Exception {
    final Read readRequest = c.element().getKey();
    final Integer partitionIndexToRead = c.element().getValue();
    ReaderContext readerContext = getReaderContext(readRequest, partitionIndexToRead);
    for (int i = 0; i < readerContext.numSplits(); i++) {
        HCatReader reader = DataTransferFactory.getHCatReader(readerContext, i);
        Iterator<HCatRecord> hcatIterator = reader.read();
        while (hcatIterator.hasNext()) {
            final HCatRecord record = hcatIterator.next();
            c.output(record);
        }
    }
}