Search in sources :

Example 1 with SequenceFormat

use of org.apache.storm.hdfs.bolt.format.SequenceFormat in project storm by apache.

the class TestSequenceFileBolt method makeSeqBolt.

private SequenceFileBolt makeSeqBolt(String nameNodeAddr, int countSync, float rotationSizeMB) {
    SyncPolicy fieldsSyncPolicy = new CountSyncPolicy(countSync);
    FileRotationPolicy fieldsRotationPolicy = new FileSizeRotationPolicy(rotationSizeMB, FileSizeRotationPolicy.Units.MB);
    FileNameFormat fieldsFileNameFormat = new DefaultFileNameFormat().withPath(testRoot);
    SequenceFormat seqFormat = new DefaultSequenceFormat("key", "value");
    return new SequenceFileBolt().withFsUrl(nameNodeAddr).withFileNameFormat(fieldsFileNameFormat).withRotationPolicy(fieldsRotationPolicy).withSequenceFormat(seqFormat).withSyncPolicy(fieldsSyncPolicy);
}
Also used : DefaultSequenceFormat(org.apache.storm.hdfs.bolt.format.DefaultSequenceFormat) SequenceFormat(org.apache.storm.hdfs.bolt.format.SequenceFormat) CountSyncPolicy(org.apache.storm.hdfs.bolt.sync.CountSyncPolicy) CountSyncPolicy(org.apache.storm.hdfs.bolt.sync.CountSyncPolicy) SyncPolicy(org.apache.storm.hdfs.bolt.sync.SyncPolicy) DefaultFileNameFormat(org.apache.storm.hdfs.bolt.format.DefaultFileNameFormat) FileNameFormat(org.apache.storm.hdfs.bolt.format.FileNameFormat) FileRotationPolicy(org.apache.storm.hdfs.bolt.rotation.FileRotationPolicy) FileSizeRotationPolicy(org.apache.storm.hdfs.bolt.rotation.FileSizeRotationPolicy) DefaultFileNameFormat(org.apache.storm.hdfs.bolt.format.DefaultFileNameFormat) DefaultSequenceFormat(org.apache.storm.hdfs.bolt.format.DefaultSequenceFormat)

Aggregations

DefaultFileNameFormat (org.apache.storm.hdfs.bolt.format.DefaultFileNameFormat)1 DefaultSequenceFormat (org.apache.storm.hdfs.bolt.format.DefaultSequenceFormat)1 FileNameFormat (org.apache.storm.hdfs.bolt.format.FileNameFormat)1 SequenceFormat (org.apache.storm.hdfs.bolt.format.SequenceFormat)1 FileRotationPolicy (org.apache.storm.hdfs.bolt.rotation.FileRotationPolicy)1 FileSizeRotationPolicy (org.apache.storm.hdfs.bolt.rotation.FileSizeRotationPolicy)1 CountSyncPolicy (org.apache.storm.hdfs.bolt.sync.CountSyncPolicy)1 SyncPolicy (org.apache.storm.hdfs.bolt.sync.SyncPolicy)1