Examples with AddressBook com.twitter.data.proto.tutorial.AddressBookProtos.AddressBook used on opensource projects

Example 1 with AddressBook

use of com.twitter.data.proto.tutorial.AddressBookProtos.AddressBook in project elephant-bird by twitter.

the class TestTypedProtobufWritable method testMessageReadWriteEmpty.

@Test
public void testMessageReadWriteEmpty() throws IOException {
    DataOutputStream dos = new DataOutputStream(new FileOutputStream("test3.txt"));
    TypedProtobufWritable<AddressBook> empty = new TypedProtobufWritable<AddressBook>();
    empty.write(dos);
    dos.close();
    DataInputStream dis = new DataInputStream(new FileInputStream("test3.txt"));
    TypedProtobufWritable<Message> after = new TypedProtobufWritable<Message>();
    after.readFields(dis);
    dis.close();
    AddressBook ab2 = (AddressBook) after.get();
    assertNull(ab2);
}

Example 2 with AddressBook

use of com.twitter.data.proto.tutorial.AddressBookProtos.AddressBook in project elephant-bird by twitter.

the class TestTypedProtobufWritable method testMessageReadWrite.

@Test
public void testMessageReadWrite() throws IOException {
    DataOutputStream dos = new DataOutputStream(new FileOutputStream("test2.txt"));
    referenceAbWritable.write(dos);
    dos.close();
    DataInputStream dis = new DataInputStream(new FileInputStream("test2.txt"));
    TypedProtobufWritable<Message> after = new TypedProtobufWritable<Message>();
    after.readFields(dis);
    dis.close();
    AddressBook ab2 = (AddressBook) after.get();
    assertEquals(referenceAb, ab2);
    assertEquals(referenceAbWritable.hashCode(), after.hashCode());
}

Example 3 with AddressBook

use of com.twitter.data.proto.tutorial.AddressBookProtos.AddressBook in project elephant-bird by twitter.

the class TestProtoToPig method testProtoToPig.

@Test
public void testProtoToPig() throws IOException {
    AddressBook abProto = Fixtures.buildAddressBookProto();
    Tuple abProtoTuple = tf_.newTuple(new DataByteArray(abProto.toByteArray()));
    ProtobufBytesToTuple abProtoToPig = new ProtobufBytesToTuple(AddressBook.class.getCanonicalName());
    Tuple abTuple = abProtoToPig.exec(abProtoTuple);
    assertEquals("{(Elephant Bird,123,elephant@bird.com,{(415-999-9999,HOME),(415-666-6666,MOBILE),(415-333-3333,WORK)}),(Elephant Bird,123,elephant@bird.com,{(415-999-9999,HOME),(415-666-6666,MOBILE),(415-333-3333,WORK)})},", abTuple.toDelimitedString(","));
}

Example 4 with AddressBook

use of com.twitter.data.proto.tutorial.AddressBookProtos.AddressBook in project elephant-bird by twitter.

the class ProtobufDeserializerTest method testDeserializer.

@Test
public final void testDeserializer() throws SerDeException {
    BytesWritable serialized = new BytesWritable(test_ab.toByteArray());
    AddressBook ab2 = (AddressBook) deserializer.deserialize(serialized);
    assertTrue(test_ab.equals(ab2));
}

Example 5 with AddressBook

use of com.twitter.data.proto.tutorial.AddressBookProtos.AddressBook in project elephant-bird by twitter.

the class CrunchElephantBirdExample method run.

@Override
public int run(String[] args) throws Exception {
    if (args.length != 3) {
        System.err.println("Usage: <address_book.proto> <people.proto> <output_dir>");
        return 1;
    }
    // Creates a reference to a collection of address books that we will read into memory in our
    // map tasks.
    PCollection<AddressBook> addresses = read(LzoProtobufSource.at(new Path(args[0]), AddressBook.class));
    // Processes a large collection of Person records, limiting it to those people who are found in
    // at least one of the AddressBook records. The address books are assumed to be small, and so we
    // pass a readable version of them into the function we use to process the Person records on the
    // map-side.
    PCollection<Person> found = read(LzoProtobufSource.at(new Path(args[1]), Person.class)).filter(new PersonInAddressBookFn(addresses.asReadable(false)));
    // Write the distinct Person records that made it past the address book filter to the given output directory
    // as protocol buffer records.
    write(distinct(found), new LzoProtobufTarget(new Path(args[2])));
    // Execute the pipeline and return a success indicator
    return done().succeeded() ? 0 : 1;
}