Search in sources :

Example 1 with Utf8

use of org.apache.avro.util.Utf8 in project hadoop by apache.

the class EventWriter method toAvro.

static JhCounters toAvro(Counters counters, String name) {
    JhCounters result = new JhCounters();
    result.setName(new Utf8(name));
    result.setGroups(new ArrayList<JhCounterGroup>(0));
    if (counters == null)
        return result;
    for (CounterGroup group : counters) {
        JhCounterGroup g = new JhCounterGroup();
        g.setName(new Utf8(group.getName()));
        g.setDisplayName(new Utf8(group.getDisplayName()));
        g.setCounts(new ArrayList<JhCounter>(group.size()));
        for (Counter counter : group) {
            JhCounter c = new JhCounter();
            c.setName(new Utf8(counter.getName()));
            c.setDisplayName(new Utf8(counter.getDisplayName()));
            c.setValue(counter.getValue());
            g.getCounts().add(c);
        }
        result.getGroups().add(g);
    }
    return result;
}
Also used : Counter(org.apache.hadoop.mapreduce.Counter) CounterGroup(org.apache.hadoop.mapreduce.CounterGroup) Utf8(org.apache.avro.util.Utf8)

Example 2 with Utf8

use of org.apache.avro.util.Utf8 in project hadoop by apache.

the class TaskAttemptFinishedEvent method getDatum.

public Object getDatum() {
    if (datum == null) {
        datum = new TaskAttemptFinished();
        datum.setTaskid(new Utf8(attemptId.getTaskID().toString()));
        datum.setAttemptId(new Utf8(attemptId.toString()));
        datum.setTaskType(new Utf8(taskType.name()));
        datum.setTaskStatus(new Utf8(taskStatus));
        datum.setFinishTime(finishTime);
        if (rackName != null) {
            datum.setRackname(new Utf8(rackName));
        }
        datum.setHostname(new Utf8(hostname));
        datum.setState(new Utf8(state));
        datum.setCounters(EventWriter.toAvro(counters));
    }
    return datum;
}
Also used : Utf8(org.apache.avro.util.Utf8)

Example 3 with Utf8

use of org.apache.avro.util.Utf8 in project hadoop by apache.

the class TaskFailedEvent method getDatum.

public Object getDatum() {
    if (datum == null) {
        datum = new TaskFailed();
        datum.setTaskid(new Utf8(id.toString()));
        datum.setError(new Utf8(error));
        datum.setFinishTime(finishTime);
        datum.setTaskType(new Utf8(taskType.name()));
        datum.setFailedDueToAttempt(failedDueToAttempt == null ? null : new Utf8(failedDueToAttempt.toString()));
        datum.setStatus(new Utf8(status));
        datum.setCounters(EventWriter.toAvro(counters));
    }
    return datum;
}
Also used : Utf8(org.apache.avro.util.Utf8)

Example 4 with Utf8

use of org.apache.avro.util.Utf8 in project hadoop by apache.

the class TaskFinishedEvent method getDatum.

public Object getDatum() {
    if (datum == null) {
        datum = new TaskFinished();
        datum.setTaskid(new Utf8(taskid.toString()));
        if (successfulAttemptId != null) {
            datum.setSuccessfulAttemptId(new Utf8(successfulAttemptId.toString()));
        }
        datum.setFinishTime(finishTime);
        datum.setCounters(EventWriter.toAvro(counters));
        datum.setTaskType(new Utf8(taskType.name()));
        datum.setStatus(new Utf8(status));
    }
    return datum;
}
Also used : Utf8(org.apache.avro.util.Utf8)

Example 5 with Utf8

use of org.apache.avro.util.Utf8 in project hadoop by apache.

the class MapAttemptFinishedEvent method getDatum.

public Object getDatum() {
    if (datum == null) {
        datum = new MapAttemptFinished();
        datum.setTaskid(new Utf8(attemptId.getTaskID().toString()));
        datum.setAttemptId(new Utf8(attemptId.toString()));
        datum.setTaskType(new Utf8(taskType.name()));
        datum.setTaskStatus(new Utf8(taskStatus));
        datum.setMapFinishTime(mapFinishTime);
        datum.setFinishTime(finishTime);
        datum.setHostname(new Utf8(hostname));
        datum.setPort(port);
        if (rackName != null) {
            datum.setRackname(new Utf8(rackName));
        }
        datum.setState(new Utf8(state));
        datum.setCounters(EventWriter.toAvro(counters));
        datum.setClockSplits(AvroArrayUtils.toAvro(ProgressSplitsBlock.arrayGetWallclockTime(allSplits)));
        datum.setCpuUsages(AvroArrayUtils.toAvro(ProgressSplitsBlock.arrayGetCPUTime(allSplits)));
        datum.setVMemKbytes(AvroArrayUtils.toAvro(ProgressSplitsBlock.arrayGetVMemKbytes(allSplits)));
        datum.setPhysMemKbytes(AvroArrayUtils.toAvro(ProgressSplitsBlock.arrayGetPhysMemKbytes(allSplits)));
    }
    return datum;
}
Also used : Utf8(org.apache.avro.util.Utf8)

Aggregations

Utf8 (org.apache.avro.util.Utf8)123 Test (org.junit.Test)34 WebPage (org.apache.gora.examples.generated.WebPage)32 GenericRecord (org.apache.avro.generic.GenericRecord)17 Schema (org.apache.avro.Schema)14 GenericData (org.apache.avro.generic.GenericData)13 ByteBuffer (java.nio.ByteBuffer)12 HashMap (java.util.HashMap)12 Map (java.util.Map)12 Employee (org.apache.gora.examples.generated.Employee)11 IOException (java.io.IOException)7 ArrayList (java.util.ArrayList)7 Field (org.apache.avro.Schema.Field)6 Record (org.apache.avro.generic.GenericData.Record)5 File (java.io.File)4 SpecificDatumReader (org.apache.avro.specific.SpecificDatumReader)4 Metadata (org.apache.gora.examples.generated.Metadata)4 ByteArrayInputStream (java.io.ByteArrayInputStream)3 Iterator (java.util.Iterator)3 List (java.util.List)3