Search in sources :

Example 1 with TaskResult

use of alluxio.stress.TaskResult in project alluxio by Alluxio.

the class StressBenchDefinition method join.

@Override
public String join(StressBenchConfig config, Map<WorkerInfo, String> taskResults) throws Exception {
    if (taskResults.isEmpty()) {
        throw new IOException("No results from any workers.");
    }
    AtomicReference<IOException> error = new AtomicReference<>(null);
    List<TaskResult> results = taskResults.entrySet().stream().map(entry -> {
        try {
            return JsonSerializable.fromJson(entry.getValue().trim(), new TaskResult[0]);
        } catch (IOException | ClassNotFoundException e) {
            // add log here because the exception details are lost at the client side
            LOG.warn("Failed to parse result into class {}", TaskResult.class, e);
            error.set(new IOException(String.format("Failed to parse task output from %s into result class %s: %s", entry.getKey().getAddress().getHost(), TaskResult.class, entry.getValue().trim()), e));
        }
        return null;
    }).collect(Collectors.toList());
    if (error.get() != null) {
        throw error.get();
    }
    return results.get(0).aggregator().aggregate(results).toJson();
}
Also used : JsonSerializable(alluxio.util.JsonSerializable) LoggerFactory(org.slf4j.LoggerFactory) UnderFileSystemConfiguration(alluxio.underfs.UnderFileSystemConfiguration) PropertyKey(alluxio.conf.PropertyKey) AtomicReference(java.util.concurrent.atomic.AtomicReference) ArrayList(java.util.ArrayList) Lists(com.google.common.collect.Lists) RunTaskContext(alluxio.job.RunTaskContext) CloseableResource(alluxio.resource.CloseableResource) WorkerInfo(alluxio.wire.WorkerInfo) Map(java.util.Map) UfsIOParameters(alluxio.stress.worker.UfsIOParameters) AlluxioConfiguration(alluxio.conf.AlluxioConfiguration) Logger(org.slf4j.Logger) ServerConfiguration(alluxio.conf.ServerConfiguration) ImmutableMap(com.google.common.collect.ImmutableMap) Set(java.util.Set) SelectExecutorsContext(alluxio.job.SelectExecutorsContext) IOException(java.io.IOException) Pair(alluxio.collections.Pair) Collectors(java.util.stream.Collectors) Sets(com.google.common.collect.Sets) List(java.util.List) PlanDefinition(alluxio.job.plan.PlanDefinition) StressBenchConfig(alluxio.stress.job.StressBenchConfig) UnderFileSystem(alluxio.underfs.UnderFileSystem) BaseParameters(alluxio.stress.BaseParameters) Comparator(java.util.Comparator) TaskResult(alluxio.stress.TaskResult) Collections(java.util.Collections) ShellUtils(alluxio.util.ShellUtils) MountPointInfo(alluxio.wire.MountPointInfo) TaskResult(alluxio.stress.TaskResult) AtomicReference(java.util.concurrent.atomic.AtomicReference) IOException(java.io.IOException)

Aggregations

Pair (alluxio.collections.Pair)1 AlluxioConfiguration (alluxio.conf.AlluxioConfiguration)1 PropertyKey (alluxio.conf.PropertyKey)1 ServerConfiguration (alluxio.conf.ServerConfiguration)1 RunTaskContext (alluxio.job.RunTaskContext)1 SelectExecutorsContext (alluxio.job.SelectExecutorsContext)1 PlanDefinition (alluxio.job.plan.PlanDefinition)1 CloseableResource (alluxio.resource.CloseableResource)1 BaseParameters (alluxio.stress.BaseParameters)1 TaskResult (alluxio.stress.TaskResult)1 StressBenchConfig (alluxio.stress.job.StressBenchConfig)1 UfsIOParameters (alluxio.stress.worker.UfsIOParameters)1 UnderFileSystem (alluxio.underfs.UnderFileSystem)1 UnderFileSystemConfiguration (alluxio.underfs.UnderFileSystemConfiguration)1 JsonSerializable (alluxio.util.JsonSerializable)1 ShellUtils (alluxio.util.ShellUtils)1 MountPointInfo (alluxio.wire.MountPointInfo)1 WorkerInfo (alluxio.wire.WorkerInfo)1 ImmutableMap (com.google.common.collect.ImmutableMap)1 Lists (com.google.common.collect.Lists)1