Examples with Resources co.cask.cdap.api.Resources used on opensource projects

Example 16 with Resources

use of co.cask.cdap.api.Resources in project cdap by caskdata.

the class PurchaseHistoryBuilder method configure.

@Override
public void configure() {
    setDescription("Purchase History Builder");
    setDriverResources(new Resources(1024));
    setMapperResources(new Resources(1024));
    setReducerResources(new Resources(1024));
}

Example 17 with Resources

use of co.cask.cdap.api.Resources in project cdap by caskdata.

the class PurchaseHistoryBuilder method initialize.

@Override
public void initialize() throws Exception {
    MapReduceContext context = getContext();
    Job job = context.getHadoopJob();
    job.setReducerClass(PerUserReducer.class);
    context.addInput(Input.ofDataset("purchases"), PurchaseMapper.class);
    context.addOutput(Output.ofDataset("history"));
    // override default memory usage if the corresponding runtime arguments are set.
    Map<String, String> runtimeArgs = context.getRuntimeArguments();
    String mapperMemoryMBStr = runtimeArgs.get(MAPPER_MEMORY_MB);
    if (mapperMemoryMBStr != null) {
        context.setMapperResources(new Resources(Integer.parseInt(mapperMemoryMBStr)));
    }
    String reducerMemoryMBStr = runtimeArgs.get(REDUCER_MEMORY_MB);
    if (reducerMemoryMBStr != null) {
        context.setReducerResources(new Resources(Integer.parseInt(reducerMemoryMBStr)));
    }
}

Example 18 with Resources

use of co.cask.cdap.api.Resources in project cdap by caskdata.

the class PurchaseStore method configure.

@Override
public void configure(FlowletConfigurer configurer) {
    super.configure(configurer);
    setDescription("Store the incoming Purchase objects in the purchases dataset");
    setResources(new Resources(1024));
}

Example 19 with Resources

use of co.cask.cdap.api.Resources in project cdap by caskdata.

the class FlowletSpecificationCodec method deserialize.

@Override
public FlowletSpecification deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
    JsonObject jsonObj = json.getAsJsonObject();
    String className = jsonObj.get("className").getAsString();
    String name = jsonObj.get("name").getAsString();
    String description = jsonObj.get("description").getAsString();
    FailurePolicy policy = FailurePolicy.valueOf(jsonObj.get("failurePolicy").getAsString());
    Set<String> dataSets = deserializeSet(jsonObj.get("datasets"), context, String.class);
    Map<String, String> properties = deserializeMap(jsonObj.get("properties"), context, String.class);
    Resources resources = context.deserialize(jsonObj.get("resources"), Resources.class);
    return new DefaultFlowletSpecification(className, name, description, policy, dataSets, properties, resources);
}

Example 20 with Resources

use of co.cask.cdap.api.Resources in project cdap by caskdata.

the class MapReduceSpecificationCodec method deserializeResources.

/**
   * Deserialize the resources field from the serialized object.
   *
   * @param jsonObj The object representing the MapReduceSpecification
   * @param prefix Field name prefix. Either "mapper" or "reducer"
   * @param context The context to deserialize object.
   * @return A {@link Resources} or {@code null}.
   */
private Resources deserializeResources(JsonObject jsonObj, String prefix, JsonDeserializationContext context) {
    // See if it of new format
    String name = prefix + "Resources";
    JsonElement element = jsonObj.get(name);
    if (element != null) {
        return context.deserialize(element, Resources.class);
    }
    // Try the old format, which is an int field representing the memory in MB.
    name = prefix + "MemoryMB";
    element = jsonObj.get(name);
    if (element != null) {
        return new Resources(element.getAsInt());
    }
    return null;
}