Search in sources :

Example 1 with LocalResourceType

use of org.apache.hadoop.yarn.api.records.LocalResourceType in project samza by apache.

the class LocalizerResourceMapper method buildResourceMapping.

private Map<String, LocalResource> buildResourceMapping() {
    ImmutableMap.Builder<String, LocalResource> localResourceMapBuilder = ImmutableMap.builder();
    List<String> resourceNames = resourceConfig.getResourceNames();
    for (String resourceName : resourceNames) {
        String resourceLocalName = resourceConfig.getResourceLocalName(resourceName);
        LocalResourceType resourceType = resourceConfig.getResourceLocalType(resourceName);
        LocalResourceVisibility resourceVisibility = resourceConfig.getResourceLocalVisibility(resourceName);
        Path resourcePath = resourceConfig.getResourcePath(resourceName);
        LocalResource localResource = createLocalResource(resourcePath, resourceType, resourceVisibility);
        localResourceMapBuilder.put(resourceLocalName, localResource);
        log.info("preparing local resource: {}", resourceLocalName);
    }
    return localResourceMapBuilder.build();
}
Also used : Path(org.apache.hadoop.fs.Path) LocalResourceType(org.apache.hadoop.yarn.api.records.LocalResourceType) ImmutableMap(com.google.common.collect.ImmutableMap) LocalResource(org.apache.hadoop.yarn.api.records.LocalResource) LocalResourceVisibility(org.apache.hadoop.yarn.api.records.LocalResourceVisibility)

Aggregations

ImmutableMap (com.google.common.collect.ImmutableMap)1 Path (org.apache.hadoop.fs.Path)1 LocalResource (org.apache.hadoop.yarn.api.records.LocalResource)1 LocalResourceType (org.apache.hadoop.yarn.api.records.LocalResourceType)1 LocalResourceVisibility (org.apache.hadoop.yarn.api.records.LocalResourceVisibility)1