Search in sources :

Example 1 with WeaveRunnableSpecification

use of com.continuuity.weave.api.WeaveRunnableSpecification in project weave by continuuity.

the class WeaveContainerMain method main.

/**
   * Main method for launching a {@link WeaveContainerService} which runs
   * a {@link com.continuuity.weave.api.WeaveRunnable}.
   */
public static void main(final String[] args) throws Exception {
    // Try to load the secure store from localized file, which AM requested RM to localize it for this container.
    loadSecureStore();
    String zkConnectStr = System.getenv(EnvKeys.WEAVE_ZK_CONNECT);
    File weaveSpecFile = new File(Constants.Files.WEAVE_SPEC);
    RunId appRunId = RunIds.fromString(System.getenv(EnvKeys.WEAVE_APP_RUN_ID));
    RunId runId = RunIds.fromString(System.getenv(EnvKeys.WEAVE_RUN_ID));
    String runnableName = System.getenv(EnvKeys.WEAVE_RUNNABLE_NAME);
    int instanceId = Integer.parseInt(System.getenv(EnvKeys.WEAVE_INSTANCE_ID));
    int instanceCount = Integer.parseInt(System.getenv(EnvKeys.WEAVE_INSTANCE_COUNT));
    ZKClientService zkClientService = ZKClientServices.delegate(ZKClients.reWatchOnExpire(ZKClients.retryOnFailure(ZKClientService.Builder.of(zkConnectStr).build(), RetryStrategies.fixDelay(1, TimeUnit.SECONDS))));
    DiscoveryService discoveryService = new ZKDiscoveryService(zkClientService);
    WeaveSpecification weaveSpec = loadWeaveSpec(weaveSpecFile);
    renameLocalFiles(weaveSpec.getRunnables().get(runnableName));
    WeaveRunnableSpecification runnableSpec = weaveSpec.getRunnables().get(runnableName).getRunnableSpecification();
    ContainerInfo containerInfo = new EnvContainerInfo();
    Arguments arguments = decodeArgs();
    BasicWeaveContext context = new BasicWeaveContext(runId, appRunId, containerInfo.getHost(), arguments.getRunnableArguments().get(runnableName).toArray(new String[0]), arguments.getArguments().toArray(new String[0]), runnableSpec, instanceId, discoveryService, instanceCount, containerInfo.getMemoryMB(), containerInfo.getVirtualCores());
    Configuration conf = new YarnConfiguration(new HdfsConfiguration(new Configuration()));
    Service service = new WeaveContainerService(context, containerInfo, getContainerZKClient(zkClientService, appRunId, runnableName), runId, runnableSpec, getClassLoader(), createAppLocation(conf));
    new WeaveContainerMain().doMain(zkClientService, service);
}
Also used : YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) Configuration(org.apache.hadoop.conf.Configuration) HdfsConfiguration(org.apache.hadoop.hdfs.HdfsConfiguration) EnvContainerInfo(com.continuuity.weave.internal.EnvContainerInfo) Arguments(com.continuuity.weave.internal.Arguments) DiscoveryService(com.continuuity.weave.discovery.DiscoveryService) ZKDiscoveryService(com.continuuity.weave.discovery.ZKDiscoveryService) ZKClientService(com.continuuity.weave.zookeeper.ZKClientService) Service(com.google.common.util.concurrent.Service) WeaveSpecification(com.continuuity.weave.api.WeaveSpecification) HdfsConfiguration(org.apache.hadoop.hdfs.HdfsConfiguration) WeaveRunnableSpecification(com.continuuity.weave.api.WeaveRunnableSpecification) ZKDiscoveryService(com.continuuity.weave.discovery.ZKDiscoveryService) ZKClientService(com.continuuity.weave.zookeeper.ZKClientService) YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) EnvContainerInfo(com.continuuity.weave.internal.EnvContainerInfo) ContainerInfo(com.continuuity.weave.internal.ContainerInfo) BasicWeaveContext(com.continuuity.weave.internal.BasicWeaveContext) RunId(com.continuuity.weave.api.RunId) DiscoveryService(com.continuuity.weave.discovery.DiscoveryService) ZKDiscoveryService(com.continuuity.weave.discovery.ZKDiscoveryService) LocalFile(com.continuuity.weave.api.LocalFile) File(java.io.File)

Example 2 with WeaveRunnableSpecification

use of com.continuuity.weave.api.WeaveRunnableSpecification in project weave by continuuity.

the class RuntimeSpecificationCodec method deserialize.

@Override
public RuntimeSpecification deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
    JsonObject jsonObj = json.getAsJsonObject();
    String name = jsonObj.get("name").getAsString();
    WeaveRunnableSpecification runnable = context.deserialize(jsonObj.get("runnable"), WeaveRunnableSpecification.class);
    ResourceSpecification resources = context.deserialize(jsonObj.get("resources"), ResourceSpecification.class);
    Collection<LocalFile> files = context.deserialize(jsonObj.get("files"), new TypeToken<Collection<LocalFile>>() {
    }.getType());
    return new DefaultRuntimeSpecification(name, runnable, resources, files);
}
Also used : LocalFile(com.continuuity.weave.api.LocalFile) DefaultRuntimeSpecification(com.continuuity.weave.internal.DefaultRuntimeSpecification) TypeToken(com.google.common.reflect.TypeToken) JsonObject(com.google.gson.JsonObject) ResourceSpecification(com.continuuity.weave.api.ResourceSpecification) WeaveRunnableSpecification(com.continuuity.weave.api.WeaveRunnableSpecification)

Aggregations

LocalFile (com.continuuity.weave.api.LocalFile)2 WeaveRunnableSpecification (com.continuuity.weave.api.WeaveRunnableSpecification)2 ResourceSpecification (com.continuuity.weave.api.ResourceSpecification)1 RunId (com.continuuity.weave.api.RunId)1 WeaveSpecification (com.continuuity.weave.api.WeaveSpecification)1 DiscoveryService (com.continuuity.weave.discovery.DiscoveryService)1 ZKDiscoveryService (com.continuuity.weave.discovery.ZKDiscoveryService)1 Arguments (com.continuuity.weave.internal.Arguments)1 BasicWeaveContext (com.continuuity.weave.internal.BasicWeaveContext)1 ContainerInfo (com.continuuity.weave.internal.ContainerInfo)1 DefaultRuntimeSpecification (com.continuuity.weave.internal.DefaultRuntimeSpecification)1 EnvContainerInfo (com.continuuity.weave.internal.EnvContainerInfo)1 ZKClientService (com.continuuity.weave.zookeeper.ZKClientService)1 TypeToken (com.google.common.reflect.TypeToken)1 Service (com.google.common.util.concurrent.Service)1 JsonObject (com.google.gson.JsonObject)1 File (java.io.File)1 Configuration (org.apache.hadoop.conf.Configuration)1 HdfsConfiguration (org.apache.hadoop.hdfs.HdfsConfiguration)1 YarnConfiguration (org.apache.hadoop.yarn.conf.YarnConfiguration)1