Search in sources :

Example 1 with ContainerExit

use of com.spotify.docker.client.messages.ContainerExit in project helios by spotify.

the class JobExpirationTest method test.

@Test
public void test() throws Exception {
    startDefaultMaster();
    final HeliosClient client = defaultClient();
    startDefaultAgent(testHost());
    awaitHostStatus(client, testHost(), UP, LONG_WAIT_SECONDS, SECONDS);
    final JobId jobId = createJob(testJobName, testJobVersion, BUSYBOX, IDLE_COMMAND, DateTime.now().plusSeconds(10).toDate());
    deployJob(jobId, testHost());
    // Make sure the job runs
    final TaskStatus taskStatus = awaitJobState(client, testHost(), jobId, RUNNING, WAIT_TIMEOUT_SECONDS, SECONDS);
    // Then make sure it expires
    Polling.await(LONG_WAIT_SECONDS, SECONDS, new Callable<JobId>() {

        @Override
        public JobId call() throws Exception {
            if (client.jobs().get().containsKey(jobId)) {
                // job still exists, return null to continue polling
                return null;
            } else {
                // job no longer exists, return non-null to exit polling
                return jobId;
            }
        }
    });
    // Wait for the agent to kill the container
    final ContainerExit exit = docker.waitContainer(taskStatus.getContainerId());
    assertThat(exit.statusCode(), is(0));
}
Also used : ContainerExit(com.spotify.docker.client.messages.ContainerExit) HeliosClient(com.spotify.helios.client.HeliosClient) TaskStatus(com.spotify.helios.common.descriptors.TaskStatus) JobId(com.spotify.helios.common.descriptors.JobId) ExpectedException(org.junit.rules.ExpectedException) Test(org.junit.Test)

Example 2 with ContainerExit

use of com.spotify.docker.client.messages.ContainerExit in project helios by spotify.

the class SupervisorTest method verifySupervisorRestartsExitedContainer.

@Test
public void verifySupervisorRestartsExitedContainer() throws Exception {
    final String containerId1 = "deadbeef1";
    final String containerId2 = "deadbeef2";
    final ContainerCreation createResponse1 = ContainerCreation.builder().id(containerId1).build();
    final ContainerCreation createResponse2 = ContainerCreation.builder().id(containerId2).build();
    when(docker.createContainer(any(ContainerConfig.class), any(String.class))).thenReturn(createResponse1);
    final ImageInfo imageInfo = mock(ImageInfo.class);
    when(docker.inspectImage(IMAGE)).thenReturn(imageInfo);
    when(docker.inspectContainer(eq(containerId1))).thenReturn(runningResponse);
    final SettableFuture<ContainerExit> waitFuture1 = SettableFuture.create();
    final SettableFuture<ContainerExit> waitFuture2 = SettableFuture.create();
    when(docker.waitContainer(containerId1)).thenAnswer(futureAnswer(waitFuture1));
    when(docker.waitContainer(containerId2)).thenAnswer(futureAnswer(waitFuture2));
    // Start the job
    sut.setGoal(START);
    verify(docker, timeout(30000)).createContainer(any(ContainerConfig.class), any(String.class));
    verify(docker, timeout(30000)).startContainer(eq(containerId1));
    verify(docker, timeout(30000)).waitContainer(containerId1);
    // Indicate that the container exited
    when(docker.inspectContainer(eq(containerId1))).thenReturn(stoppedResponse);
    when(docker.createContainer(any(ContainerConfig.class), any(String.class))).thenReturn(createResponse2);
    when(docker.inspectContainer(eq(containerId2))).thenReturn(runningResponse);
    waitFuture1.set(ContainerExit.create(1));
    // Verify that the container was restarted
    verify(docker, timeout(30000)).createContainer(any(ContainerConfig.class), any(String.class));
    verify(docker, timeout(30000)).startContainer(eq(containerId2));
    verify(docker, timeout(30000)).waitContainer(containerId2);
}
Also used : ContainerConfig(com.spotify.docker.client.messages.ContainerConfig) ContainerCreation(com.spotify.docker.client.messages.ContainerCreation) ContainerExit(com.spotify.docker.client.messages.ContainerExit) ImageInfo(com.spotify.docker.client.messages.ImageInfo) Test(org.junit.Test)

Example 3 with ContainerExit

use of com.spotify.docker.client.messages.ContainerExit in project helios by spotify.

the class HeliosSoloDeployment method checkDockerAndGetGateway.

/**
   * Checks that the local Docker daemon is reachable from inside a container.
   * This method also gets the gateway IP address for this HeliosSoloDeployment.
   *
   * @return The gateway IP address of the gateway probe container.
   * @throws HeliosDeploymentException if we can't deploy the probe container or can't reach
   *         Docker daemon's API from inside the container.
   */
private String checkDockerAndGetGateway() throws HeliosDeploymentException {
    final String probeName = randomString();
    final HostConfig hostConfig = HostConfig.builder().binds(binds).build();
    final ContainerConfig containerConfig = ContainerConfig.builder().env(env).hostConfig(hostConfig).image(PROBE_IMAGE).cmd(probeCommand(probeName)).build();
    final ContainerCreation creation;
    try {
        pullIfAbsent(PROBE_IMAGE);
        creation = dockerClient.createContainer(containerConfig, probeName);
    } catch (DockerException | InterruptedException e) {
        throw new HeliosDeploymentException("helios-solo probe container creation failed", e);
    }
    final ContainerExit exit;
    final String gateway;
    try {
        dockerClient.startContainer(creation.id());
        gateway = dockerClient.inspectContainer(creation.id()).networkSettings().gateway();
        exit = dockerClient.waitContainer(creation.id());
    } catch (DockerException | InterruptedException e) {
        killContainer(creation.id());
        throw new HeliosDeploymentException("helios-solo probe container failed", e);
    } finally {
        removeContainer(creation.id());
    }
    if (exit.statusCode() != 0) {
        throw new HeliosDeploymentException(String.format("Docker was not reachable (curl exit status %d) using DOCKER_HOST=%s and " + "DOCKER_CERT_PATH=%s from within a container. Please ensure that " + "DOCKER_HOST contains a full hostname or IP address, not localhost, " + "127.0.0.1, etc.", exit.statusCode(), containerDockerHost.bindUri(), containerDockerHost.dockerCertPath()));
    }
    return gateway;
}
Also used : ContainerConfig(com.spotify.docker.client.messages.ContainerConfig) DockerException(com.spotify.docker.client.exceptions.DockerException) ContainerCreation(com.spotify.docker.client.messages.ContainerCreation) HostConfig(com.spotify.docker.client.messages.HostConfig) ContainerExit(com.spotify.docker.client.messages.ContainerExit)

Example 4 with ContainerExit

use of com.spotify.docker.client.messages.ContainerExit in project helios by spotify.

the class TaskRunner method run0.

private int run0() throws InterruptedException, DockerException {
    // Delay
    Thread.sleep(delayMillis);
    // Check if the container is already running
    final ContainerInfo info = getContainerInfo(existingContainerId);
    final String containerId;
    if (info != null && info.state().running()) {
        containerId = existingContainerId;
        this.containerId = Optional.of(existingContainerId);
    } else {
        // Create and start container if necessary
        containerId = createAndStartContainer();
        this.containerId = Optional.of(containerId);
        if (healthChecker.isPresent()) {
            listener.healthChecking();
            final RetryScheduler retryScheduler = BoundedRandomExponentialBackoff.newBuilder().setMinIntervalMillis(SECONDS.toMillis(1)).setMaxIntervalMillis(SECONDS.toMillis(30)).build().newScheduler();
            while (!healthChecker.get().check(containerId)) {
                final ContainerState state = getContainerState(containerId);
                if (state == null) {
                    final String err = "container " + containerId + " was not found during health " + "checking, or has no State object";
                    log.warn(err);
                    throw new RuntimeException(err);
                }
                if (!state.running()) {
                    final String err = "container " + containerId + " exited during health checking. " + "Exit code: " + state.exitCode() + ", Config: " + config;
                    log.warn(err);
                    throw new RuntimeException(err);
                }
                final long retryMillis = retryScheduler.nextMillis();
                log.warn("container failed healthcheck, will retry in {}ms: {}: {}", retryMillis, config, containerId);
                Thread.sleep(retryMillis);
            }
            log.info("healthchecking complete of containerId={} taskConfig={}", containerId, config);
        } else {
            log.info("no healthchecks configured for containerId={} taskConfig={}", containerId, config);
        }
    }
    listener.running();
    // Register and wait for container to exit
    serviceRegistrationHandle = Optional.fromNullable(registrar.register(config.registration()));
    final ContainerExit exit;
    try {
        exit = docker.waitContainer(containerId);
    } finally {
        unregister();
        this.containerId = Optional.absent();
    }
    log.info("container exited: {}: {}: {}", config, containerId, exit.statusCode());
    listener.exited(exit.statusCode());
    return exit.statusCode();
}
Also used : HeliosRuntimeException(com.spotify.helios.common.HeliosRuntimeException) ContainerInfo(com.spotify.docker.client.messages.ContainerInfo) ContainerExit(com.spotify.docker.client.messages.ContainerExit) ContainerState(com.spotify.docker.client.messages.ContainerState)

Example 5 with ContainerExit

use of com.spotify.docker.client.messages.ContainerExit in project helios by spotify.

the class ReapingTest method test.

@Test
public void test() throws Exception {
    startDefaultMaster();
    final String id = "test-" + toHexString(new SecureRandom().nextInt());
    final String namespace = "helios-" + id;
    final String intruder1 = intruder(namespace);
    final String intruder2 = intruder(namespace);
    // Start a container in the agent namespace
    startContainer(intruder1);
    // Start agent
    final HeliosClient client = defaultClient();
    startDefaultAgent(testHost(), "--id=" + id);
    awaitHostRegistered(client, testHost(), LONG_WAIT_SECONDS, SECONDS);
    awaitHostStatus(client, testHost(), UP, LONG_WAIT_SECONDS, SECONDS);
    // With LXC, killing a container results in exit code 0.
    // In docker 1.5 killing a container results in exit code 137, in previous versions it's -1.
    final List<Integer> expectedExitCodes = docker.info().executionDriver().startsWith("lxc-") ? Collections.singletonList(0) : asList(-1, 137);
    // Wait for the agent to kill the container
    final ContainerExit exit1 = docker.waitContainer(intruder1);
    assertThat(exit1.statusCode(), isIn(expectedExitCodes));
    // Start another container in the agent namespace
    startContainer(intruder2);
    // Wait for the agent to kill the second container as well
    final ContainerExit exit2 = docker.waitContainer(intruder2);
    assertThat(exit2.statusCode(), isIn(expectedExitCodes));
}
Also used : SecureRandom(java.security.SecureRandom) ContainerExit(com.spotify.docker.client.messages.ContainerExit) Integer.toHexString(java.lang.Integer.toHexString) HeliosClient(com.spotify.helios.client.HeliosClient) Test(org.junit.Test)

Aggregations

ContainerExit (com.spotify.docker.client.messages.ContainerExit)8 Test (org.junit.Test)6 ContainerConfig (com.spotify.docker.client.messages.ContainerConfig)5 ImageInfo (com.spotify.docker.client.messages.ImageInfo)4 ContainerCreation (com.spotify.docker.client.messages.ContainerCreation)3 HeliosClient (com.spotify.helios.client.HeliosClient)2 DockerException (com.spotify.docker.client.exceptions.DockerException)1 ContainerInfo (com.spotify.docker.client.messages.ContainerInfo)1 ContainerState (com.spotify.docker.client.messages.ContainerState)1 HostConfig (com.spotify.docker.client.messages.HostConfig)1 HeliosRuntimeException (com.spotify.helios.common.HeliosRuntimeException)1 Job (com.spotify.helios.common.descriptors.Job)1 JobId (com.spotify.helios.common.descriptors.JobId)1 TaskStatus (com.spotify.helios.common.descriptors.TaskStatus)1 Integer.toHexString (java.lang.Integer.toHexString)1 SecureRandom (java.security.SecureRandom)1 ExpectedException (org.junit.rules.ExpectedException)1