Search in sources :

Example 1 with TarContainerPacker

use of org.apache.hadoop.ozone.container.keyvalue.TarContainerPacker in project ozone by apache.

the class TestDatanodeUpgradeToScmHA method importContainer.

/**
 * Imports the container found in {@code source} to the datanode with the ID
 * {@code containerID}.
 */
public void importContainer(long containerID, File source) throws Exception {
    DownloadAndImportReplicator replicator = new DownloadAndImportReplicator(dsm.getContainer().getContainerSet(), dsm.getContainer().getController(), new SimpleContainerDownloader(conf, null), new TarContainerPacker());
    File tempFile = tempFolder.newFile();
    Files.copy(source.toPath(), tempFile.toPath(), StandardCopyOption.REPLACE_EXISTING);
    replicator.importContainer(containerID, tempFile.toPath());
}
Also used : DownloadAndImportReplicator(org.apache.hadoop.ozone.container.replication.DownloadAndImportReplicator) SimpleContainerDownloader(org.apache.hadoop.ozone.container.replication.SimpleContainerDownloader) TarContainerPacker(org.apache.hadoop.ozone.container.keyvalue.TarContainerPacker) File(java.io.File)

Example 2 with TarContainerPacker

use of org.apache.hadoop.ozone.container.keyvalue.TarContainerPacker in project ozone by apache.

the class ClosedContainerReplicator method initializeReplicationSupervisor.

@NotNull
private void initializeReplicationSupervisor(ConfigurationSource conf) throws IOException {
    String fakeDatanodeUuid = datanode;
    if (fakeDatanodeUuid.isEmpty()) {
        fakeDatanodeUuid = UUID.randomUUID().toString();
    }
    ContainerSet containerSet = new ContainerSet();
    ContainerMetrics metrics = ContainerMetrics.create(conf);
    MutableVolumeSet volumeSet = new MutableVolumeSet(fakeDatanodeUuid, conf, null, StorageVolume.VolumeType.DATA_VOLUME, null);
    Map<ContainerType, Handler> handlers = new HashMap<>();
    for (ContainerType containerType : ContainerType.values()) {
        final Handler handler = Handler.getHandlerForContainerType(containerType, conf, fakeDatanodeUuid, containerSet, volumeSet, metrics, containerReplicaProto -> {
        });
        handler.setClusterID(UUID.randomUUID().toString());
        handlers.put(containerType, handler);
    }
    ContainerController controller = new ContainerController(containerSet, handlers);
    ContainerReplicator replicator = new DownloadAndImportReplicator(containerSet, controller, new SimpleContainerDownloader(conf, null), new TarContainerPacker());
    supervisor = new ReplicationSupervisor(containerSet, replicator, 10);
}
Also used : ReplicationSupervisor(org.apache.hadoop.ozone.container.replication.ReplicationSupervisor) HashMap(java.util.HashMap) DownloadAndImportReplicator(org.apache.hadoop.ozone.container.replication.DownloadAndImportReplicator) ContainerType(org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos.ContainerType) Handler(org.apache.hadoop.ozone.container.common.interfaces.Handler) ContainerController(org.apache.hadoop.ozone.container.ozoneimpl.ContainerController) SimpleContainerDownloader(org.apache.hadoop.ozone.container.replication.SimpleContainerDownloader) TarContainerPacker(org.apache.hadoop.ozone.container.keyvalue.TarContainerPacker) ContainerSet(org.apache.hadoop.ozone.container.common.impl.ContainerSet) MutableVolumeSet(org.apache.hadoop.ozone.container.common.volume.MutableVolumeSet) ContainerMetrics(org.apache.hadoop.ozone.container.common.helpers.ContainerMetrics) ContainerReplicator(org.apache.hadoop.ozone.container.replication.ContainerReplicator) NotNull(org.jetbrains.annotations.NotNull)

Aggregations

TarContainerPacker (org.apache.hadoop.ozone.container.keyvalue.TarContainerPacker)2 DownloadAndImportReplicator (org.apache.hadoop.ozone.container.replication.DownloadAndImportReplicator)2 SimpleContainerDownloader (org.apache.hadoop.ozone.container.replication.SimpleContainerDownloader)2 File (java.io.File)1 HashMap (java.util.HashMap)1 ContainerType (org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos.ContainerType)1 ContainerMetrics (org.apache.hadoop.ozone.container.common.helpers.ContainerMetrics)1 ContainerSet (org.apache.hadoop.ozone.container.common.impl.ContainerSet)1 Handler (org.apache.hadoop.ozone.container.common.interfaces.Handler)1 MutableVolumeSet (org.apache.hadoop.ozone.container.common.volume.MutableVolumeSet)1 ContainerController (org.apache.hadoop.ozone.container.ozoneimpl.ContainerController)1 ContainerReplicator (org.apache.hadoop.ozone.container.replication.ContainerReplicator)1 ReplicationSupervisor (org.apache.hadoop.ozone.container.replication.ReplicationSupervisor)1 NotNull (org.jetbrains.annotations.NotNull)1