Search in sources :

Example 6 with BackupOperationRequest

use of com.instaclustr.esop.impl.backup.BackupOperationRequest in project esop by instaclustr.

the class KubernetesAWSS3BackupRestoreTest method getBackupOperationRequest.

@Override
protected BackupOperationRequest getBackupOperationRequest() {
    final BackupOperationRequest backupOperationRequest = new BackupOperationRequest();
    backupOperationRequest.k8sSecretName = SIDECAR_SECRET_NAME;
    return backupOperationRequest;
}
Also used : BackupOperationRequest(com.instaclustr.esop.impl.backup.BackupOperationRequest)

Example 7 with BackupOperationRequest

use of com.instaclustr.esop.impl.backup.BackupOperationRequest in project esop by instaclustr.

the class AzureBackupRestoreTest method testDownload.

@Test
public void testDownload() throws Exception {
    AzureBucketService azureBucketService = new AzureBucketService(cloudStorageAccountFactory, getBackupOperationRequest());
    Path tmp = Files.createTempDirectory("tmp");
    tmp.toFile().deleteOnExit();
    try {
        azureBucketService.create(BUCKET_NAME);
        CloudBlobClient cloudBlobClient = cloudStorageAccountFactory.build(getBackupOperationRequest()).createCloudBlobClient();
        CloudBlobContainer container = cloudBlobClient.getContainerReference(BUCKET_NAME);
        CloudBlockBlob blob1 = container.getBlockBlobReference("cluster/dc/node/manifests/snapshot-name-" + BUCKET_NAME);
        blob1.uploadText("hello");
        CloudBlockBlob blob2 = container.getBlockBlobReference("snapshot/in/dir/name-" + BUCKET_NAME);
        blob2.uploadText("hello world");
        final RestoreOperationRequest restoreOperationRequest = new RestoreOperationRequest();
        restoreOperationRequest.storageLocation = new StorageLocation("azure://" + BUCKET_NAME + "/cluster/dc/node");
        final BackupOperationRequest backupOperationRequest = new BackupOperationRequest();
        backupOperationRequest.storageLocation = new StorageLocation("azure://" + BUCKET_NAME + "/cluster/dc/node");
        final AzureRestorer azureRestorer = new AzureRestorer(cloudStorageAccountFactory, restoreOperationRequest);
        final AzureBackuper azureBackuper = new AzureBackuper(cloudStorageAccountFactory, backupOperationRequest);
        // 1
        final Path downloadedFile = azureRestorer.downloadNodeFileToDir(tmp, Paths.get("manifests"), s -> s.contains("manifests/snapshot-name"));
        assertTrue(Files.exists(downloadedFile));
        // 2
        final String content = azureRestorer.downloadNodeFileToString(Paths.get("manifests"), s -> s.contains("manifests/snapshot-name"));
        Assert.assertEquals("hello", content);
        // 3
        final String content2 = azureRestorer.downloadFileToString(Paths.get("snapshot/in/dir"), s -> s.endsWith("name-" + BUCKET_NAME));
        Assert.assertEquals("hello world", content2);
        // 4
        azureRestorer.downloadFile(tmp.resolve("some-file"), azureRestorer.objectKeyToRemoteReference(Paths.get("snapshot/in/dir/name-" + BUCKET_NAME)));
        Assert.assertTrue(Files.exists(tmp.resolve("some-file")));
        Assert.assertEquals("hello world", new String(Files.readAllBytes(tmp.resolve("some-file"))));
        // backup
        azureBackuper.uploadText("hello world", azureBackuper.objectKeyToRemoteReference(Paths.get("topology/some-file-in-here.txt")));
        String text = azureRestorer.downloadFileToString(azureBackuper.objectKeyToRemoteReference(Paths.get("topology/some-file-in-here.txt")));
        Assert.assertEquals("hello world", text);
        String topology = azureRestorer.downloadFileToString(Paths.get("topology/some-file-in"), fileName -> fileName.contains("topology/some-file-in"));
        Assert.assertEquals("hello world", topology);
    } finally {
        azureBucketService.delete(BUCKET_NAME);
        deleteDirectory(Paths.get(target("commitlog_download_dir")));
        Files.deleteIfExists(tmp.resolve("some-file"));
    }
}
Also used : Path(java.nio.file.Path) AzureBucketService(com.instaclustr.esop.azure.AzureBucketService) CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) AzureRestorer(com.instaclustr.esop.azure.AzureRestorer) BackupOperationRequest(com.instaclustr.esop.impl.backup.BackupOperationRequest) AzureBackuper(com.instaclustr.esop.azure.AzureBackuper) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer) RestoreOperationRequest(com.instaclustr.esop.impl.restore.RestoreOperationRequest) CloudBlockBlob(com.microsoft.azure.storage.blob.CloudBlockBlob) StorageLocation(com.instaclustr.esop.impl.StorageLocation) Test(org.testng.annotations.Test)

Example 8 with BackupOperationRequest

use of com.instaclustr.esop.impl.backup.BackupOperationRequest in project esop by instaclustr.

the class KubernetesAzureBackupRestoreTest method getBackupOperationRequest.

@Override
protected BackupOperationRequest getBackupOperationRequest() {
    final BackupOperationRequest backupOperationRequest = new BackupOperationRequest();
    backupOperationRequest.k8sSecretName = SIDECAR_SECRET_NAME;
    return backupOperationRequest;
}
Also used : BackupOperationRequest(com.instaclustr.esop.impl.backup.BackupOperationRequest)

Example 9 with BackupOperationRequest

use of com.instaclustr.esop.impl.backup.BackupOperationRequest in project esop by instaclustr.

the class GoogleStorageBackupRestoreTest method testDownload.

@Test
public void testDownload() throws Exception {
    GCPBucketService gcpBucketService = new GCPBucketService(googleStorageFactory, getBackupOperationRequest());
    Path tmp = Files.createTempDirectory("tmp");
    tmp.toFile().deleteOnExit();
    try {
        gcpBucketService.create(BUCKET_NAME);
        Storage storage = googleStorageFactory.build(getBackupOperationRequest());
        Bucket bucket = storage.get(BUCKET_NAME);
        bucket.create("cluster/dc/node/manifests/snapshot-name-" + BUCKET_NAME, "hello".getBytes());
        bucket.create("snapshot/in/dir/name-" + BUCKET_NAME, "hello world".getBytes());
        final RestoreOperationRequest restoreOperationRequest = new RestoreOperationRequest();
        restoreOperationRequest.storageLocation = new StorageLocation("gcp://" + BUCKET_NAME + "/cluster/dc/node");
        final BackupOperationRequest backupOperationRequest = new BackupOperationRequest();
        backupOperationRequest.storageLocation = new StorageLocation("gcp://" + BUCKET_NAME + "/cluster/dc/node");
        final GCPRestorer gcpRestorer = new GCPRestorer(googleStorageFactory, restoreOperationRequest);
        final GCPBackuper gcpBackuper = new GCPBackuper(googleStorageFactory, backupOperationRequest);
        // 1
        final Path downloadedFile = gcpRestorer.downloadNodeFileToDir(tmp, Paths.get("manifests"), s -> s.contains("manifests/snapshot-name"));
        assertTrue(Files.exists(downloadedFile));
        // 2
        final String content = gcpRestorer.downloadNodeFileToString(Paths.get("manifests"), s -> s.contains("manifests/snapshot-name"));
        Assert.assertEquals("hello", content);
        // 3
        // final String content2 = gcpRestorer.downloadFileToString(Paths.get("snapshot/in/dir"), s -> s.endsWith("name-" + BUCKET_NAME));
        // Assert.assertEquals("hello world", content2);
        // 4
        gcpRestorer.downloadFile(tmp.resolve("some-file"), gcpRestorer.objectKeyToRemoteReference(Paths.get("snapshot/in/dir/name-" + BUCKET_NAME)));
        Assert.assertTrue(Files.exists(tmp.resolve("some-file")));
        Assert.assertEquals("hello world", new String(Files.readAllBytes(tmp.resolve("some-file"))));
        // backup
        gcpBackuper.uploadText("hello world", gcpBackuper.objectKeyToRemoteReference(Paths.get("topology/some-file-in-here.txt")));
        String topology = gcpRestorer.downloadFileToString(Paths.get("topology/some-file-in"), fileName -> fileName.contains("topology/some-file-in"));
        Assert.assertEquals("hello world", topology);
    } finally {
        gcpBucketService.delete(BUCKET_NAME);
        deleteDirectory(Paths.get(target("commitlog_download_dir")));
        Files.deleteIfExists(tmp.resolve("some-file"));
    }
}
Also used : Path(java.nio.file.Path) Storage(com.google.cloud.storage.Storage) BackupOperationRequest(com.instaclustr.esop.impl.backup.BackupOperationRequest) Bucket(com.google.cloud.storage.Bucket) GCPBucketService(com.instaclustr.esop.gcp.GCPBucketService) GCPRestorer(com.instaclustr.esop.gcp.GCPRestorer) RestoreOperationRequest(com.instaclustr.esop.impl.restore.RestoreOperationRequest) StorageLocation(com.instaclustr.esop.impl.StorageLocation) GCPBackuper(com.instaclustr.esop.gcp.GCPBackuper) Test(org.testng.annotations.Test)

Example 10 with BackupOperationRequest

use of com.instaclustr.esop.impl.backup.BackupOperationRequest in project esop by instaclustr.

the class KubernetesGoogleStorageBackupRestoreTest method getBackupOperationRequest.

@Override
protected BackupOperationRequest getBackupOperationRequest() {
    final BackupOperationRequest backupOperationRequest = new BackupOperationRequest();
    backupOperationRequest.k8sSecretName = SIDECAR_SECRET_NAME;
    return backupOperationRequest;
}
Also used : BackupOperationRequest(com.instaclustr.esop.impl.backup.BackupOperationRequest)

Aggregations

BackupOperationRequest (com.instaclustr.esop.impl.backup.BackupOperationRequest)10 Path (java.nio.file.Path)6 Test (org.testng.annotations.Test)5 StorageLocation (com.instaclustr.esop.impl.StorageLocation)4 RestoreOperationRequest (com.instaclustr.esop.impl.restore.RestoreOperationRequest)4 AmazonS3 (com.amazonaws.services.s3.AmazonS3)2 ManifestEntry (com.instaclustr.esop.impl.ManifestEntry)2 Snapshots (com.instaclustr.esop.impl.Snapshots)2 Snapshot (com.instaclustr.esop.impl.Snapshots.Snapshot)2 Backuper (com.instaclustr.esop.impl.backup.Backuper)2 UploadTracker (com.instaclustr.esop.impl.backup.UploadTracker)2 UploadUnit (com.instaclustr.esop.impl.backup.UploadTracker.UploadUnit)2 ClearSnapshotOperationRequest (com.instaclustr.esop.impl.backup.coordination.ClearSnapshotOperation.ClearSnapshotOperationRequest)2 TakeSnapshotOperationRequest (com.instaclustr.esop.impl.backup.coordination.TakeSnapshotOperation.TakeSnapshotOperationRequest)2 HashSpec (com.instaclustr.esop.impl.hash.HashSpec)2 CqlSession (com.datastax.oss.driver.api.core.CqlSession)1 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1 Cassandra (com.github.nosan.embedded.cassandra.Cassandra)1 Bucket (com.google.cloud.storage.Bucket)1 Storage (com.google.cloud.storage.Storage)1