Search in sources :

Example 1 with OpenKeySession

use of org.apache.hadoop.ozone.om.helpers.OpenKeySession in project ozone by apache.

the class RpcClient method createKey.

@Override
public OzoneOutputStream createKey(String volumeName, String bucketName, String keyName, long size, ReplicationConfig replicationConfig, Map<String, String> metadata) throws IOException {
    verifyVolumeName(volumeName);
    verifyBucketName(bucketName);
    if (checkKeyNameEnabled) {
        HddsClientUtils.verifyKeyName(keyName);
    }
    HddsClientUtils.checkNotNull(keyName, replicationConfig);
    String requestId = UUID.randomUUID().toString();
    OmKeyArgs.Builder builder = new OmKeyArgs.Builder().setVolumeName(volumeName).setBucketName(bucketName).setKeyName(keyName).setDataSize(size).setReplicationConfig(replicationConfig).addAllMetadata(metadata).setAcls(getAclList()).setLatestVersionLocation(getLatestVersionLocation);
    if (Boolean.parseBoolean(metadata.get(OzoneConsts.GDPR_FLAG))) {
        try {
            GDPRSymmetricKey gKey = new GDPRSymmetricKey(new SecureRandom());
            builder.addAllMetadata(gKey.getKeyDetails());
        } catch (Exception e) {
            if (e instanceof InvalidKeyException && e.getMessage().contains("Illegal key size or default parameters")) {
                LOG.error("Missing Unlimited Strength Policy jars. Please install " + "Java Cryptography Extension (JCE) Unlimited Strength " + "Jurisdiction Policy Files");
            }
            throw new IOException(e);
        }
    }
    OpenKeySession openKey = ozoneManagerClient.openKey(builder.build());
    return createOutputStream(openKey, requestId, replicationConfig);
}
Also used : GDPRSymmetricKey(org.apache.hadoop.ozone.security.GDPRSymmetricKey) CacheBuilder(com.google.common.cache.CacheBuilder) SecureRandom(java.security.SecureRandom) IOException(java.io.IOException) OpenKeySession(org.apache.hadoop.ozone.om.helpers.OpenKeySession) InvalidKeyException(java.security.InvalidKeyException) OmKeyArgs(org.apache.hadoop.ozone.om.helpers.OmKeyArgs) InvalidKeyException(java.security.InvalidKeyException) IOException(java.io.IOException) OMException(org.apache.hadoop.ozone.om.exceptions.OMException)

Example 2 with OpenKeySession

use of org.apache.hadoop.ozone.om.helpers.OpenKeySession in project ozone by apache.

the class RpcClient method createMultipartKey.

@Override
public OzoneOutputStream createMultipartKey(String volumeName, String bucketName, String keyName, long size, int partNumber, String uploadID) throws IOException {
    verifyVolumeName(volumeName);
    verifyBucketName(bucketName);
    if (checkKeyNameEnabled) {
        HddsClientUtils.verifyKeyName(keyName);
    }
    HddsClientUtils.checkNotNull(keyName, uploadID);
    Preconditions.checkArgument(partNumber > 0 && partNumber <= 10000, "Part " + "number should be greater than zero and less than or equal to 10000");
    Preconditions.checkArgument(size >= 0, "size should be greater than or " + "equal to zero");
    String requestId = UUID.randomUUID().toString();
    OmKeyArgs keyArgs = new OmKeyArgs.Builder().setVolumeName(volumeName).setBucketName(bucketName).setKeyName(keyName).setDataSize(size).setIsMultipartKey(true).setMultipartUploadID(uploadID).setMultipartUploadPartNumber(partNumber).setAcls(getAclList()).build();
    OpenKeySession openKey = ozoneManagerClient.openKey(keyArgs);
    KeyOutputStream keyOutputStream = new KeyOutputStream.Builder().setHandler(openKey).setXceiverClientManager(xceiverClientManager).setOmClient(ozoneManagerClient).setRequestID(requestId).setReplicationConfig(openKey.getKeyInfo().getReplicationConfig()).setMultipartNumber(partNumber).setMultipartUploadID(uploadID).setIsMultipartKey(true).enableUnsafeByteBufferConversion(unsafeByteBufferConversion).setConfig(clientConfig).build();
    keyOutputStream.addPreallocateBlocks(openKey.getKeyInfo().getLatestVersionLocations(), openKey.getOpenVersion());
    FileEncryptionInfo feInfo = openKey.getKeyInfo().getFileEncryptionInfo();
    if (feInfo != null) {
        KeyProvider.KeyVersion decrypted = getDEK(feInfo);
        final CryptoOutputStream cryptoOut = new CryptoOutputStream(keyOutputStream, OzoneKMSUtil.getCryptoCodec(conf, feInfo), decrypted.getMaterial(), feInfo.getIV());
        return new OzoneOutputStream(cryptoOut);
    } else {
        return new OzoneOutputStream(keyOutputStream);
    }
}
Also used : KeyProvider(org.apache.hadoop.crypto.key.KeyProvider) CryptoOutputStream(org.apache.hadoop.crypto.CryptoOutputStream) OzoneOutputStream(org.apache.hadoop.ozone.client.io.OzoneOutputStream) OpenKeySession(org.apache.hadoop.ozone.om.helpers.OpenKeySession) KeyOutputStream(org.apache.hadoop.ozone.client.io.KeyOutputStream) FileEncryptionInfo(org.apache.hadoop.fs.FileEncryptionInfo) OmKeyArgs(org.apache.hadoop.ozone.om.helpers.OmKeyArgs)

Example 3 with OpenKeySession

use of org.apache.hadoop.ozone.om.helpers.OpenKeySession in project ozone by apache.

the class RpcClient method createFile.

@Override
public OzoneOutputStream createFile(String volumeName, String bucketName, String keyName, long size, ReplicationConfig replicationConfig, boolean overWrite, boolean recursive) throws IOException {
    OmKeyArgs keyArgs = new OmKeyArgs.Builder().setVolumeName(volumeName).setBucketName(bucketName).setKeyName(keyName).setDataSize(size).setReplicationConfig(replicationConfig).setAcls(getAclList()).setLatestVersionLocation(getLatestVersionLocation).build();
    OpenKeySession keySession = ozoneManagerClient.createFile(keyArgs, overWrite, recursive);
    return createOutputStream(keySession, UUID.randomUUID().toString(), replicationConfig);
}
Also used : CacheBuilder(com.google.common.cache.CacheBuilder) OpenKeySession(org.apache.hadoop.ozone.om.helpers.OpenKeySession) OmKeyArgs(org.apache.hadoop.ozone.om.helpers.OmKeyArgs)

Example 4 with OpenKeySession

use of org.apache.hadoop.ozone.om.helpers.OpenKeySession in project ozone by apache.

the class TestOmMetrics method doKeyOps.

/**
 * Test key operations with ignoring thrown exception.
 */
private void doKeyOps(OmKeyArgs keyArgs) {
    OpenKeySession keySession = null;
    try {
        keySession = writeClient.openKey(keyArgs);
    } catch (IOException ignored) {
    }
    try {
        long id = (keySession != null) ? keySession.getId() : 0;
        writeClient.commitKey(keyArgs, id);
    } catch (IOException ignored) {
    }
    try {
        ozoneManager.lookupKey(keyArgs);
    } catch (IOException ignored) {
    }
    try {
        ozoneManager.listKeys(keyArgs.getVolumeName(), keyArgs.getBucketName(), null, null, 0);
    } catch (IOException ignored) {
    }
    try {
        ozoneManager.listTrash(keyArgs.getVolumeName(), keyArgs.getBucketName(), null, null, 0);
    } catch (IOException ignored) {
    }
    try {
        writeClient.deleteKey(keyArgs);
    } catch (IOException ignored) {
    }
    try {
        writeClient.initiateMultipartUpload(keyArgs);
    } catch (IOException ignored) {
    }
}
Also used : OpenKeySession(org.apache.hadoop.ozone.om.helpers.OpenKeySession) IOException(java.io.IOException)

Example 5 with OpenKeySession

use of org.apache.hadoop.ozone.om.helpers.OpenKeySession in project ozone by apache.

the class TestKeyManagerImpl method testCreateDirectory.

@Test
public void testCreateDirectory() throws IOException {
    // Create directory where the parent directory does not exist
    StringBuffer keyNameBuf = new StringBuffer();
    keyNameBuf.append(RandomStringUtils.randomAlphabetic(5));
    OmKeyArgs keyArgs = createBuilder().setKeyName(keyNameBuf.toString()).build();
    for (int i = 0; i < 5; i++) {
        keyNameBuf.append("/").append(RandomStringUtils.randomAlphabetic(5));
    }
    String keyName = keyNameBuf.toString();
    writeClient.createDirectory(keyArgs);
    Path path = Paths.get(keyName);
    while (path != null) {
        // verify parent directories are created
        Assert.assertTrue(keyManager.getFileStatus(keyArgs).isDirectory());
        path = path.getParent();
    }
    // make sure create directory fails where parent is a file
    keyName = RandomStringUtils.randomAlphabetic(5);
    keyArgs = createBuilder().setKeyName(keyName).build();
    OpenKeySession keySession = writeClient.openKey(keyArgs);
    keyArgs.setLocationInfoList(keySession.getKeyInfo().getLatestVersionLocations().getLocationList());
    writeClient.commitKey(keyArgs, keySession.getId());
    try {
        writeClient.createDirectory(keyArgs);
        Assert.fail("Creation should fail for directory.");
    } catch (OMException e) {
        Assert.assertEquals(e.getResult(), OMException.ResultCodes.FILE_ALREADY_EXISTS);
    }
    // create directory where parent is root
    keyName = RandomStringUtils.randomAlphabetic(5);
    keyArgs = createBuilder().setKeyName(keyName).build();
    writeClient.createDirectory(keyArgs);
    OzoneFileStatus fileStatus = keyManager.getFileStatus(keyArgs);
    Assert.assertTrue(fileStatus.isDirectory());
    Assert.assertTrue(fileStatus.getKeyInfo().getKeyLocationVersions().get(0).getLocationList().isEmpty());
}
Also used : Path(java.nio.file.Path) OpenKeySession(org.apache.hadoop.ozone.om.helpers.OpenKeySession) OmKeyArgs(org.apache.hadoop.ozone.om.helpers.OmKeyArgs) OMException(org.apache.hadoop.ozone.om.exceptions.OMException) OzoneFileStatus(org.apache.hadoop.ozone.om.helpers.OzoneFileStatus) Test(org.junit.Test)

Aggregations

OpenKeySession (org.apache.hadoop.ozone.om.helpers.OpenKeySession)23 OmKeyArgs (org.apache.hadoop.ozone.om.helpers.OmKeyArgs)22 Test (org.junit.Test)10 OMException (org.apache.hadoop.ozone.om.exceptions.OMException)6 ArrayList (java.util.ArrayList)5 OmKeyInfo (org.apache.hadoop.ozone.om.helpers.OmKeyInfo)4 IOException (java.io.IOException)3 OmKeyLocationInfo (org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo)3 OzoneFileStatus (org.apache.hadoop.ozone.om.helpers.OzoneFileStatus)3 CacheBuilder (com.google.common.cache.CacheBuilder)2 BlockID (org.apache.hadoop.hdds.client.BlockID)2 DatanodeDetails (org.apache.hadoop.hdds.protocol.DatanodeDetails)2 ContainerInfo (org.apache.hadoop.hdds.scm.container.ContainerInfo)2 ContainerWithPipeline (org.apache.hadoop.hdds.scm.container.common.helpers.ContainerWithPipeline)2 ExcludeList (org.apache.hadoop.hdds.scm.container.common.helpers.ExcludeList)2 Pipeline (org.apache.hadoop.hdds.scm.pipeline.Pipeline)2 DeleteKeyArgs (org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.DeleteKeyArgs)2 KeyArgs (org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs)2 OMRequest (org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.OMRequest)2 Path (java.nio.file.Path)1