Examples with OSDClient com.pspace.ifs.ksan.gw.object.osdclient.OSDClient used on opensource projects

Search in sources :

Example 1 with OSDClient

use of com.pspace.ifs.ksan.gw.object.osdclient.OSDClient in project ksan by infinistor.

the class S3ObjectOperation method deletePart.

public void deletePart(String diskID) throws Exception {
    String host = GWDiskConfig.getInstance().getOSDIP(diskID);
    if (host == null) {
        logger.error(GWConstants.LOG_S3OBJECT_OPERATION_DISK_IP_NULL, diskID);
        return;
    }
    String path = GWDiskConfig.getInstance().getPath(diskID);
    if (path == null) {
        logger.error(GWConstants.LOG_S3OBJECT_OPERATION_DISK_PATH_NULL, diskID);
        return;
    }
    if (GWUtils.getLocalIP().equals(host)) {
        File tmpFile = new File(makeTempPath(path, objMeta.getObjId(), s3Parameter.getPartNumber()));
        tmpFile.delete();
    } else {
        OSDClient client = OSDClientManager.getInstance().getOSDClient(host);
        client.deletePart(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), s3Parameter.getPartNumber());
        OSDClientManager.getInstance().returnOSDClient(client);
    }
}
Also used : OSDClient(com.pspace.ifs.ksan.gw.object.osdclient.OSDClient) File(java.io.File)

Example 2 with OSDClient

use of com.pspace.ifs.ksan.gw.object.osdclient.OSDClient in project ksan by infinistor.

the class S3ObjectOperation method putObjectNormal.

private S3Object putObjectNormal(long length, InputStream is) throws GWException {
    S3Object s3Object = new S3Object();
    File filePrimary = null;
    File tmpFilePrimary = null;
    FileOutputStream fosPrimary = null;
    File fileReplica = null;
    File tmpFileReplica = null;
    FileOutputStream fosReplica = null;
    File trashPrimary = null;
    File trashReplica = null;
    OSDClient clientPrimary = null;
    OSDClient clientReplica = null;
    long totalReads = 0L;
    long existFileSize = 0L;
    long putSize = 0L;
    long calSize = 0L;
    try {
        MessageDigest md5er = MessageDigest.getInstance(GWConstants.MD5);
        byte[] buffer = new byte[GWConstants.MAXBUFSIZE];
        int readLength = 0;
        existFileSize = objMeta.getSize();
        putSize = length;
        boolean isPrimaryCache = false;
        boolean isReplicaCache = false;
        logger.debug("performance mode : {}", GWConfig.getPerformanceMode());
        logger.debug("objMeta - replicaCount : {}", objMeta.getReplicaCount());
        // No option
        if (GWConfig.isNoOption()) {
            if (objMeta.getReplicaCount() > 1) {
                existFileSize *= objMeta.getReplicaCount();
                putSize *= objMeta.getReplicaCount();
                logger.debug("bucket : {}, object : {}", objMeta.getBucket(), objMeta.getPath());
                logger.debug("primary disk id : {}, osd ip : {}", objMeta.getPrimaryDisk().getId(), objMeta.getPrimaryDisk().getOsdIp());
                logger.debug("replica disk id : {}, osd ip : {}", objMeta.getReplicaDisk().getId(), objMeta.getReplicaDisk().getOsdIp());
                if (GWUtils.getLocalIP().equals(objMeta.getPrimaryDisk().getOsdIp())) {
                    if (!Strings.isNullOrEmpty(GWConfig.getCacheDisk()) && length <= (GWConfig.getCacheFileSize() * GWConstants.MEGABYTES)) {
                        filePrimary = new File(makeCachePath(makeObjPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId)));
                        tmpFilePrimary = new File(makeCachePath(makeTempPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId)));
                        trashPrimary = new File(makeTrashPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
                        File file = new File(makeObjPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
                        com.google.common.io.Files.createParentDirs(file);
                        com.google.common.io.Files.createParentDirs(filePrimary);
                        com.google.common.io.Files.createParentDirs(tmpFilePrimary);
                        logger.debug("filePrimary path : {}", filePrimary.getAbsolutePath());
                        logger.debug("tmpFilePrimary path : {}", tmpFilePrimary.getAbsolutePath());
                        fosPrimary = new FileOutputStream(tmpFilePrimary, false);
                        isPrimaryCache = true;
                    } else {
                        filePrimary = new File(makeObjPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
                        tmpFilePrimary = new File(makeTempPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
                        trashPrimary = new File(makeTrashPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
                        com.google.common.io.Files.createParentDirs(filePrimary);
                        com.google.common.io.Files.createParentDirs(tmpFilePrimary);
                        fosPrimary = new FileOutputStream(tmpFilePrimary, false);
                    }
                } else {
                    clientPrimary = OSDClientManager.getInstance().getOSDClient(objMeta.getPrimaryDisk().getOsdIp());
                    clientPrimary.putInit(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId, length, GWConstants.FILE_ATTRUBUTE_REPLICATION_PRIMARY, objMeta.getReplicaDisk().getId(), GWConfig.getPerformanceMode());
                }
                if (GWUtils.getLocalIP().equals(objMeta.getReplicaDisk().getOsdIp())) {
                    if (!Strings.isNullOrEmpty(GWConfig.getCacheDisk()) && length <= GWConfig.getCacheFileSize() * GWConstants.MEGABYTES) {
                        fileReplica = new File(makeCachePath(makeObjPath(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId)));
                        tmpFileReplica = new File(makeCachePath(makeTempPath(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId)));
                        trashReplica = new File(makeCachePath(makeTrashPath(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId)));
                        File file = new File(makeObjPath(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId));
                        com.google.common.io.Files.createParentDirs(file);
                        com.google.common.io.Files.createParentDirs(fileReplica);
                        com.google.common.io.Files.createParentDirs(tmpFileReplica);
                        logger.debug("fileReplica path : {}", fileReplica.getAbsolutePath());
                        logger.debug("tmpFileReplica path : {}", tmpFileReplica.getAbsolutePath());
                        fosReplica = new FileOutputStream(tmpFileReplica, false);
                        isReplicaCache = true;
                    } else {
                        fileReplica = new File(makeObjPath(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId));
                        tmpFileReplica = new File(makeTempPath(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId));
                        trashReplica = new File(makeTrashPath(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId));
                        com.google.common.io.Files.createParentDirs(fileReplica);
                        com.google.common.io.Files.createParentDirs(tmpFileReplica);
                        fosReplica = new FileOutputStream(tmpFileReplica, false);
                    }
                } else {
                    clientReplica = OSDClientManager.getInstance().getOSDClient(objMeta.getReplicaDisk().getOsdIp());
                    clientReplica.putInit(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId, length, GWConstants.FILE_ATTRIBUTE_REPLICATION_REPLICA, GWConstants.FILE_ATTRIBUTE_REPLICA_DISK_ID_NULL, GWConfig.getPerformanceMode());
                }
                while ((readLength = is.read(buffer, 0, GWConstants.BUFSIZE)) >= 0) {
                    totalReads += readLength;
                    if (filePrimary == null) {
                        clientPrimary.put(buffer, 0, readLength);
                    } else {
                        fosPrimary.write(buffer, 0, readLength);
                    }
                    if (fileReplica == null) {
                        clientReplica.put(buffer, 0, readLength);
                    } else {
                        fosReplica.write(buffer, 0, readLength);
                    }
                    md5er.update(buffer, 0, readLength);
                }
                if (filePrimary == null) {
                    clientPrimary.putFlush();
                    OSDClientManager.getInstance().returnOSDClient(clientPrimary);
                } else {
                    fosPrimary.flush();
                    if (filePrimary.exists()) {
                        retryRenameTo(filePrimary, trashPrimary);
                    }
                    setAttributeFileReplication(tmpFilePrimary, GWConstants.FILE_ATTRUBUTE_REPLICATION_PRIMARY, objMeta.getReplicaDisk().getId());
                    retryRenameTo(tmpFilePrimary, filePrimary);
                    if (isPrimaryCache) {
                        String path = makeObjPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId);
                        Files.createSymbolicLink(Paths.get(path), Paths.get(filePrimary.getAbsolutePath()));
                    }
                }
                if (fileReplica == null) {
                    clientReplica.putFlush();
                    OSDClientManager.getInstance().returnOSDClient(clientReplica);
                } else {
                    fosReplica.flush();
                    if (fileReplica.exists()) {
                        retryRenameTo(fileReplica, trashReplica);
                    }
                    setAttributeFileReplication(tmpFileReplica, GWConstants.FILE_ATTRIBUTE_REPLICATION_REPLICA, GWConstants.FILE_ATTRIBUTE_REPLICA_DISK_ID_NULL);
                    retryRenameTo(tmpFileReplica, fileReplica);
                    if (isReplicaCache) {
                        String path = makeObjPath(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId);
                        Files.createSymbolicLink(Paths.get(path), Paths.get(fileReplica.getAbsolutePath()));
                    }
                }
            } else {
                File file = null;
                File tmpFile = null;
                File trashFile = null;
                if (GWUtils.getLocalIP().equals(objMeta.getPrimaryDisk().getOsdIp())) {
                    if (!Strings.isNullOrEmpty(GWConfig.getCacheDisk()) && length <= GWConfig.getCacheFileSize() * GWConstants.MEGABYTES) {
                        file = new File(makeCachePath(makeObjPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId)));
                        tmpFile = new File(makeCachePath(makeTempPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId)));
                        trashFile = new File(makeCachePath(makeTrashPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId)));
                        File link = new File(makeObjPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
                        com.google.common.io.Files.createParentDirs(file);
                        com.google.common.io.Files.createParentDirs(tmpFile);
                        com.google.common.io.Files.createParentDirs(link);
                        fosPrimary = new FileOutputStream(tmpFile, false);
                        isPrimaryCache = true;
                    } else {
                        file = new File(makeObjPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
                        tmpFile = new File(makeTempPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
                        trashFile = new File(makeTrashPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
                        com.google.common.io.Files.createParentDirs(file);
                        com.google.common.io.Files.createParentDirs(tmpFile);
                        fosPrimary = new FileOutputStream(tmpFile, false);
                    }
                } else {
                    clientPrimary = OSDClientManager.getInstance().getOSDClient(objMeta.getPrimaryDisk().getOsdIp());
                    clientPrimary.putInit(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId, length, GWConstants.FILE_ATTRUBUTE_REPLICATION_PRIMARY, objMeta.getReplicaDisk().getId(), GWConfig.getPerformanceMode());
                }
                while ((readLength = is.read(buffer, 0, GWConstants.BUFSIZE)) >= 0) {
                    totalReads += readLength;
                    if (file == null) {
                        clientPrimary.put(buffer, 0, readLength);
                    } else {
                        fosPrimary.write(buffer, 0, readLength);
                    }
                    md5er.update(buffer, 0, readLength);
                }
                if (file == null) {
                    clientPrimary.putFlush();
                    OSDClientManager.getInstance().returnOSDClient(clientPrimary);
                } else {
                    fosPrimary.flush();
                    if (file.exists()) {
                        retryRenameTo(file, trashFile);
                    }
                    if (objMeta.getReplicaDisk() != null) {
                        setAttributeFileReplication(tmpFile, GWConstants.FILE_ATTRUBUTE_REPLICATION_PRIMARY, objMeta.getReplicaDisk().getId());
                    }
                    retryRenameTo(tmpFile, file);
                    if (isPrimaryCache) {
                        String path = makeObjPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId);
                        Files.createSymbolicLink(Paths.get(path), Paths.get(file.getAbsolutePath()));
                    }
                }
            }
        } else // No replication option
        if (GWConfig.isNoReplica()) {
            File file = null;
            File tmpFile = null;
            File trashFile = null;
            if (GWUtils.getLocalIP().equals(objMeta.getPrimaryDisk().getOsdIp())) {
                if (!Strings.isNullOrEmpty(GWConfig.getCacheDisk()) && length <= GWConfig.getCacheFileSize() * GWConstants.MEGABYTES) {
                    file = new File(makeCachePath(makeObjPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId)));
                    tmpFile = new File(makeCachePath(makeTempPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId)));
                    trashFile = new File(makeCachePath(makeTrashPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId)));
                    File link = new File(makeObjPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
                    com.google.common.io.Files.createParentDirs(file);
                    com.google.common.io.Files.createParentDirs(tmpFile);
                    com.google.common.io.Files.createParentDirs(link);
                    fosPrimary = new FileOutputStream(tmpFile, false);
                    isPrimaryCache = true;
                } else {
                    file = new File(makeObjPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
                    tmpFile = new File(makeTempPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
                    trashFile = new File(makeTrashPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
                    com.google.common.io.Files.createParentDirs(file);
                    com.google.common.io.Files.createParentDirs(tmpFile);
                    fosPrimary = new FileOutputStream(tmpFile, false);
                }
            } else {
                clientPrimary = OSDClientManager.getInstance().getOSDClient(objMeta.getPrimaryDisk().getOsdIp());
                clientPrimary.putInit(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId, length, GWConstants.FILE_ATTRUBUTE_REPLICATION_PRIMARY, objMeta.getReplicaDisk().getId(), GWConfig.getPerformanceMode());
            }
            while ((readLength = is.read(buffer, 0, GWConstants.BUFSIZE)) >= 0) {
                totalReads += readLength;
                if (file == null) {
                    clientPrimary.put(buffer, 0, readLength);
                } else {
                    fosPrimary.write(buffer, 0, readLength);
                }
                md5er.update(buffer, 0, readLength);
            }
            if (file == null) {
                clientPrimary.putFlush();
                OSDClientManager.getInstance().returnOSDClient(clientPrimary);
            } else {
                fosPrimary.flush();
                if (file.exists()) {
                    retryRenameTo(file, trashFile);
                }
                if (objMeta.getReplicaDisk() != null) {
                    setAttributeFileReplication(tmpFile, GWConstants.FILE_ATTRUBUTE_REPLICATION_PRIMARY, objMeta.getReplicaDisk().getId());
                }
                retryRenameTo(tmpFile, file);
                if (isPrimaryCache) {
                    String path = makeObjPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId);
                    Files.createSymbolicLink(Paths.get(path), Paths.get(file.getAbsolutePath()));
                }
            }
        } else // No IO option
        if (GWConfig.isNoIO()) {
            while ((readLength = is.read(buffer, 0, GWConstants.BUFSIZE)) > 0) {
                totalReads += readLength;
                md5er.update(buffer, 0, readLength);
            }
        } else // No disk option
        if (GWConfig.isNoDisk()) {
            if (!GWUtils.getLocalIP().equals(objMeta.getPrimaryDisk().getOsdIp())) {
                clientPrimary = OSDClientManager.getInstance().getOSDClient(objMeta.getPrimaryDisk().getOsdIp());
                clientPrimary.putInit(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId, length, GWConstants.FILE_ATTRUBUTE_REPLICATION_PRIMARY, objMeta.getReplicaDisk().getId(), GWConfig.getPerformanceMode());
            }
            if (objMeta.getReplicaDisk() != null && !GWUtils.getLocalIP().equals(objMeta.getReplicaDisk().getOsdIp())) {
                clientReplica = OSDClientManager.getInstance().getOSDClient(objMeta.getReplicaDisk().getOsdIp());
                clientReplica.putInit(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId, length, GWConstants.FILE_ATTRIBUTE_REPLICATION_REPLICA, GWConstants.FILE_ATTRIBUTE_REPLICA_DISK_ID_NULL, GWConfig.getPerformanceMode());
            }
            while ((readLength = is.read(buffer, 0, GWConstants.BUFSIZE)) >= 0) {
                totalReads += readLength;
                if (clientPrimary != null) {
                    clientPrimary.put(buffer, 0, readLength);
                }
                if (clientReplica != null) {
                    clientReplica.put(buffer, 0, readLength);
                }
                md5er.update(buffer, 0, readLength);
            }
            if (clientPrimary != null) {
                clientPrimary.putFlush();
                OSDClientManager.getInstance().returnOSDClient(clientPrimary);
            }
            if (clientReplica != null) {
                clientReplica.putFlush();
                OSDClientManager.getInstance().returnOSDClient(clientReplica);
            }
        } else {
            logger.error(GWConstants.LOG_S3OBJECT_OPERATION_OPTION_NO_CASE, GWConfig.getPerformanceMode());
            throw new GWException(GWErrorCode.SERVER_ERROR, s3Parameter);
        }
        byte[] digest = md5er.digest();
        String eTag = base16().lowerCase().encode(digest);
        s3Object.setEtag(eTag);
        s3Object.setLastModified(new Date());
        s3Object.setFileSize(totalReads);
        s3Object.setVersionId(versionId);
        s3Object.setDeleteMarker(GWConstants.OBJECT_TYPE_FILE);
        calSize = putSize - existFileSize;
        if (GWConfig.isNoOption()) {
            updateBucketUsed(objMeta.getBucket(), calSize);
        }
    } catch (NoSuchAlgorithmException | IOException e) {
        PrintStack.logging(logger, e);
        throw new GWException(GWErrorCode.SERVER_ERROR, s3Parameter);
    } catch (ResourceNotFoundException e) {
        PrintStack.logging(logger, e);
        throw new GWException(GWErrorCode.NO_SUCH_KEY, s3Parameter);
    } catch (Exception e) {
        PrintStack.logging(logger, e);
        throw new GWException(GWErrorCode.SERVER_ERROR, s3Parameter);
    } finally {
        if (objMeta.getReplicaCount() > 1) {
            if (fosPrimary != null) {
                try {
                    fosPrimary.close();
                } catch (IOException e) {
                    PrintStack.logging(logger, e);
                    throw new GWException(GWErrorCode.SERVER_ERROR, s3Parameter);
                }
            }
            if (fosReplica != null) {
                try {
                    fosReplica.close();
                } catch (IOException e) {
                    PrintStack.logging(logger, e);
                    throw new GWException(GWErrorCode.SERVER_ERROR, s3Parameter);
                }
            }
        }
    }
    return s3Object;
}
Also used : NoSuchAlgorithmException(java.security.NoSuchAlgorithmException) IOException(java.io.IOException) Date(java.util.Date) GWException(com.pspace.ifs.ksan.gw.exception.GWException) ResourceNotFoundException(com.pspace.ifs.ksan.objmanager.ObjManagerException.ResourceNotFoundException) IOException(java.io.IOException) NoSuchAlgorithmException(java.security.NoSuchAlgorithmException) OSDClient(com.pspace.ifs.ksan.gw.object.osdclient.OSDClient) FileOutputStream(java.io.FileOutputStream) GWException(com.pspace.ifs.ksan.gw.exception.GWException) MessageDigest(java.security.MessageDigest) ResourceNotFoundException(com.pspace.ifs.ksan.objmanager.ObjManagerException.ResourceNotFoundException) File(java.io.File)

Example 3 with OSDClient

use of com.pspace.ifs.ksan.gw.object.osdclient.OSDClient in project ksan by infinistor.

the class S3ObjectOperation method completeMultipart.

public S3Object completeMultipart(SortedMap<Integer, Part> listPart) throws Exception {
    S3Object s3Object = new S3Object();
    byte[] buffer = new byte[GWConstants.MAXBUFSIZE];
    MessageDigest md5er = MessageDigest.getInstance(GWConstants.MD5);
    long totalLength = 0L;
    long existFileSize = 0L;
    long putSize = 0L;
    long calSize = 0L;
    File tmpFile = new File(makeTempPath(GWDiskConfig.getInstance().getLocalPath(), objMeta.getObjId(), versionId));
    try (FileOutputStream tmpOut = new FileOutputStream(tmpFile)) {
        com.google.common.io.Files.createParentDirs(tmpFile);
        // for each part object
        for (Iterator<Map.Entry<Integer, Part>> it = listPart.entrySet().iterator(); it.hasNext(); ) {
            Map.Entry<Integer, Part> entry = it.next();
            if (GWDiskConfig.getInstance().getLocalDiskID().equals(entry.getValue().getDiskID())) {
                // part is in local disk
                File partFile = new File(makeTempPath(GWDiskConfig.getInstance().getLocalPath(), objMeta.getObjId(), String.valueOf(entry.getValue().getPartNumber())));
                try (FileInputStream fis = new FileInputStream(partFile)) {
                    int readLength = 0;
                    while ((readLength = fis.read(buffer, 0, GWConstants.MAXBUFSIZE)) != -1) {
                        totalLength += readLength;
                        tmpOut.write(buffer, 0, readLength);
                        md5er.update(buffer, 0, readLength);
                    }
                    tmpOut.flush();
                    if (!partFile.delete()) {
                        logger.error(GWConstants.LOG_S3OBJECT_OPERATION_FAILED_FILE_DELETE, partFile.getName());
                    }
                }
            } else {
                OSDClient client = OSDClientManager.getInstance().getOSDClient(entry.getValue().getDiskID());
                client.getPartInit(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), String.valueOf(entry.getValue().getPartNumber()), entry.getValue().getPartSize(), tmpOut, md5er);
                totalLength += client.getPart();
                OSDClientManager.getInstance().returnOSDClient(client);
            }
        }
        byte[] digest = md5er.digest();
        String eTag = base16().lowerCase().encode(digest);
        s3Object.setEtag(eTag);
        s3Object.setLastModified(new Date());
        s3Object.setFileSize(totalLength);
        s3Object.setDeleteMarker(GWConstants.OBJECT_TYPE_FILE);
    }
    OSDClient clientPrimary = null;
    OSDClient clientReplica = null;
    int readLength = 0;
    existFileSize = objMeta.getSize();
    putSize = totalLength;
    if (objMeta.getReplicaCount() > 1) {
        existFileSize *= objMeta.getReplicaCount();
        putSize *= objMeta.getReplicaCount();
        if (!GWUtils.getLocalIP().equals(objMeta.getPrimaryDisk().getOsdIp())) {
            clientPrimary = OSDClientManager.getInstance().getOSDClient(objMeta.getPrimaryDisk().getOsdIp());
            clientPrimary.putInit(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId, totalLength, GWConstants.FILE_ATTRUBUTE_REPLICATION_PRIMARY, objMeta.getReplicaDisk().getId(), GWConfig.getPerformanceMode());
        }
        if (!GWUtils.getLocalIP().equals(objMeta.getReplicaDisk().getOsdIp())) {
            clientReplica = OSDClientManager.getInstance().getOSDClient(objMeta.getReplicaDisk().getOsdIp());
            clientReplica.putInit(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId, totalLength, GWConstants.FILE_ATTRIBUTE_REPLICATION_REPLICA, GWConstants.FILE_ATTRIBUTE_REPLICA_DISK_ID_NULL, GWConfig.getPerformanceMode());
        }
        if (clientPrimary != null || clientReplica != null) {
            try (FileInputStream fis = new FileInputStream(tmpFile)) {
                readLength = 0;
                while ((readLength = fis.read(buffer, 0, GWConstants.MAXBUFSIZE)) != -1) {
                    if (clientPrimary != null) {
                        clientPrimary.put(buffer, 0, readLength);
                    }
                    if (clientReplica != null) {
                        clientReplica.put(buffer, 0, readLength);
                    }
                }
            }
        }
        if (GWUtils.getLocalIP().equals(objMeta.getPrimaryDisk().getOsdIp())) {
            File file = new File(makeObjPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
            File trashFile = new File(makeTrashPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
            com.google.common.io.Files.createParentDirs(file);
            if (file.exists()) {
                retryRenameTo(file, trashFile);
            }
            setAttributeFileReplication(tmpFile, GWConstants.FILE_ATTRUBUTE_REPLICATION_PRIMARY, objMeta.getReplicaDisk().getId());
            retryRenameTo(tmpFile, file);
        }
        if (GWUtils.getLocalIP().equals(objMeta.getReplicaDisk().getOsdIp())) {
            File file = new File(makeObjPath(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId));
            File trashFile = new File(makeTrashPath(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId));
            com.google.common.io.Files.createParentDirs(file);
            if (file.exists()) {
                retryRenameTo(file, trashFile);
            }
            setAttributeFileReplication(tmpFile, GWConstants.FILE_ATTRIBUTE_REPLICATION_REPLICA, GWConstants.FILE_ATTRIBUTE_REPLICA_DISK_ID_NULL);
            retryRenameTo(tmpFile, file);
        }
    } else {
        File file = new File(makeObjPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
        File trashFile = new File(makeTrashPath(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId));
        com.google.common.io.Files.createParentDirs(file);
        if (file.exists()) {
            retryRenameTo(file, trashFile);
        }
        setAttributeFileReplication(tmpFile, GWConstants.FILE_ATTRUBUTE_REPLICATION_PRIMARY, GWConstants.FILE_ATTRIBUTE_REPLICA_DISK_ID_NULL);
        retryRenameTo(tmpFile, file);
    }
    calSize = putSize - existFileSize;
    updateBucketUsed(objMeta.getBucket(), calSize);
    return s3Object;
}
Also used : FileInputStream(java.io.FileInputStream) Date(java.util.Date) OSDClient(com.pspace.ifs.ksan.gw.object.osdclient.OSDClient) Part(com.pspace.ifs.ksan.gw.object.multipart.Part) FileOutputStream(java.io.FileOutputStream) MessageDigest(java.security.MessageDigest) File(java.io.File) Map(java.util.Map) SortedMap(java.util.SortedMap)

Example 4 with OSDClient

use of com.pspace.ifs.ksan.gw.object.osdclient.OSDClient in project ksan by infinistor.

the class S3ObjectOperation method copyObjectLocalToOSD.

private void copyObjectLocalToOSD(String srcPath, String srcObjId, String srcVersionId, String osdIP, String path, String objId, String versionId, String replication, String replicaDiskID) throws GWException {
    byte[] buffer = new byte[GWConstants.MAXBUFSIZE];
    File srcFile = new File(makeObjPath(srcPath, srcObjId, srcVersionId));
    OSDClient client = null;
    try {
        client = OSDClientManager.getInstance().getOSDClient(objMeta.getPrimaryDisk().getOsdIp());
        client.putInit(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId, s3Meta.getContentLength(), replication, replicaDiskID, "");
    } catch (Exception e) {
        PrintStack.logging(logger, e);
        throw new GWException(GWErrorCode.SERVER_ERROR, s3Parameter);
    }
    try (FileInputStream fis = new FileInputStream(srcFile)) {
        int readLength = 0;
        while ((readLength = fis.read(buffer, 0, GWConstants.MAXBUFSIZE)) != -1) {
            client.put(buffer, 0, readLength);
        }
        client.putFlush();
        OSDClientManager.getInstance().returnOSDClient(client);
    } catch (Exception e) {
        PrintStack.logging(logger, e);
        throw new GWException(GWErrorCode.SERVER_ERROR, s3Parameter);
    }
}
Also used : OSDClient(com.pspace.ifs.ksan.gw.object.osdclient.OSDClient) GWException(com.pspace.ifs.ksan.gw.exception.GWException) File(java.io.File) GWException(com.pspace.ifs.ksan.gw.exception.GWException) ResourceNotFoundException(com.pspace.ifs.ksan.objmanager.ObjManagerException.ResourceNotFoundException) IOException(java.io.IOException) NoSuchAlgorithmException(java.security.NoSuchAlgorithmException) FileInputStream(java.io.FileInputStream)

Example 5 with OSDClient

use of com.pspace.ifs.ksan.gw.object.osdclient.OSDClient in project ksan by infinistor.

the class S3ObjectOperation method deleteObject.

public boolean deleteObject() throws GWException {
    try {
        if (objMeta.getReplicaCount() > 1) {
            if (GWUtils.getLocalIP().equals(objMeta.getPrimaryDisk().getOsdIp())) {
                deleteObjectLocal(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId());
            } else {
                OSDClient client = OSDClientManager.getInstance().getOSDClient(objMeta.getPrimaryDisk().getOsdIp());
                client.delete(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), objMeta.getVersionId());
                OSDClientManager.getInstance().returnOSDClient(client);
            }
            if (GWUtils.getLocalIP().equals(objMeta.getReplicaDisk().getOsdIp())) {
                deleteObjectLocal(objMeta.getReplicaDisk().getPath(), objMeta.getObjId());
            } else {
                OSDClient client = OSDClientManager.getInstance().getOSDClient(objMeta.getReplicaDisk().getOsdIp());
                client.delete(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), objMeta.getVersionId());
                OSDClientManager.getInstance().returnOSDClient(client);
            }
        } else {
            deleteObjectLocal(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId());
        }
    } catch (Exception e) {
        PrintStack.logging(logger, e);
        throw new GWException(GWErrorCode.SERVER_ERROR, s3Parameter);
    }
    logger.info(GWConstants.LOG_S3OBJECT_OPERATION_DELETE, objMeta.getBucket(), objMeta.getPath(), versionId);
    return true;
}
Also used : OSDClient(com.pspace.ifs.ksan.gw.object.osdclient.OSDClient) GWException(com.pspace.ifs.ksan.gw.exception.GWException) GWException(com.pspace.ifs.ksan.gw.exception.GWException) ResourceNotFoundException(com.pspace.ifs.ksan.objmanager.ObjManagerException.ResourceNotFoundException) IOException(java.io.IOException) NoSuchAlgorithmException(java.security.NoSuchAlgorithmException)

Aggregations

OSDClient (com.pspace.ifs.ksan.gw.object.osdclient.OSDClient)9 GWException (com.pspace.ifs.ksan.gw.exception.GWException)7 ResourceNotFoundException (com.pspace.ifs.ksan.objmanager.ObjManagerException.ResourceNotFoundException)7 IOException (java.io.IOException)7 NoSuchAlgorithmException (java.security.NoSuchAlgorithmException)7 File (java.io.File)5 Date (java.util.Date)4 Part (com.pspace.ifs.ksan.gw.object.multipart.Part)2 FileInputStream (java.io.FileInputStream)2 FileOutputStream (java.io.FileOutputStream)2 MessageDigest (java.security.MessageDigest)2 Map (java.util.Map)2 SortedMap (java.util.SortedMap)2 OSDData (com.pspace.ifs.ksan.osd.OSDData)1
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial