Examples with OSDClient com.pspace.ifs.ksan.gw.object.osdclient.OSDClient used on opensource projects

Search in sources :

Example 6 with OSDClient

use of com.pspace.ifs.ksan.gw.object.osdclient.OSDClient in project ksan by infinistor.

the class S3ObjectOperation method getObject.

public void getObject(S3Range s3Range) throws Exception {
    OSDClient client = null;
    String sourceRange = "";
    long actualSize = 0L;
    long fileSize = objMeta.getSize();
    if (s3Range != null && s3Range.getListRange().size() > 0) {
        fileSize = 0L;
        for (S3Range.Range range : s3Range.getListRange()) {
            if (Strings.isNullOrEmpty(sourceRange)) {
                sourceRange = String.valueOf(range.getOffset()) + GWConstants.COMMA + String.valueOf(range.getLength());
            } else {
                sourceRange += GWConstants.SLASH + String.valueOf(range.getOffset()) + GWConstants.COMMA + String.valueOf(range.getLength());
            }
            fileSize += range.getLength();
        }
    }
    try {
        if (objMeta.getReplicaCount() > 1) {
            logger.debug("bucket : {}, object : {}", objMeta.getBucket(), objMeta.getPath());
            logger.debug("primary disk id : {}, osd ip : {}", objMeta.getPrimaryDisk().getId(), objMeta.getPrimaryDisk().getOsdIp());
            logger.debug("replica disk id : {}, osd ip : {}", objMeta.getReplicaDisk().getId(), objMeta.getReplicaDisk().getOsdIp());
            if (GWUtils.getLocalIP().equals(objMeta.getPrimaryDisk().getOsdIp())) {
                actualSize = getObjectLocal(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), sourceRange);
                s3Parameter.addResponseSize(actualSize);
            } else if (GWUtils.getLocalIP().equals(objMeta.getReplicaDisk().getOsdIp())) {
                actualSize = getObjectLocal(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), sourceRange);
                s3Parameter.addResponseSize(actualSize);
            } else {
                try {
                    client = OSDClientManager.getInstance().getOSDClient(objMeta.getPrimaryDisk().getOsdIp());
                    client.getInit(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), objMeta.getVersionId(), fileSize, sourceRange, s3Parameter.getResponse().getOutputStream());
                    actualSize = client.get();
                    logger.debug("end ..... client get : actualSize : {}", actualSize);
                    OSDClientManager.getInstance().returnOSDClient(client);
                    s3Parameter.addResponseSize(actualSize);
                } catch (Exception e) {
                    client = OSDClientManager.getInstance().getOSDClient(objMeta.getReplicaDisk().getOsdIp());
                    client.getInit(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), objMeta.getVersionId(), fileSize, sourceRange, s3Parameter.getResponse().getOutputStream());
                    actualSize = client.get();
                    OSDClientManager.getInstance().returnOSDClient(client);
                    s3Parameter.addResponseSize(actualSize);
                }
            }
        } else {
            if (GWUtils.getLocalIP().equals(objMeta.getPrimaryDisk().getOsdIp())) {
                actualSize = getObjectLocal(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), sourceRange);
            } else {
                try {
                    client = OSDClientManager.getInstance().getOSDClient(objMeta.getPrimaryDisk().getOsdIp());
                    client.getInit(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), objMeta.getVersionId(), fileSize, sourceRange, s3Parameter.getResponse().getOutputStream());
                    actualSize = client.get();
                    OSDClientManager.getInstance().returnOSDClient(client);
                    s3Parameter.addResponseSize(actualSize);
                } catch (Exception e) {
                    client = OSDClientManager.getInstance().getOSDClient(objMeta.getReplicaDisk().getOsdIp());
                    client.getInit(objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), objMeta.getVersionId(), fileSize, sourceRange, s3Parameter.getResponse().getOutputStream());
                    actualSize = client.get();
                    OSDClientManager.getInstance().returnOSDClient(client);
                    s3Parameter.addResponseSize(actualSize);
                }
            }
        }
    } catch (Exception e) {
        PrintStack.logging(logger, e);
        logger.error(e.getMessage());
        throw new GWException(GWErrorCode.SERVER_ERROR, s3Parameter);
    }
    logger.debug(GWConstants.LOG_S3OBJECT_OPERATION_FILE_SIZE, actualSize);
}
Also used : OSDClient(com.pspace.ifs.ksan.gw.object.osdclient.OSDClient) GWException(com.pspace.ifs.ksan.gw.exception.GWException) GWException(com.pspace.ifs.ksan.gw.exception.GWException) ResourceNotFoundException(com.pspace.ifs.ksan.objmanager.ObjManagerException.ResourceNotFoundException) IOException(java.io.IOException) NoSuchAlgorithmException(java.security.NoSuchAlgorithmException)

Example 7 with OSDClient

use of com.pspace.ifs.ksan.gw.object.osdclient.OSDClient in project ksan by infinistor.

the class S3ObjectOperation method uploadPartCopy.

public S3Object uploadPartCopy(String path, Metadata srcObjMeta, S3Range s3Range) throws GWException {
    S3Object s3Object = new S3Object();
    try {
        String copySourceRange = "";
        if (s3Range != null && s3Range.getListRange().size() > 0) {
            for (S3Range.Range range : s3Range.getListRange()) {
                if (Strings.isNullOrEmpty(copySourceRange)) {
                    copySourceRange = String.valueOf(range.getOffset()) + GWConstants.COMMA + String.valueOf(range.getLength());
                } else {
                    copySourceRange += GWConstants.SLASH + String.valueOf(range.getOffset()) + GWConstants.COMMA + String.valueOf(range.getLength());
                }
            }
        }
        logger.debug(GWConstants.LOG_S3OBJECT_OPERATION_COPY_SOURCE_RANGE, copySourceRange);
        OSDData osdData = null;
        if (objMeta.getReplicaCount() > 1) {
            if (GWUtils.getLocalIP().equals(srcObjMeta.getPrimaryDisk().getOsdIp())) {
                osdData = uploadPartCopyLocal(srcObjMeta.getPrimaryDisk().getPath(), srcObjMeta.getObjId(), srcObjMeta.getVersionId(), copySourceRange, path, objMeta.getObjId(), s3Parameter.getPartNumber());
            } else if (GWUtils.getLocalIP().equals(srcObjMeta.getReplicaDisk().getOsdIp())) {
                osdData = uploadPartCopyLocal(srcObjMeta.getReplicaDisk().getPath(), srcObjMeta.getObjId(), srcObjMeta.getVersionId(), copySourceRange, path, objMeta.getObjId(), s3Parameter.getPartNumber());
            } else {
                try {
                    OSDClient client = OSDClientManager.getInstance().getOSDClient(srcObjMeta.getPrimaryDisk().getOsdIp());
                    osdData = client.partCopy(srcObjMeta.getPrimaryDisk().getPath(), srcObjMeta.getObjId(), srcObjMeta.getVersionId(), copySourceRange, path, objMeta.getObjId(), s3Parameter.getPartNumber());
                    OSDClientManager.getInstance().returnOSDClient(client);
                } catch (Exception e) {
                    OSDClient client = OSDClientManager.getInstance().getOSDClient(srcObjMeta.getReplicaDisk().getOsdIp());
                    osdData = client.partCopy(srcObjMeta.getReplicaDisk().getPath(), srcObjMeta.getObjId(), srcObjMeta.getVersionId(), copySourceRange, path, objMeta.getObjId(), s3Parameter.getPartNumber());
                    OSDClientManager.getInstance().returnOSDClient(client);
                }
            }
        } else {
            osdData = uploadPartCopyLocal(srcObjMeta.getPrimaryDisk().getPath(), srcObjMeta.getObjId(), srcObjMeta.getVersionId(), copySourceRange, path, objMeta.getObjId(), s3Parameter.getPartNumber());
        }
        if (osdData != null) {
            s3Object.setEtag(osdData.getETag());
            s3Object.setFileSize(osdData.getFileSize());
        } else {
            logger.error(GWConstants.LOG_S3OBJECT_OPERATION_OSD_ERROR);
            throw new GWException(GWErrorCode.SERVER_ERROR, s3Parameter);
        }
        s3Object.setLastModified(new Date());
        s3Object.setDeleteMarker(GWConstants.OBJECT_TYPE_FILE);
    } catch (Exception e) {
        logger.error(e.getMessage());
        throw new GWException(GWErrorCode.SERVER_ERROR, s3Parameter);
    }
    return s3Object;
}
Also used : OSDClient(com.pspace.ifs.ksan.gw.object.osdclient.OSDClient) OSDData(com.pspace.ifs.ksan.osd.OSDData) GWException(com.pspace.ifs.ksan.gw.exception.GWException) GWException(com.pspace.ifs.ksan.gw.exception.GWException) ResourceNotFoundException(com.pspace.ifs.ksan.objmanager.ObjManagerException.ResourceNotFoundException) IOException(java.io.IOException) NoSuchAlgorithmException(java.security.NoSuchAlgorithmException) Date(java.util.Date)

Example 8 with OSDClient

use of com.pspace.ifs.ksan.gw.object.osdclient.OSDClient in project ksan by infinistor.

the class S3ObjectOperation method copyObject.

public S3Object copyObject(Metadata srcObjMeta) throws GWException {
    S3Object s3Object = new S3Object();
    try {
        logger.info(GWConstants.LOG_S3OBJECT_OPERATION_LOCAL_IP, GWUtils.getLocalIP());
        logger.info(GWConstants.LOG_S3OBJECT_OPERATION_OBJ_PRIMARY_IP, objMeta.getPrimaryDisk().getOsdIp());
        if (objMeta.getReplicaDisk() != null) {
            logger.info(GWConstants.LOG_S3OBJECT_OPERATION_OBJ_REPLICA_IP, objMeta.getReplicaDisk().getOsdIp());
        }
        if (objMeta.getReplicaCount() > 1) {
            // check primary local src, obj
            if (GWUtils.getLocalIP().equals(srcObjMeta.getPrimaryDisk().getOsdIp())) {
                if (srcObjMeta.getPrimaryDisk().getOsdIp().equals(objMeta.getPrimaryDisk().getOsdIp())) {
                    copyObjectLocal(srcObjMeta.getPrimaryDisk().getPath(), srcObjMeta.getObjId(), srcObjMeta.getVersionId(), objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId, GWConstants.FILE_ATTRUBUTE_REPLICATION_PRIMARY, objMeta.getReplicaDisk().getId());
                } else {
                    // src local, obj replica
                    // put src to replica
                    copyObjectLocalToOSD(srcObjMeta.getPrimaryDisk().getPath(), srcObjMeta.getObjId(), srcObjMeta.getVersionId(), objMeta.getPrimaryDisk().getOsdIp(), objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId, GWConstants.FILE_ATTRUBUTE_REPLICATION_PRIMARY, objMeta.getReplicaDisk().getId());
                }
            } else {
                OSDClient client = OSDClientManager.getInstance().getOSDClient(objMeta.getPrimaryDisk().getOsdIp());
                client.copy(srcObjMeta.getPrimaryDisk().getPath(), srcObjMeta.getObjId(), srcObjMeta.getVersionId(), objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId, GWConstants.FILE_ATTRUBUTE_REPLICATION_PRIMARY, objMeta.getReplicaDisk().getId());
                OSDClientManager.getInstance().returnOSDClient(client);
            }
            // check replica local src, obj
            if (GWUtils.getLocalIP().equals(srcObjMeta.getReplicaDisk().getOsdIp())) {
                if (srcObjMeta.getReplicaDisk().getOsdIp().equals(objMeta.getReplicaDisk().getOsdIp())) {
                    copyObjectLocal(srcObjMeta.getReplicaDisk().getPath(), srcObjMeta.getObjId(), srcObjMeta.getVersionId(), objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId, GWConstants.FILE_ATTRIBUTE_REPLICATION_REPLICA, GWConstants.FILE_ATTRIBUTE_REPLICA_DISK_ID_NULL);
                } else {
                    // src local, obj replica
                    // put src to replica
                    copyObjectLocalToOSD(srcObjMeta.getReplicaDisk().getPath(), srcObjMeta.getObjId(), srcObjMeta.getVersionId(), objMeta.getReplicaDisk().getOsdIp(), objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId, GWConstants.FILE_ATTRIBUTE_REPLICATION_REPLICA, GWConstants.FILE_ATTRIBUTE_REPLICA_DISK_ID_NULL);
                }
            } else {
                OSDClient client = OSDClientManager.getInstance().getOSDClient(objMeta.getReplicaDisk().getOsdIp());
                client.copy(srcObjMeta.getReplicaDisk().getPath(), srcObjMeta.getObjId(), srcObjMeta.getVersionId(), objMeta.getReplicaDisk().getPath(), objMeta.getObjId(), versionId, GWConstants.FILE_ATTRIBUTE_REPLICATION_REPLICA, GWConstants.FILE_ATTRIBUTE_REPLICA_DISK_ID_NULL);
                OSDClientManager.getInstance().returnOSDClient(client);
            }
        } else {
            copyObjectLocal(srcObjMeta.getPrimaryDisk().getPath(), srcObjMeta.getObjId(), srcObjMeta.getVersionId(), objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), versionId, GWConstants.FILE_ATTRUBUTE_REPLICATION_PRIMARY, GWConstants.FILE_ATTRIBUTE_REPLICA_DISK_ID_NULL);
        }
        s3Object.setEtag(srcObjMeta.getEtag());
        s3Object.setLastModified(new Date());
        s3Object.setFileSize(srcObjMeta.getSize());
        s3Object.setVersionId(versionId);
        s3Object.setDeleteMarker(GWConstants.OBJECT_TYPE_FILE);
    } catch (Exception e) {
        PrintStack.logging(logger, e);
        throw new GWException(GWErrorCode.SERVER_ERROR, s3Parameter);
    }
    return s3Object;
}
Also used : OSDClient(com.pspace.ifs.ksan.gw.object.osdclient.OSDClient) GWException(com.pspace.ifs.ksan.gw.exception.GWException) Date(java.util.Date) GWException(com.pspace.ifs.ksan.gw.exception.GWException) ResourceNotFoundException(com.pspace.ifs.ksan.objmanager.ObjManagerException.ResourceNotFoundException) IOException(java.io.IOException) NoSuchAlgorithmException(java.security.NoSuchAlgorithmException)

Example 9 with OSDClient

use of com.pspace.ifs.ksan.gw.object.osdclient.OSDClient in project ksan by infinistor.

the class S3ObjectOperation method abortMultipart.

public void abortMultipart(SortedMap<Integer, Part> listPart) throws GWException {
    try {
        for (Iterator<Map.Entry<Integer, Part>> it = listPart.entrySet().iterator(); it.hasNext(); ) {
            Map.Entry<Integer, Part> entry = it.next();
            logger.info("key : {}, diskId : {}", entry.getKey(), objMeta.getPrimaryDisk().getId());
            if (GWDiskConfig.getInstance().getLocalDiskID().equals(entry.getValue().getDiskID())) {
                // part is in local disk
                File partFile = new File(makeTempPath(GWDiskConfig.getInstance().getLocalPath(), objMeta.getObjId(), String.valueOf(entry.getValue().getPartNumber())));
                partFile.delete();
            } else {
                String host = GWDiskConfig.getInstance().getOSDIP(entry.getValue().getDiskID());
                OSDClient client = OSDClientManager.getInstance().getOSDClient(host);
                client.deletePart(objMeta.getPrimaryDisk().getPath(), objMeta.getObjId(), s3Parameter.getPartNumber());
                OSDClientManager.getInstance().returnOSDClient(client);
            }
        }
    } catch (Exception e) {
        PrintStack.logging(logger, e);
        throw new GWException(GWErrorCode.SERVER_ERROR, s3Parameter);
    }
}
Also used : OSDClient(com.pspace.ifs.ksan.gw.object.osdclient.OSDClient) Part(com.pspace.ifs.ksan.gw.object.multipart.Part) GWException(com.pspace.ifs.ksan.gw.exception.GWException) Map(java.util.Map) SortedMap(java.util.SortedMap) File(java.io.File) GWException(com.pspace.ifs.ksan.gw.exception.GWException) ResourceNotFoundException(com.pspace.ifs.ksan.objmanager.ObjManagerException.ResourceNotFoundException) IOException(java.io.IOException) NoSuchAlgorithmException(java.security.NoSuchAlgorithmException)

Aggregations

OSDClient (com.pspace.ifs.ksan.gw.object.osdclient.OSDClient)9 GWException (com.pspace.ifs.ksan.gw.exception.GWException)7 ResourceNotFoundException (com.pspace.ifs.ksan.objmanager.ObjManagerException.ResourceNotFoundException)7 IOException (java.io.IOException)7 NoSuchAlgorithmException (java.security.NoSuchAlgorithmException)7 File (java.io.File)5 Date (java.util.Date)4 Part (com.pspace.ifs.ksan.gw.object.multipart.Part)2 FileInputStream (java.io.FileInputStream)2 FileOutputStream (java.io.FileOutputStream)2 MessageDigest (java.security.MessageDigest)2 Map (java.util.Map)2 SortedMap (java.util.SortedMap)2 OSDData (com.pspace.ifs.ksan.osd.OSDData)1
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial