Search in sources :

Example 16 with StoredObject

use of org.javaswift.joss.model.StoredObject in project dataverse by IQSS.

the class SwiftAccessIO method deleteAuxObject.

@Override
public void deleteAuxObject(String auxItemTag) throws IOException {
    StoredObject swiftAuxObject = openSwiftAuxFile(auxItemTag);
    if (swiftAuxObject == null) {
        throw new FileNotFoundException("No such Aux object: " + auxItemTag);
    }
    swiftAuxObject.delete();
}
Also used : StoredObject(org.javaswift.joss.model.StoredObject) FileNotFoundException(java.io.FileNotFoundException)

Example 17 with StoredObject

use of org.javaswift.joss.model.StoredObject in project dataverse by IQSS.

the class SwiftAccessIO method deleteAllAuxObjects.

@Override
public void deleteAllAuxObjects() throws IOException {
    if (this.swiftContainer == null || this.swiftFileObject == null) {
        throw new IOException("This SwiftAccessIO() hasn't been properly initialized yet. (did you execute SwiftAccessIO.open()?)");
    }
    Collection<StoredObject> victims;
    String lastVictim = null;
    while ((victims = this.swiftContainer.list(this.swiftFileObject.getName() + ".", lastVictim, LIST_PAGE_LIMIT)) != null && victims.size() > 0) {
        for (StoredObject victim : victims) {
            lastVictim = victim.getName();
            logger.info("trying to delete " + lastVictim);
            victim.delete();
        }
    }
}
Also used : StoredObject(org.javaswift.joss.model.StoredObject) IOException(java.io.IOException)

Example 18 with StoredObject

use of org.javaswift.joss.model.StoredObject in project dataverse by IQSS.

the class SwiftAccessIO method listAuxObjects.

@Override
public List<String> listAuxObjects() throws IOException {
    if (this.swiftContainer == null || this.swiftFileObject == null) {
        throw new IOException("This SwiftAccessIO() hasn't been properly initialized yet.");
    }
    String namePrefix = this.swiftFileObject.getName() + ".";
    Collection<StoredObject> items;
    String lastItemName = null;
    List<String> ret = new ArrayList<>();
    while ((items = this.swiftContainer.list(namePrefix, lastItemName, LIST_PAGE_LIMIT)) != null && items.size() > 0) {
        for (StoredObject item : items) {
            lastItemName = item.getName().substring(namePrefix.length());
            ret.add(lastItemName);
        }
    }
    return ret;
}
Also used : StoredObject(org.javaswift.joss.model.StoredObject) ArrayList(java.util.ArrayList) IOException(java.io.IOException)

Example 19 with StoredObject

use of org.javaswift.joss.model.StoredObject in project dataverse by IQSS.

the class SwiftAccessIO method saveInputStreamAsAux.

// this method copies a local InputStream into this DataAccess Auxiliary location:
@Override
public void saveInputStreamAsAux(InputStream inputStream, String auxItemTag) throws IOException {
    if (swiftFileObject == null) {
        open();
    }
    try {
        StoredObject swiftAuxObject = openSwiftAuxFile(true, auxItemTag);
        swiftAuxObject.uploadObject(inputStream);
    } catch (IOException ex) {
        String failureMsg = ex.getMessage();
        if (failureMsg == null) {
            failureMsg = "Swift AccessIO: Unknown exception occured while saving a local InputStream as a Swift StoredObject";
        }
        throw new IOException(failureMsg);
    }
}
Also used : StoredObject(org.javaswift.joss.model.StoredObject) IOException(java.io.IOException)

Example 20 with StoredObject

use of org.javaswift.joss.model.StoredObject in project stocator by CODAIT.

the class SwiftAPIClient method setCorrectSize.

/**
 * Swift has a bug where container listing might wrongly report size 0
 * for large objects. It's seems to be a well known issue in Swift without
 * solution.
 * We have to provide work around for this.
 * If container listing reports size 0 for some object, we send
 * additional HEAD on that object to verify it's size.
 *
 * @param tmp JOSS StoredObject
 * @param cObj JOSS Container object
 */
private void setCorrectSize(StoredObject tmp, Container cObj) {
    long objectSize = tmp.getContentLength();
    if (objectSize == 0) {
        // we may hit a well known Swift bug.
        // container listing reports 0 for large objects.
        StoredObject soDirect = cObj.getObject(tmp.getName());
        long contentLength = soDirect.getContentLength();
        if (contentLength > 0) {
            tmp.setContentLength(contentLength);
        }
    }
}
Also used : StoredObject(org.javaswift.joss.model.StoredObject)

Aggregations

StoredObject (org.javaswift.joss.model.StoredObject)26 IOException (java.io.IOException)9 Container (org.javaswift.joss.model.Container)8 FileNotFoundException (java.io.FileNotFoundException)4 ArrayList (java.util.ArrayList)3 StocatorPath (com.ibm.stocator.fs.common.StocatorPath)2 ConfigurationParseException (com.ibm.stocator.fs.common.exception.ConfigurationParseException)2 JossAccount (com.ibm.stocator.fs.swift.auth.JossAccount)2 DataFile (edu.harvard.iq.dataverse.DataFile)2 UnsupportedEncodingException (java.io.UnsupportedEncodingException)2 FileStatus (org.apache.hadoop.fs.FileStatus)2 Path (org.apache.hadoop.fs.Path)2 AlreadyExistsException (org.javaswift.joss.exception.AlreadyExistsException)2 CommandException (org.javaswift.joss.exception.CommandException)2 Account (org.javaswift.joss.model.Account)2 DirectoryOrObject (org.javaswift.joss.model.DirectoryOrObject)2 Dataset (edu.harvard.iq.dataverse.Dataset)1 File (java.io.File)1 Date (java.util.Date)1 Properties (java.util.Properties)1