Search in sources :

Example 1 with XAttr

use of org.apache.hadoop.fs.XAttr in project hadoop by apache.

the class INodeDirectory method getLocalStoragePolicyID.

@Override
public byte getLocalStoragePolicyID() {
    XAttrFeature f = getXAttrFeature();
    XAttr xattr = f == null ? null : f.getXAttr(BlockStoragePolicySuite.getStoragePolicyXAttrPrefixedName());
    if (xattr != null) {
        return (xattr.getValue())[0];
    }
    return BLOCK_STORAGE_POLICY_ID_UNSPECIFIED;
}
Also used : XAttr(org.apache.hadoop.fs.XAttr)

Example 2 with XAttr

use of org.apache.hadoop.fs.XAttr in project hadoop by apache.

the class TestJsonUtil method testToJsonFromXAttrs.

@Test
public void testToJsonFromXAttrs() throws IOException {
    String jsonString = "{\"XAttrs\":[{\"name\":\"user.a1\",\"value\":\"0x313233\"}," + "{\"name\":\"user.a2\",\"value\":\"0x313131\"}]}";
    XAttr xAttr1 = (new XAttr.Builder()).setNameSpace(XAttr.NameSpace.USER).setName("a1").setValue(XAttrCodec.decodeValue("0x313233")).build();
    XAttr xAttr2 = (new XAttr.Builder()).setNameSpace(XAttr.NameSpace.USER).setName("a2").setValue(XAttrCodec.decodeValue("0x313131")).build();
    List<XAttr> xAttrs = Lists.newArrayList();
    xAttrs.add(xAttr1);
    xAttrs.add(xAttr2);
    Assert.assertEquals(jsonString, JsonUtil.toJsonString(xAttrs, XAttrCodec.HEX));
}
Also used : XAttr(org.apache.hadoop.fs.XAttr) Test(org.junit.Test)

Example 3 with XAttr

use of org.apache.hadoop.fs.XAttr in project hadoop by apache.

the class TestOfflineImageViewerForXAttr method createOriginalFSImage.

/**
   * Create a populated namespace for later testing. Save its contents to a data
   * structure and store its fsimage location. We only want to generate the
   * fsimage file once and use it for multiple tests.
   */
@BeforeClass
public static void createOriginalFSImage() throws IOException {
    MiniDFSCluster cluster = null;
    Configuration conf = new Configuration();
    try {
        cluster = new MiniDFSCluster.Builder(conf).build();
        cluster.waitActive();
        DistributedFileSystem hdfs = cluster.getFileSystem();
        // Create a name space with XAttributes
        Path dir = new Path("/dir1");
        hdfs.mkdirs(dir);
        hdfs.setXAttr(dir, "user.attr1", "value1".getBytes());
        hdfs.setXAttr(dir, "user.attr2", "value2".getBytes());
        // Write results to the fsimage file
        hdfs.setSafeMode(HdfsConstants.SafeModeAction.SAFEMODE_ENTER, false);
        hdfs.saveNamespace();
        List<XAttr> attributes = new ArrayList<XAttr>();
        attributes.add(XAttrHelper.buildXAttr("user.attr1", "value1".getBytes()));
        attr1JSon = JsonUtil.toJsonString(attributes, null);
        attributes.add(XAttrHelper.buildXAttr("user.attr2", "value2".getBytes()));
        // Determine the location of the fsimage file
        originalFsimage = FSImageTestUtil.findLatestImageFile(FSImageTestUtil.getFSImage(cluster.getNameNode()).getStorage().getStorageDir(0));
        if (originalFsimage == null) {
            throw new RuntimeException("Didn't generate or can't find fsimage");
        }
        LOG.debug("original FS image file is " + originalFsimage);
    } finally {
        if (cluster != null)
            cluster.shutdown();
    }
}
Also used : Path(org.apache.hadoop.fs.Path) MiniDFSCluster(org.apache.hadoop.hdfs.MiniDFSCluster) Configuration(org.apache.hadoop.conf.Configuration) ArrayList(java.util.ArrayList) DistributedFileSystem(org.apache.hadoop.hdfs.DistributedFileSystem) XAttr(org.apache.hadoop.fs.XAttr) BeforeClass(org.junit.BeforeClass)

Example 4 with XAttr

use of org.apache.hadoop.fs.XAttr in project hadoop by apache.

the class FSEditLogOp method readXAttrsFromXml.

private static List<XAttr> readXAttrsFromXml(Stanza st) throws InvalidXmlException {
    if (!st.hasChildren("XATTR")) {
        return null;
    }
    List<Stanza> stanzas = st.getChildren("XATTR");
    List<XAttr> xattrs = Lists.newArrayListWithCapacity(stanzas.size());
    for (Stanza a : stanzas) {
        XAttr.Builder builder = new XAttr.Builder();
        builder.setNameSpace(XAttr.NameSpace.valueOf(a.getValue("NAMESPACE"))).setName(a.getValue("NAME"));
        String v = a.getValueOrNull("VALUE");
        if (v != null) {
            try {
                builder.setValue(XAttrCodec.decodeValue(v));
            } catch (IOException e) {
                throw new InvalidXmlException(e.toString());
            }
        }
        xattrs.add(builder.build());
    }
    return xattrs;
}
Also used : InvalidXmlException(org.apache.hadoop.hdfs.util.XMLUtils.InvalidXmlException) Stanza(org.apache.hadoop.hdfs.util.XMLUtils.Stanza) IOException(java.io.IOException) XAttr(org.apache.hadoop.fs.XAttr)

Example 5 with XAttr

use of org.apache.hadoop.fs.XAttr in project hadoop by apache.

the class FSEditLogOp method appendXAttrsToXml.

private static void appendXAttrsToXml(ContentHandler contentHandler, List<XAttr> xAttrs) throws SAXException {
    for (XAttr xAttr : xAttrs) {
        contentHandler.startElement("", "", "XATTR", new AttributesImpl());
        XMLUtils.addSaxString(contentHandler, "NAMESPACE", xAttr.getNameSpace().toString());
        XMLUtils.addSaxString(contentHandler, "NAME", xAttr.getName());
        if (xAttr.getValue() != null) {
            try {
                XMLUtils.addSaxString(contentHandler, "VALUE", XAttrCodec.encodeValue(xAttr.getValue(), XAttrCodec.HEX));
            } catch (IOException e) {
                throw new SAXException(e);
            }
        }
        contentHandler.endElement("", "", "XATTR");
    }
}
Also used : AttributesImpl(org.xml.sax.helpers.AttributesImpl) IOException(java.io.IOException) XAttr(org.apache.hadoop.fs.XAttr) SAXException(org.xml.sax.SAXException)

Aggregations

XAttr (org.apache.hadoop.fs.XAttr)43 IOException (java.io.IOException)13 Test (org.junit.Test)7 HdfsProtos (org.apache.hadoop.hdfs.protocol.proto.HdfsProtos)5 FileNotFoundException (java.io.FileNotFoundException)4 XAttrProto (org.apache.hadoop.hdfs.protocol.proto.XAttrProtos.XAttrProto)4 InvalidProtocolBufferException (com.google.protobuf.InvalidProtocolBufferException)2 ByteArrayOutputStream (java.io.ByteArrayOutputStream)2 ArrayList (java.util.ArrayList)2 HadoopIllegalArgumentException (org.apache.hadoop.HadoopIllegalArgumentException)2 CipherSuite (org.apache.hadoop.crypto.CipherSuite)2 CryptoProtocolVersion (org.apache.hadoop.crypto.CryptoProtocolVersion)2 AccessControlException (org.apache.hadoop.security.AccessControlException)2 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1 ObjectReader (com.fasterxml.jackson.databind.ObjectReader)1 VisibleForTesting (com.google.common.annotations.VisibleForTesting)1 ImmutableList (com.google.common.collect.ImmutableList)1 ByteArrayInputStream (java.io.ByteArrayInputStream)1 DataInputStream (java.io.DataInputStream)1 DataOutputStream (java.io.DataOutputStream)1