Search in sources :

Example 11 with ExprTree

use of org.apache.asterix.external.classad.ExprTree in project asterixdb by apache.

the class ClassAdParser method parseOrderedList.

private void parseOrderedList(AOrderedListType oltype, Value listVal, DataOutput out) throws IOException, AsterixException {
    ArrayBackedValueStorage itemBuffer = getTempBuffer();
    OrderedListBuilder orderedListBuilder = (OrderedListBuilder) getOrderedListBuilder();
    IAType itemType = null;
    if (oltype != null) {
        itemType = oltype.getItemType();
    }
    orderedListBuilder.reset(oltype);
    for (ExprTree tree : listVal.getListVal().getExprList()) {
        itemBuffer.reset();
        writeFieldValueToBuffer(itemType, itemBuffer.getDataOutput(), null, tree, null);
        orderedListBuilder.addItem(itemBuffer);
    }
    orderedListBuilder.write(out, true);
}
Also used : ArrayBackedValueStorage(org.apache.hyracks.data.std.util.ArrayBackedValueStorage) OrderedListBuilder(org.apache.asterix.builders.OrderedListBuilder) ExprTree(org.apache.asterix.external.classad.ExprTree) IAType(org.apache.asterix.om.types.IAType)

Example 12 with ExprTree

use of org.apache.asterix.external.classad.ExprTree in project asterixdb by apache.

the class ClassAdToADMTest method testSchemaless.

/**
     *
     */
public void testSchemaless() {
    try {
        ClassAdObjectPool objectPool = new ClassAdObjectPool();
        ClassAd pAd = new ClassAd(objectPool);
        String[] files = new String[] { "/classad/jobads.txt" };
        ClassAdParser parser = new ClassAdParser(objectPool);
        CharArrayLexerSource lexerSource = new CharArrayLexerSource();
        for (String path : files) {
            List<Path> paths = new ArrayList<>();
            Map<String, String> config = new HashMap<>();
            config.put(ExternalDataConstants.KEY_RECORD_START, "[");
            config.put(ExternalDataConstants.KEY_RECORD_END, "]");
            paths.add(Paths.get(getClass().getResource(path).toURI()));
            FileSystemWatcher watcher = new FileSystemWatcher(paths, null, false);
            LocalFSInputStream in = new LocalFSInputStream(watcher);
            SemiStructuredRecordReader recordReader = new SemiStructuredRecordReader();
            recordReader.configure(in, config);
            try {
                Value val = new Value(objectPool);
                while (recordReader.hasNext()) {
                    val.reset();
                    IRawRecord<char[]> record = recordReader.next();
                    lexerSource.setNewSource(record.get());
                    parser.setLexerSource(lexerSource);
                    parser.parseNext(pAd);
                    Map<CaseInsensitiveString, ExprTree> attrs = pAd.getAttrList();
                    for (Entry<CaseInsensitiveString, ExprTree> entry : attrs.entrySet()) {
                        ExprTree tree = entry.getValue();
                        switch(tree.getKind()) {
                            case ATTRREF_NODE:
                            case CLASSAD_NODE:
                            case EXPR_ENVELOPE:
                            case EXPR_LIST_NODE:
                            case FN_CALL_NODE:
                            case OP_NODE:
                                break;
                            case LITERAL_NODE:
                                break;
                            default:
                                System.out.println("Something is wrong");
                                break;
                        }
                    }
                }
            } finally {
                recordReader.close();
            }
        }
    } catch (Exception e) {
        e.printStackTrace();
        assertTrue(false);
    }
}
Also used : Path(java.nio.file.Path) ClassAdObjectPool(org.apache.asterix.external.classad.object.pool.ClassAdObjectPool) ClassAd(org.apache.asterix.external.classad.ClassAd) HashMap(java.util.HashMap) CharArrayLexerSource(org.apache.asterix.external.classad.CharArrayLexerSource) ArrayList(java.util.ArrayList) CaseInsensitiveString(org.apache.asterix.external.classad.CaseInsensitiveString) CaseInsensitiveString(org.apache.asterix.external.classad.CaseInsensitiveString) HyracksDataException(org.apache.hyracks.api.exceptions.HyracksDataException) ClassAdParser(org.apache.asterix.external.library.ClassAdParser) FileSystemWatcher(org.apache.asterix.external.util.FileSystemWatcher) SemiStructuredRecordReader(org.apache.asterix.external.input.record.reader.stream.SemiStructuredRecordReader) Value(org.apache.asterix.external.classad.Value) ExprTree(org.apache.asterix.external.classad.ExprTree) LocalFSInputStream(org.apache.asterix.external.input.stream.LocalFSInputStream)

Aggregations

ExprTree (org.apache.asterix.external.classad.ExprTree)12 ClassAdParser (org.apache.asterix.external.library.ClassAdParser)5 AMutableString (org.apache.asterix.om.base.AMutableString)4 AMutableCharArrayString (org.apache.asterix.external.classad.AMutableCharArrayString)3 ClassAd (org.apache.asterix.external.classad.ClassAd)3 Value (org.apache.asterix.external.classad.Value)3 IAType (org.apache.asterix.om.types.IAType)3 MutableBoolean (org.apache.commons.lang3.mutable.MutableBoolean)3 ArrayBackedValueStorage (org.apache.hyracks.data.std.util.ArrayBackedValueStorage)3 ArrayList (java.util.ArrayList)2 CaseInsensitiveString (org.apache.asterix.external.classad.CaseInsensitiveString)2 HyracksDataException (org.apache.hyracks.api.exceptions.HyracksDataException)2 Path (java.nio.file.Path)1 BitSet (java.util.BitSet)1 HashMap (java.util.HashMap)1 TreeSet (java.util.TreeSet)1 IARecordBuilder (org.apache.asterix.builders.IARecordBuilder)1 OrderedListBuilder (org.apache.asterix.builders.OrderedListBuilder)1 UnorderedListBuilder (org.apache.asterix.builders.UnorderedListBuilder)1 CharArrayLexerSource (org.apache.asterix.external.classad.CharArrayLexerSource)1