Search in sources :

Example 11 with ClassAd

use of org.apache.asterix.external.classad.ClassAd in project asterixdb by apache.

the class ClassAdToADMTest method testSchemaless.

/**
     *
     */
public void testSchemaless() {
    try {
        ClassAdObjectPool objectPool = new ClassAdObjectPool();
        ClassAd pAd = new ClassAd(objectPool);
        String[] files = new String[] { "/classad/jobads.txt" };
        ClassAdParser parser = new ClassAdParser(objectPool);
        CharArrayLexerSource lexerSource = new CharArrayLexerSource();
        for (String path : files) {
            List<Path> paths = new ArrayList<>();
            Map<String, String> config = new HashMap<>();
            config.put(ExternalDataConstants.KEY_RECORD_START, "[");
            config.put(ExternalDataConstants.KEY_RECORD_END, "]");
            paths.add(Paths.get(getClass().getResource(path).toURI()));
            FileSystemWatcher watcher = new FileSystemWatcher(paths, null, false);
            LocalFSInputStream in = new LocalFSInputStream(watcher);
            SemiStructuredRecordReader recordReader = new SemiStructuredRecordReader();
            recordReader.configure(in, config);
            try {
                Value val = new Value(objectPool);
                while (recordReader.hasNext()) {
                    val.reset();
                    IRawRecord<char[]> record = recordReader.next();
                    lexerSource.setNewSource(record.get());
                    parser.setLexerSource(lexerSource);
                    parser.parseNext(pAd);
                    Map<CaseInsensitiveString, ExprTree> attrs = pAd.getAttrList();
                    for (Entry<CaseInsensitiveString, ExprTree> entry : attrs.entrySet()) {
                        ExprTree tree = entry.getValue();
                        switch(tree.getKind()) {
                            case ATTRREF_NODE:
                            case CLASSAD_NODE:
                            case EXPR_ENVELOPE:
                            case EXPR_LIST_NODE:
                            case FN_CALL_NODE:
                            case OP_NODE:
                                break;
                            case LITERAL_NODE:
                                break;
                            default:
                                System.out.println("Something is wrong");
                                break;
                        }
                    }
                }
            } finally {
                recordReader.close();
            }
        }
    } catch (Exception e) {
        e.printStackTrace();
        assertTrue(false);
    }
}
Also used : Path(java.nio.file.Path) ClassAdObjectPool(org.apache.asterix.external.classad.object.pool.ClassAdObjectPool) ClassAd(org.apache.asterix.external.classad.ClassAd) HashMap(java.util.HashMap) CharArrayLexerSource(org.apache.asterix.external.classad.CharArrayLexerSource) ArrayList(java.util.ArrayList) CaseInsensitiveString(org.apache.asterix.external.classad.CaseInsensitiveString) CaseInsensitiveString(org.apache.asterix.external.classad.CaseInsensitiveString) HyracksDataException(org.apache.hyracks.api.exceptions.HyracksDataException) ClassAdParser(org.apache.asterix.external.library.ClassAdParser) FileSystemWatcher(org.apache.asterix.external.util.FileSystemWatcher) SemiStructuredRecordReader(org.apache.asterix.external.input.record.reader.stream.SemiStructuredRecordReader) Value(org.apache.asterix.external.classad.Value) ExprTree(org.apache.asterix.external.classad.ExprTree) LocalFSInputStream(org.apache.asterix.external.input.stream.LocalFSInputStream)

Aggregations

ClassAd (org.apache.asterix.external.classad.ClassAd)11 Value (org.apache.asterix.external.classad.Value)6 ClassAdParser (org.apache.asterix.external.library.ClassAdParser)4 MutableBoolean (org.apache.commons.lang3.mutable.MutableBoolean)4 HyracksDataException (org.apache.hyracks.api.exceptions.HyracksDataException)4 ArrayList (java.util.ArrayList)3 AMutableCharArrayString (org.apache.asterix.external.classad.AMutableCharArrayString)3 ExprList (org.apache.asterix.external.classad.ExprList)3 ExprTree (org.apache.asterix.external.classad.ExprTree)3 ClassAdObjectPool (org.apache.asterix.external.classad.object.pool.ClassAdObjectPool)3 Path (java.nio.file.Path)2 HashMap (java.util.HashMap)2 CaseInsensitiveString (org.apache.asterix.external.classad.CaseInsensitiveString)2 CharArrayLexerSource (org.apache.asterix.external.classad.CharArrayLexerSource)2 TokenValue (org.apache.asterix.external.classad.TokenValue)2 SemiStructuredRecordReader (org.apache.asterix.external.input.record.reader.stream.SemiStructuredRecordReader)2 LocalFSInputStream (org.apache.asterix.external.input.stream.LocalFSInputStream)2 FileSystemWatcher (org.apache.asterix.external.util.FileSystemWatcher)2 AMutableDouble (org.apache.asterix.om.base.AMutableDouble)2 AMutableInt64 (org.apache.asterix.om.base.AMutableInt64)2