Search in sources :

Example 6 with StorageException

use of edu.uci.ics.texera.api.exception.StorageException in project textdb by TextDB.

the class Utils method getTexeraHomePath.

/**
 * Gets the real path of the texera home directory by:
 *   1): check if the current directory is texera/core (where TEXERA_HOME should be),
 *   if it's not then:
 *   2): search the parents all the way up to find the texera home path
 *   if it's not then:
 *   3): search the siblings and children to find the texera home path
 *
 *   Finding texera home directory will fail
 *
 * @return the real absolute path to texera home directory
 * @throws StorageException if can not find texera home
 */
public static Path getTexeraHomePath() throws StorageException {
    if (TEXERA_HOME_PATH != null) {
        return TEXERA_HOME_PATH;
    }
    try {
        Path currentWorkingDirectory = Paths.get(".").toRealPath();
        // check if the current directory is the texera home path
        if (isTexeraHomePath(currentWorkingDirectory)) {
            TEXERA_HOME_PATH = currentWorkingDirectory;
            return currentWorkingDirectory;
        }
        // search parents all the way up to find texera home path
        Path searchParents = currentWorkingDirectory.getParent();
        while (searchParents.getParent() != null) {
            if (isTexeraHomePath(searchParents)) {
                TEXERA_HOME_PATH = searchParents;
                return searchParents;
            }
            searchParents = searchParents.getParent();
        }
        // from current path's parent directory, search itschildren to find texera home path
        // current max depth is set to 2 (current path's siblings and direct children)
        Optional<Path> searchChildren = Files.walk(currentWorkingDirectory.getParent(), 2).filter(path -> isTexeraHomePath(path)).findAny();
        if (searchChildren.isPresent()) {
            TEXERA_HOME_PATH = searchChildren.get();
            return searchChildren.get();
        }
        throw new StorageException("Finding texera home path failed. Current working directory is " + currentWorkingDirectory);
    } catch (IOException e) {
        throw new StorageException(e);
    }
}
Also used : Path(java.nio.file.Path) List(java.util.List) Files(java.nio.file.Files) Paths(java.nio.file.Paths) TexeraProject(edu.uci.ics.texera.api.constants.DataConstants.TexeraProject) Optional(java.util.Optional) IOException(java.io.IOException) DataConstants(edu.uci.ics.texera.api.constants.DataConstants) StorageException(edu.uci.ics.texera.api.exception.StorageException) Path(java.nio.file.Path) Collectors(java.util.stream.Collectors) IOException(java.io.IOException) StorageException(edu.uci.ics.texera.api.exception.StorageException)

Example 7 with StorageException

use of edu.uci.ics.texera.api.exception.StorageException in project textdb by TextDB.

the class MedlineIndexWriter method writeMedlineIndex.

public static void writeMedlineIndex(Path medlineFilepath, String tableName) throws IOException, StorageException, ParseException {
    RelationManager relationManager = RelationManager.getInstance();
    DataWriter dataWriter = relationManager.getTableDataWriter(tableName);
    dataWriter.open();
    BufferedReader reader = Files.newBufferedReader(medlineFilepath);
    String line;
    while ((line = reader.readLine()) != null) {
        try {
            dataWriter.insertTuple(recordToTuple(line));
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    reader.close();
    dataWriter.close();
}
Also used : BufferedReader(java.io.BufferedReader) IOException(java.io.IOException) StorageException(edu.uci.ics.texera.api.exception.StorageException) ParseException(java.text.ParseException) RelationManager(edu.uci.ics.texera.storage.RelationManager) DataWriter(edu.uci.ics.texera.storage.DataWriter)

Example 8 with StorageException

use of edu.uci.ics.texera.api.exception.StorageException in project textdb by TextDB.

the class RunTests method main.

/*
     * Write Indices Run all performance tests.
     * 
     * Passed in below arguments: 
     * file folder path (where data set stored)
     * result folder path (where performance test results stored) 
     * standard index folder path (where standard index stored) 
     * trigram index folder path(where trigram index stored) 
     * queries folder path (where query files stored)
     * 
     * If above arguments are not passed in, default paths will be used (refer
     * to PerfTestUtils.java) If some of the arguments are not applicable,
     * define them as empty string.
     * 
     * Make necessary changes for arguments, such as query file name, threshold
     * list, and regexQueries
     *
     */
public static void main(String[] args) {
    try {
        PerfTestUtils.setFileFolder(args[0]);
        PerfTestUtils.setResultFolder(args[1]);
        PerfTestUtils.setStandardIndexFolder(args[2]);
        PerfTestUtils.setTrigramIndexFolder(args[3]);
        PerfTestUtils.setQueryFolder(args[4]);
    } catch (ArrayIndexOutOfBoundsException e) {
        System.out.println("missing arguments will be set to default");
    }
    try {
        PerfTestUtils.deleteDirectory(new File(PerfTestUtils.standardIndexFolder));
        PerfTestUtils.deleteDirectory(new File(PerfTestUtils.trigramIndexFolder));
        PerfTestUtils.writeStandardAnalyzerIndices();
        PerfTestUtils.writeTrigramIndices();
        List<Double> thresholds = Arrays.asList(0.8, 0.65, 0.5, 0.35);
        List<String> regexQueries = Arrays.asList("mosquitos?", "v[ir]{2}[us]{2}", "market(ing)?", "medic(ine|al|ation|are|aid)?", "[A-Z][aeiou|AEIOU][A-Za-z]*");
        KeywordMatcherPerformanceTest.runTest("sample_queries.txt");
        DictionaryMatcherPerformanceTest.runTest("sample_queries.txt");
        FuzzyTokenMatcherPerformanceTest.runTest("sample_queries.txt", thresholds);
        RegexMatcherPerformanceTest.runTest(regexQueries);
        NlpExtractorPerformanceTest.runTest();
    } catch (StorageException | DataflowException | IOException e) {
        e.printStackTrace();
    } catch (Exception e) {
        e.printStackTrace();
    }
}
Also used : DataflowException(edu.uci.ics.texera.api.exception.DataflowException) IOException(java.io.IOException) File(java.io.File) StorageException(edu.uci.ics.texera.api.exception.StorageException) DataflowException(edu.uci.ics.texera.api.exception.DataflowException) IOException(java.io.IOException) StorageException(edu.uci.ics.texera.api.exception.StorageException)

Example 9 with StorageException

use of edu.uci.ics.texera.api.exception.StorageException in project textdb by TextDB.

the class DataWriter method close.

public void close() throws StorageException {
    if (this.luceneIndexWriter != null) {
        try {
            this.luceneIndexWriter.close();
            this.isOpen = false;
        } catch (IOException e) {
            throw new StorageException(e.getMessage(), e);
        }
    }
}
Also used : IOException(java.io.IOException) StorageException(edu.uci.ics.texera.api.exception.StorageException)

Example 10 with StorageException

use of edu.uci.ics.texera.api.exception.StorageException in project textdb by TextDB.

the class DataWriter method open.

public void open() throws StorageException {
    if (this.luceneIndexWriter == null || !this.luceneIndexWriter.isOpen()) {
        try {
            Directory directory = FSDirectory.open(this.indexDirectory);
            IndexWriterConfig conf = new IndexWriterConfig(analyzer);
            this.luceneIndexWriter = new IndexWriter(directory, conf);
            this.isOpen = true;
        } catch (IOException e) {
            throw new StorageException(e.getMessage(), e);
        }
    }
}
Also used : IndexWriter(org.apache.lucene.index.IndexWriter) IOException(java.io.IOException) StorageException(edu.uci.ics.texera.api.exception.StorageException) Directory(org.apache.lucene.store.Directory) FSDirectory(org.apache.lucene.store.FSDirectory) IndexWriterConfig(org.apache.lucene.index.IndexWriterConfig)

Aggregations

StorageException (edu.uci.ics.texera.api.exception.StorageException)24 IOException (java.io.IOException)16 Tuple (edu.uci.ics.texera.api.tuple.Tuple)8 Schema (edu.uci.ics.texera.api.schema.Schema)4 Term (org.apache.lucene.index.Term)4 MatchAllDocsQuery (org.apache.lucene.search.MatchAllDocsQuery)4 Query (org.apache.lucene.search.Query)4 TermQuery (org.apache.lucene.search.TermQuery)4 JsonNode (com.fasterxml.jackson.databind.JsonNode)3 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)3 DataflowException (edu.uci.ics.texera.api.exception.DataflowException)3 IDField (edu.uci.ics.texera.api.field.IDField)3 IField (edu.uci.ics.texera.api.field.IField)3 DataWriter (edu.uci.ics.texera.storage.DataWriter)3 StringField (edu.uci.ics.texera.api.field.StringField)2 Attribute (edu.uci.ics.texera.api.schema.Attribute)2 File (java.io.File)2 Path (java.nio.file.Path)2 ParseException (java.text.ParseException)2 Analyzer (org.apache.lucene.analysis.Analyzer)2