Examples with SeqClassifierFlags edu.stanford.nlp.sequences.SeqClassifierFlags used on opensource projects

Example 6 with SeqClassifierFlags

use of edu.stanford.nlp.sequences.SeqClassifierFlags in project CoreNLP by stanfordnlp.

the class ChineseStringUtilsTest method testMultithreadedCombineSegmentedSentence.

/**
   * A small test with stubbed data that is meant to expose multithreading initialization errors
   * in combineSegmentedSentence.
   *
   * In my testing this reliably reproduces the crash seen in the issue:
   * https://github.com/stanfordnlp/CoreNLP/issues/263
   *
   * @throws Exception Various exceptions including Interrupted, all of which should be handled by
   *                   failing the test.
   */
public void testMultithreadedCombineSegmentedSentence() throws Exception {
    SeqClassifierFlags flags = createTestFlags();
    List<CoreLabel> labels = createTestTokens();
    List<Future<Boolean>> tasks = new ArrayList<>(THREADS);
    ExecutorService executor = Executors.newFixedThreadPool(THREADS);
    for (int v = 0; v < THREADS; v++) {
        Future<Boolean> f = executor.submit(() -> {
            for (int i = 0; i < SEGMENT_ATTEMPTS_PER_THREAD; i++) {
                ChineseStringUtils.combineSegmentedSentence(labels, flags);
            }
            return true;
        });
        tasks.add(f);
    }
    for (Future<Boolean> task : tasks) {
        // multithreading issues (generally NPEs) were thrown during the test.
        assert (task.get());
    }
}