Examples with BasicLineIterator org.deeplearning4j.text.sentenceiterator.BasicLineIterator used on opensource projects

Example 36 with BasicLineIterator

use of org.deeplearning4j.text.sentenceiterator.BasicLineIterator in project deeplearning4j by deeplearning4j.

the class BasicLabelAwareIteratorTest method testHasNextDocument1.

@Test
public void testHasNextDocument1() throws Exception {
    File inputFile = new ClassPathResource("/big/raw_sentences.txt").getFile();
    SentenceIterator iter = new BasicLineIterator(inputFile.getAbsolutePath());
    BasicLabelAwareIterator iterator = new BasicLabelAwareIterator.Builder(iter).setLabelTemplate("DOCZ_").build();
    int cnt = 0;
    while (iterator.hasNextDocument()) {
        iterator.nextDocument();
        cnt++;
    }
    assertEquals(97162, cnt);
    LabelsSource generator = iterator.getLabelsSource();
    assertEquals(97162, generator.getLabels().size());
    assertEquals("DOCZ_0", generator.getLabels().get(0));
}