Search in sources :

Example 1 with LineSentenceIterator

use of org.deeplearning4j.text.sentenceiterator.LineSentenceIterator in project deeplearning4j by deeplearning4j.

the class GloveTest method before.

@Before
public void before() throws Exception {
    ClassPathResource resource = new ClassPathResource("/raw_sentences.txt");
    File file = resource.getFile();
    iter = new LineSentenceIterator(file);
    iter.setPreProcessor(new SentencePreProcessor() {

        @Override
        public String preProcess(String sentence) {
            return sentence.toLowerCase();
        }
    });
}
Also used : LineSentenceIterator(org.deeplearning4j.text.sentenceiterator.LineSentenceIterator) SentencePreProcessor(org.deeplearning4j.text.sentenceiterator.SentencePreProcessor) File(java.io.File) ClassPathResource(org.datavec.api.util.ClassPathResource) Before(org.junit.Before)

Aggregations

File (java.io.File)1 ClassPathResource (org.datavec.api.util.ClassPathResource)1 LineSentenceIterator (org.deeplearning4j.text.sentenceiterator.LineSentenceIterator)1 SentencePreProcessor (org.deeplearning4j.text.sentenceiterator.SentencePreProcessor)1 Before (org.junit.Before)1