Search in sources :

Example 1 with ClassPathResource

use of org.deeplearning4j.ui.standalone.ClassPathResource in project deeplearning4j by deeplearning4j.

the class UITest method testPosting.

@Test
public void testPosting() throws Exception {
    //        File inputFile = new ClassPathResource("/big/raw_sentences.txt").getFile();
    File inputFile = new ClassPathResource("/basic/word2vec_advance.txt").getFile();
    SentenceIterator iter = UimaSentenceIterator.createWithPath(inputFile.getAbsolutePath());
    // Split on white spaces in the line to get words
    TokenizerFactory t = new DefaultTokenizerFactory();
    t.setTokenPreProcessor(new CommonPreprocessor());
    Word2Vec vec = new Word2Vec.Builder().minWordFrequency(1).iterations(1).epochs(1).layerSize(20).stopWords(new ArrayList<String>()).useAdaGrad(false).negativeSample(5).seed(42).windowSize(5).iterate(iter).tokenizerFactory(t).build();
    vec.fit();
    File tempFile = File.createTempFile("temp", "w2v");
    tempFile.deleteOnExit();
    WordVectorSerializer.writeWordVectors(vec, tempFile);
    WordVectors vectors = WordVectorSerializer.loadTxtVectors(tempFile);
    //Initialize
    UIServer.getInstance();
    UiConnectionInfo uiConnectionInfo = new UiConnectionInfo.Builder().setAddress("localhost").setPort(9000).build();
    BarnesHutTsne tsne = new BarnesHutTsne.Builder().normalize(false).setFinalMomentum(0.8f).numDimension(2).setMaxIter(10).build();
    vectors.lookupTable().plotVocab(tsne, vectors.lookupTable().getVocabCache().numWords(), uiConnectionInfo);
    Thread.sleep(100000);
}
Also used : BarnesHutTsne(org.deeplearning4j.plot.BarnesHutTsne) TokenizerFactory(org.deeplearning4j.text.tokenization.tokenizerfactory.TokenizerFactory) DefaultTokenizerFactory(org.deeplearning4j.text.tokenization.tokenizerfactory.DefaultTokenizerFactory) UiConnectionInfo(org.deeplearning4j.ui.UiConnectionInfo) ArrayList(java.util.ArrayList) ClassPathResource(org.deeplearning4j.ui.standalone.ClassPathResource) UimaSentenceIterator(org.deeplearning4j.text.sentenceiterator.UimaSentenceIterator) SentenceIterator(org.deeplearning4j.text.sentenceiterator.SentenceIterator) DefaultTokenizerFactory(org.deeplearning4j.text.tokenization.tokenizerfactory.DefaultTokenizerFactory) CommonPreprocessor(org.deeplearning4j.text.tokenization.tokenizer.preprocessor.CommonPreprocessor) Word2Vec(org.deeplearning4j.models.word2vec.Word2Vec) WordVectors(org.deeplearning4j.models.embeddings.wordvectors.WordVectors) File(java.io.File) Test(org.junit.Test)

Aggregations

File (java.io.File)1 ArrayList (java.util.ArrayList)1 WordVectors (org.deeplearning4j.models.embeddings.wordvectors.WordVectors)1 Word2Vec (org.deeplearning4j.models.word2vec.Word2Vec)1 BarnesHutTsne (org.deeplearning4j.plot.BarnesHutTsne)1 SentenceIterator (org.deeplearning4j.text.sentenceiterator.SentenceIterator)1 UimaSentenceIterator (org.deeplearning4j.text.sentenceiterator.UimaSentenceIterator)1 CommonPreprocessor (org.deeplearning4j.text.tokenization.tokenizer.preprocessor.CommonPreprocessor)1 DefaultTokenizerFactory (org.deeplearning4j.text.tokenization.tokenizerfactory.DefaultTokenizerFactory)1 TokenizerFactory (org.deeplearning4j.text.tokenization.tokenizerfactory.TokenizerFactory)1 UiConnectionInfo (org.deeplearning4j.ui.UiConnectionInfo)1 ClassPathResource (org.deeplearning4j.ui.standalone.ClassPathResource)1 Test (org.junit.Test)1