Search in sources :

Example 96 with Config

use of edu.neu.ccs.pyramid.configuration.Config in project pyramid by cheng-li.

the class Regex method main.

public static void main(String[] args) {
    if (args.length != 1) {
        throw new IllegalArgumentException("Please specify a properties file.");
    }
    Config config = new Config(args[0]);
    System.out.println(config);
    boolean match = config.getString("string").matches(config.getString("regularExpression"));
    if (match) {
        System.out.println("match!");
    } else {
        System.out.println("not match!");
    }
}
Also used : Config(edu.neu.ccs.pyramid.configuration.Config)

Example 97 with Config

use of edu.neu.ccs.pyramid.configuration.Config in project pyramid by cheng-li.

the class IndexChecker method main.

public static void main(String[] args) throws Exception {
    if (args.length != 1) {
        throw new IllegalArgumentException("Please specify a properties file.");
    }
    Config config = new Config(args[0]);
    System.out.println(config);
    ESIndex index = loadIndex(config);
    List<String> fields = config.getStrings("fieldsToCheck");
    for (String field : fields) {
        check(index, field);
    }
    for (String field : fields) {
        checkEmpty(index, field);
    }
    index.close();
}
Also used : Config(edu.neu.ccs.pyramid.configuration.Config) ESIndex(edu.neu.ccs.pyramid.elasticsearch.ESIndex)

Example 98 with Config

use of edu.neu.ccs.pyramid.configuration.Config in project pyramid by cheng-li.

the class MLTrec2LibSvm method main.

public static void main(String[] args) throws Exception {
    Config config = new Config(args[0]);
    System.out.println(config);
    List<String> trecs = config.getStrings("trec");
    List<String> libSVMs = config.getStrings("libSVM");
    for (int i = 0; i < trecs.size(); i++) {
        MultiLabelClfDataSet trecDataset = TRECFormat.loadMultiLabelClfDataSet(new File(trecs.get(i)), DataSetType.ML_CLF_SPARSE, false);
        System.out.println(i + " -- Translating on trecs: " + trecs.get(i));
        LibSvmFormat.save(trecDataset, libSVMs.get(i));
    }
}
Also used : Config(edu.neu.ccs.pyramid.configuration.Config) File(java.io.File)

Example 99 with Config

use of edu.neu.ccs.pyramid.configuration.Config in project pyramid by cheng-li.

the class ZeroOutFeatures method main.

public static void main(String[] args) throws Exception {
    if (args.length != 1) {
        throw new IllegalArgumentException("Please specify a properties file.");
    }
    Config config = new Config(args[0]);
    MultiLabelClfDataSet dataSet = TRECFormat.loadMultiLabelClfDataSet(config.getString("inputData"), DataSetType.ML_CLF_SPARSE, true);
    List<Integer> indices = parse(config.getString("featureIndices"));
    System.out.println("features to zero out = " + indices);
    for (int j : indices) {
        List<Integer> nonZeros = new ArrayList<>();
        for (Vector.Element nonZero : dataSet.getColumn(j).nonZeroes()) {
            nonZeros.add(nonZero.index());
        }
        for (int i : nonZeros) {
            dataSet.setFeatureValue(i, j, 0);
        }
    }
    TRECFormat.save(dataSet, config.getString("outputData"));
}
Also used : Config(edu.neu.ccs.pyramid.configuration.Config) ArrayList(java.util.ArrayList) Vector(org.apache.mahout.math.Vector) MultiLabelClfDataSet(edu.neu.ccs.pyramid.dataset.MultiLabelClfDataSet)

Example 100 with Config

use of edu.neu.ccs.pyramid.configuration.Config in project pyramid by cheng-li.

the class FeatureBinarizer method main.

public static void main(String[] args) throws Exception {
    Config config = new Config(args[0]);
    System.out.println(config);
    List<String> inputs = config.getStrings("input.trecFolders");
    List<String> outputs = config.getStrings("output.trecFolders");
    if (inputs.size() != outputs.size()) {
        throw new IllegalArgumentException("inputs.size()!=outputs.size()");
    }
    for (int i = 0; i < inputs.size(); i++) {
        String input = inputs.get(i);
        String output = outputs.get(i);
        binarize(config, input, output);
    }
}
Also used : Config(edu.neu.ccs.pyramid.configuration.Config)

Aggregations

Config (edu.neu.ccs.pyramid.configuration.Config)119 File (java.io.File)68 Collectors (java.util.stream.Collectors)40 FileUtils (org.apache.commons.io.FileUtils)40 Paths (java.nio.file.Paths)39 IntStream (java.util.stream.IntStream)37 Pair (edu.neu.ccs.pyramid.util.Pair)36 Serialization (edu.neu.ccs.pyramid.util.Serialization)35 StopWatch (org.apache.commons.lang3.time.StopWatch)34 MLMeasures (edu.neu.ccs.pyramid.eval.MLMeasures)33 BufferedWriter (java.io.BufferedWriter)32 FileWriter (java.io.FileWriter)32 java.util (java.util)32 edu.neu.ccs.pyramid.dataset (edu.neu.ccs.pyramid.dataset)31 MultiLabelClassifier (edu.neu.ccs.pyramid.multilabel_classification.MultiLabelClassifier)29 EarlyStopper (edu.neu.ccs.pyramid.optimization.EarlyStopper)28 PrintUtil (edu.neu.ccs.pyramid.util.PrintUtil)26 edu.neu.ccs.pyramid.multilabel_classification.cbm (edu.neu.ccs.pyramid.multilabel_classification.cbm)25 ListUtil (edu.neu.ccs.pyramid.util.ListUtil)25 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)22