Search in sources :

Example 1 with KMeansModel

use of org.tribuo.clustering.kmeans.KMeansModel in project ml-commons by opensearch-project.

the class ModelSerDeSerTest method testModelSerDeSerKMeans.

@Test
public void testModelSerDeSerKMeans() {
    KMeansParams params = KMeansParams.builder().build();
    KMeans kMeans = new KMeans(params);
    Model model = kMeans.train(constructKMeansDataFrame(100));
    KMeansModel kMeansModel = (KMeansModel) ModelSerDeSer.deserialize(model.getContent());
    byte[] serializedModel = ModelSerDeSer.serialize(kMeansModel);
    assertFalse(Arrays.equals(serializedModel, model.getContent()));
}
Also used : KMeansParams(org.opensearch.ml.common.parameter.KMeansParams) KMeansModel(org.tribuo.clustering.kmeans.KMeansModel) KMeans(org.opensearch.ml.engine.algorithms.clustering.KMeans) Model(org.opensearch.ml.common.parameter.Model) KMeansModel(org.tribuo.clustering.kmeans.KMeansModel) Test(org.junit.Test)

Example 2 with KMeansModel

use of org.tribuo.clustering.kmeans.KMeansModel in project ml-commons by opensearch-project.

the class KMeans method trainAndPredict.

@Override
public MLOutput trainAndPredict(DataFrame dataFrame) {
    MutableDataset<ClusterID> trainDataset = TribuoUtil.generateDataset(dataFrame, new ClusteringFactory(), "KMeans training and predicting data from opensearch", TribuoOutputType.CLUSTERID);
    Integer centroids = Optional.ofNullable(parameters.getCentroids()).orElse(DEFAULT_CENTROIDS);
    Integer iterations = Optional.ofNullable(parameters.getIterations()).orElse(DEFAULT_ITERATIONS);
    KMeansTrainer trainer = new KMeansTrainer(centroids, iterations, distance, numThreads, seed);
    // won't store model in index
    KMeansModel kMeansModel = trainer.train(trainDataset);
    List<Prediction<ClusterID>> predictions = kMeansModel.predict(trainDataset);
    List<Map<String, Object>> listClusterID = new ArrayList<>();
    predictions.forEach(e -> listClusterID.add(Collections.singletonMap("ClusterID", e.getOutput().getID())));
    return MLPredictionOutput.builder().predictionResult(DataFrameBuilder.load(listClusterID)).build();
}
Also used : KMeansModel(org.tribuo.clustering.kmeans.KMeansModel) ClusterID(org.tribuo.clustering.ClusterID) Prediction(org.tribuo.Prediction) ArrayList(java.util.ArrayList) KMeansTrainer(org.tribuo.clustering.kmeans.KMeansTrainer) ClusteringFactory(org.tribuo.clustering.ClusteringFactory) Map(java.util.Map)

Example 3 with KMeansModel

use of org.tribuo.clustering.kmeans.KMeansModel in project ml-commons by opensearch-project.

the class KMeans method predict.

@Override
public MLOutput predict(DataFrame dataFrame, Model model) {
    if (model == null) {
        throw new IllegalArgumentException("No model found for KMeans prediction.");
    }
    List<Prediction<ClusterID>> predictions;
    MutableDataset<ClusterID> predictionDataset = TribuoUtil.generateDataset(dataFrame, new ClusteringFactory(), "KMeans prediction data from opensearch", TribuoOutputType.CLUSTERID);
    KMeansModel kMeansModel = (KMeansModel) ModelSerDeSer.deserialize(model.getContent());
    predictions = kMeansModel.predict(predictionDataset);
    List<Map<String, Object>> listClusterID = new ArrayList<>();
    predictions.forEach(e -> listClusterID.add(Collections.singletonMap("ClusterID", e.getOutput().getID())));
    return MLPredictionOutput.builder().predictionResult(DataFrameBuilder.load(listClusterID)).build();
}
Also used : KMeansModel(org.tribuo.clustering.kmeans.KMeansModel) ClusterID(org.tribuo.clustering.ClusterID) Prediction(org.tribuo.Prediction) ArrayList(java.util.ArrayList) ClusteringFactory(org.tribuo.clustering.ClusteringFactory) Map(java.util.Map)

Example 4 with KMeansModel

use of org.tribuo.clustering.kmeans.KMeansModel in project ml-commons by opensearch-project.

the class KMeans method train.

@Override
public Model train(DataFrame dataFrame) {
    MutableDataset<ClusterID> trainDataset = TribuoUtil.generateDataset(dataFrame, new ClusteringFactory(), "KMeans training data from opensearch", TribuoOutputType.CLUSTERID);
    Integer centroids = Optional.ofNullable(parameters.getCentroids()).orElse(DEFAULT_CENTROIDS);
    Integer iterations = Optional.ofNullable(parameters.getIterations()).orElse(DEFAULT_ITERATIONS);
    KMeansTrainer trainer = new KMeansTrainer(centroids, iterations, distance, numThreads, seed);
    KMeansModel kMeansModel = trainer.train(trainDataset);
    Model model = new Model();
    model.setName(FunctionName.KMEANS.name());
    model.setVersion(1);
    model.setContent(ModelSerDeSer.serialize(kMeansModel));
    return model;
}
Also used : KMeansModel(org.tribuo.clustering.kmeans.KMeansModel) ClusterID(org.tribuo.clustering.ClusterID) Model(org.opensearch.ml.common.parameter.Model) KMeansModel(org.tribuo.clustering.kmeans.KMeansModel) KMeansTrainer(org.tribuo.clustering.kmeans.KMeansTrainer) ClusteringFactory(org.tribuo.clustering.ClusteringFactory)

Aggregations

KMeansModel (org.tribuo.clustering.kmeans.KMeansModel)4 ClusterID (org.tribuo.clustering.ClusterID)3 ClusteringFactory (org.tribuo.clustering.ClusteringFactory)3 ArrayList (java.util.ArrayList)2 Map (java.util.Map)2 Model (org.opensearch.ml.common.parameter.Model)2 Prediction (org.tribuo.Prediction)2 KMeansTrainer (org.tribuo.clustering.kmeans.KMeansTrainer)2 Test (org.junit.Test)1 KMeansParams (org.opensearch.ml.common.parameter.KMeansParams)1 KMeans (org.opensearch.ml.engine.algorithms.clustering.KMeans)1