Search in sources :

Example 1 with Book

use of net.jgp.labs.spark.x.model.Book in project net.jgp.labs.spark by jgperrin.

the class CsvToDatasetBook method start.

private void start() {
    SparkSession spark = SparkSession.builder().appName("CSV to Dataset<Book>").master("local").getOrCreate();
    String filename = "data/books.csv";
    Dataset<Row> df = spark.read().format("csv").option("inferSchema", "true").option("header", "true").load(filename);
    df.show();
    Dataset<Book> bookDf = df.map(new BookMapper(), Encoders.bean(Book.class));
    bookDf.show();
}
Also used : SparkSession(org.apache.spark.sql.SparkSession) Book(net.jgp.labs.spark.x.model.Book) Row(org.apache.spark.sql.Row)

Aggregations

Book (net.jgp.labs.spark.x.model.Book)1 Row (org.apache.spark.sql.Row)1 SparkSession (org.apache.spark.sql.SparkSession)1