Search in sources :

Example 1 with StoreCreator

use of org.apache.carbondata.hadoop.testutil.StoreCreator in project carbondata by apache.

the class CarbonStreamOutputFormatTest method setUp.

@Override
protected void setUp() throws Exception {
    super.setUp();
    JobID jobId = CarbonInputFormatUtil.getJobId(0);
    TaskID taskId = new TaskID(jobId, TaskType.MAP, 0);
    taskAttemptId = new TaskAttemptID(taskId, 0);
    hadoopConf = new Configuration();
    hadoopConf.set("mapred.job.id", jobId.toString());
    hadoopConf.set("mapred.tip.id", taskAttemptId.getTaskID().toString());
    hadoopConf.set("mapred.task.id", taskAttemptId.toString());
    hadoopConf.setBoolean("mapred.task.is.map", true);
    hadoopConf.setInt("mapred.task.partition", 0);
    tablePath = new File("target/stream_output").getCanonicalPath();
    String dbName = "default";
    String tableName = "stream_table_output";
    AbsoluteTableIdentifier identifier = AbsoluteTableIdentifier.from(tablePath, new CarbonTableIdentifier(dbName, tableName, UUID.randomUUID().toString()));
    CarbonTable table = new StoreCreator(new File("target/store").getAbsolutePath(), new File("../hadoop/src/test/resources/data.csv").getCanonicalPath()).createTable(identifier);
    String factFilePath = new File("../hadoop/src/test/resources/data.csv").getCanonicalPath();
    carbonLoadModel = StoreCreator.buildCarbonLoadModel(table, factFilePath, identifier);
}
Also used : CarbonTable(org.apache.carbondata.core.metadata.schema.table.CarbonTable) TaskID(org.apache.hadoop.mapreduce.TaskID) StoreCreator(org.apache.carbondata.hadoop.testutil.StoreCreator) Configuration(org.apache.hadoop.conf.Configuration) CarbonTableIdentifier(org.apache.carbondata.core.metadata.CarbonTableIdentifier) AbsoluteTableIdentifier(org.apache.carbondata.core.metadata.AbsoluteTableIdentifier) TaskAttemptID(org.apache.hadoop.mapreduce.TaskAttemptID) File(java.io.File) JobID(org.apache.hadoop.mapreduce.JobID)

Aggregations

File (java.io.File)1 AbsoluteTableIdentifier (org.apache.carbondata.core.metadata.AbsoluteTableIdentifier)1 CarbonTableIdentifier (org.apache.carbondata.core.metadata.CarbonTableIdentifier)1 CarbonTable (org.apache.carbondata.core.metadata.schema.table.CarbonTable)1 StoreCreator (org.apache.carbondata.hadoop.testutil.StoreCreator)1 Configuration (org.apache.hadoop.conf.Configuration)1 JobID (org.apache.hadoop.mapreduce.JobID)1 TaskAttemptID (org.apache.hadoop.mapreduce.TaskAttemptID)1 TaskID (org.apache.hadoop.mapreduce.TaskID)1