Search in sources :

Example 6 with CarbonProjection

use of org.apache.carbondata.hadoop.CarbonProjection in project carbondata by apache.

the class CarbonTableInputFormatTest method testInputFormatMapperReadAllRowsAndFewColumnsWithFilter.

@Test
public void testInputFormatMapperReadAllRowsAndFewColumnsWithFilter() throws Exception {
    try {
        String outPath = "target/output3";
        CarbonProjection carbonProjection = new CarbonProjection();
        carbonProjection.addColumn("ID");
        carbonProjection.addColumn("country");
        carbonProjection.addColumn("salary");
        Expression expression = new EqualToExpression(new ColumnExpression("country", DataTypes.STRING), new LiteralExpression("france", DataTypes.STRING));
        runJob(outPath, carbonProjection, expression);
        Assert.assertEquals("Count lines are not matching", 101, countTheLines(outPath));
        Assert.assertEquals("Column count are not matching", 3, countTheColumns(outPath));
    } catch (Exception e) {
        Assert.assertTrue("failed", false);
    } finally {
        StoreCreator.clearDataMaps();
    }
}
Also used : CarbonProjection(org.apache.carbondata.hadoop.CarbonProjection) EqualToExpression(org.apache.carbondata.core.scan.expression.conditional.EqualToExpression) ColumnExpression(org.apache.carbondata.core.scan.expression.ColumnExpression) Expression(org.apache.carbondata.core.scan.expression.Expression) EqualToExpression(org.apache.carbondata.core.scan.expression.conditional.EqualToExpression) LiteralExpression(org.apache.carbondata.core.scan.expression.LiteralExpression) ColumnExpression(org.apache.carbondata.core.scan.expression.ColumnExpression) LiteralExpression(org.apache.carbondata.core.scan.expression.LiteralExpression) IOException(java.io.IOException) Test(org.junit.Test)

Example 7 with CarbonProjection

use of org.apache.carbondata.hadoop.CarbonProjection in project carbondata by apache.

the class CarbonTableInputFormatTest method testInputFormatMapperReadAllRowsAndFewColumns.

@Test
public void testInputFormatMapperReadAllRowsAndFewColumns() throws Exception {
    try {
        String outPath = "target/output2";
        CarbonProjection carbonProjection = new CarbonProjection();
        carbonProjection.addColumn("ID");
        carbonProjection.addColumn("country");
        carbonProjection.addColumn("salary");
        runJob(outPath, carbonProjection, null);
        Assert.assertEquals("Count lines are not matching", 1000, countTheLines(outPath));
        Assert.assertEquals("Column count are not matching", 3, countTheColumns(outPath));
    } catch (Exception e) {
        e.printStackTrace();
        Assert.assertTrue("failed", false);
    } finally {
        StoreCreator.clearDataMaps();
    }
}
Also used : CarbonProjection(org.apache.carbondata.hadoop.CarbonProjection) IOException(java.io.IOException) Test(org.junit.Test)

Example 8 with CarbonProjection

use of org.apache.carbondata.hadoop.CarbonProjection in project carbondata by apache.

the class CarbonTableInputFormatTest method testInputFormatMapperReadAllRowsAndColumns.

@Test
public void testInputFormatMapperReadAllRowsAndColumns() throws Exception {
    try {
        String outPath = "target/output";
        CarbonProjection carbonProjection = new CarbonProjection();
        carbonProjection.addColumn("ID");
        carbonProjection.addColumn("date");
        carbonProjection.addColumn("country");
        carbonProjection.addColumn("name");
        carbonProjection.addColumn("phonetype");
        carbonProjection.addColumn("serialname");
        carbonProjection.addColumn("salary");
        runJob(outPath, carbonProjection, null);
        Assert.assertEquals("Count lines are not matching", 1000, countTheLines(outPath));
        Assert.assertEquals("Column count are not matching", 7, countTheColumns(outPath));
    } catch (Exception e) {
        e.printStackTrace();
        Assert.assertTrue("failed", false);
        throw e;
    } finally {
        StoreCreator.clearDataMaps();
    }
}
Also used : CarbonProjection(org.apache.carbondata.hadoop.CarbonProjection) IOException(java.io.IOException) Test(org.junit.Test)

Aggregations

CarbonProjection (org.apache.carbondata.hadoop.CarbonProjection)8 IOException (java.io.IOException)7 Test (org.junit.Test)6 CarbonTable (org.apache.carbondata.core.metadata.schema.table.CarbonTable)2 ColumnExpression (org.apache.carbondata.core.scan.expression.ColumnExpression)2 Expression (org.apache.carbondata.core.scan.expression.Expression)2 LiteralExpression (org.apache.carbondata.core.scan.expression.LiteralExpression)2 EqualToExpression (org.apache.carbondata.core.scan.expression.conditional.EqualToExpression)2 Configuration (org.apache.hadoop.conf.Configuration)2 ColumnHandle (com.facebook.presto.spi.ColumnHandle)1 ImmutableList (com.google.common.collect.ImmutableList)1 ArrayList (java.util.ArrayList)1 QueryModel (org.apache.carbondata.core.scan.model.QueryModel)1 CarbonInputSplit (org.apache.carbondata.hadoop.CarbonInputSplit)1 CarbonFileInputFormat (org.apache.carbondata.hadoop.api.CarbonFileInputFormat)1 CarbonTableInputFormat (org.apache.carbondata.hadoop.api.CarbonTableInputFormat)1 CarbonTableCacheModel (org.apache.carbondata.presto.impl.CarbonTableCacheModel)1 JobConf (org.apache.hadoop.mapred.JobConf)1 TaskAttemptContextImpl (org.apache.hadoop.mapred.TaskAttemptContextImpl)1 TaskAttemptID (org.apache.hadoop.mapred.TaskAttemptID)1