Search in sources :

Example 41 with MoveWork

use of org.apache.hadoop.hive.ql.plan.MoveWork in project hive by apache.

the class TestGenMapRedUtilsCreateConditionalTask method testMergePathWithInvalidMoveWorkThrowsException.

@Test(expected = IllegalArgumentException.class)
public void testMergePathWithInvalidMoveWorkThrowsException() {
    final Path condInputPath = new Path("s3a://bucket/scratch/-ext-10000");
    final MoveWork mockWork = mock(MoveWork.class);
    final LineageState lineageState = new LineageState();
    when(mockWork.getLoadMultiFilesWork()).thenReturn(new LoadMultiFilesDesc());
    GenMapRedUtils.mergeMovePaths(condInputPath, mockWork, lineageState);
}
Also used : Path(org.apache.hadoop.fs.Path) MoveWork(org.apache.hadoop.hive.ql.plan.MoveWork) LineageState(org.apache.hadoop.hive.ql.session.LineageState) LoadMultiFilesDesc(org.apache.hadoop.hive.ql.plan.LoadMultiFilesDesc) Test(org.junit.Test)

Example 42 with MoveWork

use of org.apache.hadoop.hive.ql.plan.MoveWork in project hive by apache.

the class TestGenMapRedUtilsCreateConditionalTask method createMoveTask.

private Task<MoveWork> createMoveTask(Path source, Path destination) {
    Task<MoveWork> moveTask = mock(MoveTask.class);
    MoveWork moveWork = new MoveWork();
    moveWork.setLoadFileWork(new LoadFileDesc(source, destination, true, null, null, false));
    when(moveTask.getWork()).thenReturn(moveWork);
    return moveTask;
}
Also used : MoveWork(org.apache.hadoop.hive.ql.plan.MoveWork) LoadFileDesc(org.apache.hadoop.hive.ql.plan.LoadFileDesc)

Aggregations

MoveWork (org.apache.hadoop.hive.ql.plan.MoveWork)42 Path (org.apache.hadoop.fs.Path)30 LoadTableDesc (org.apache.hadoop.hive.ql.plan.LoadTableDesc)24 LoadFileDesc (org.apache.hadoop.hive.ql.plan.LoadFileDesc)11 HiveException (org.apache.hadoop.hive.ql.metadata.HiveException)10 TableDesc (org.apache.hadoop.hive.ql.plan.TableDesc)8 Context (org.apache.hadoop.hive.ql.Context)7 ConditionalTask (org.apache.hadoop.hive.ql.exec.ConditionalTask)7 Task (org.apache.hadoop.hive.ql.exec.Task)7 Partition (org.apache.hadoop.hive.ql.metadata.Partition)7 FileSinkDesc (org.apache.hadoop.hive.ql.plan.FileSinkDesc)7 Test (org.junit.Test)7 ArrayList (java.util.ArrayList)6 BasicStatsWork (org.apache.hadoop.hive.ql.plan.BasicStatsWork)6 LoadMultiFilesDesc (org.apache.hadoop.hive.ql.plan.LoadMultiFilesDesc)6 StatsWork (org.apache.hadoop.hive.ql.plan.StatsWork)6 Serializable (java.io.Serializable)5 MetaException (org.apache.hadoop.hive.metastore.api.MetaException)5 MoveTask (org.apache.hadoop.hive.ql.exec.MoveTask)5 DDLWork (org.apache.hadoop.hive.ql.plan.DDLWork)5