Search in sources :

Example 6 with HdfsTestService

use of org.apache.hudi.common.testutils.minicluster.HdfsTestService in project hudi by apache.

the class TestCluster method setup.

public void setup() throws Exception {
    hdfsTestService = new HdfsTestService();
    dfsCluster = hdfsTestService.start(true);
    conf = hdfsTestService.getHadoopConf();
    conf.setInt(ConfVars.METASTORE_SERVER_PORT.varname, port++);
    conf.setInt(ConfVars.HIVE_SERVER2_THRIFT_PORT.varname, port++);
    conf.setInt(ConfVars.HIVE_SERVER2_WEBUI_PORT.varname, port++);
    hiveTestService = new HiveTestService(conf);
    server2 = hiveTestService.start();
    dtfOut = DateTimeFormatter.ofPattern("yyyy/MM/dd");
    hiveSiteXml = File.createTempFile("hive-site", ".xml");
    hiveSiteXml.deleteOnExit();
    try (OutputStream os = new FileOutputStream(hiveSiteXml)) {
        hiveTestService.getServerConf().writeXml(os);
    }
    client = HiveMetaStoreClient.newSynchronizedClient(RetryingMetaStoreClient.getProxy(hiveTestService.getServerConf(), true));
}
Also used : FSDataOutputStream(org.apache.hadoop.fs.FSDataOutputStream) OutputStream(java.io.OutputStream) FileOutputStream(java.io.FileOutputStream) FileOutputStream(java.io.FileOutputStream) HdfsTestService(org.apache.hudi.common.testutils.minicluster.HdfsTestService)

Aggregations

HdfsTestService (org.apache.hudi.common.testutils.minicluster.HdfsTestService)6 Path (org.apache.hadoop.fs.Path)3 BeforeAll (org.junit.jupiter.api.BeforeAll)2 FileOutputStream (java.io.FileOutputStream)1 OutputStream (java.io.OutputStream)1 FSDataOutputStream (org.apache.hadoop.fs.FSDataOutputStream)1 HoodieSparkEngineContext (org.apache.hudi.client.common.HoodieSparkEngineContext)1 ZookeeperTestService (org.apache.hudi.common.testutils.minicluster.ZookeeperTestService)1 HiveTestService (org.apache.hudi.hive.testutils.HiveTestService)1 SparkConf (org.apache.spark.SparkConf)1 JavaSparkContext (org.apache.spark.api.java.JavaSparkContext)1 BeforeEach (org.junit.jupiter.api.BeforeEach)1