Search in sources :

Example 1 with HoodieRemoteException

use of org.apache.hudi.exception.HoodieRemoteException in project hudi by apache.

the class TimelineServerBasedWriteMarkers method create.

@Override
protected Option<Path> create(String partitionPath, String dataFileName, IOType type, boolean checkIfExists) {
    HoodieTimer timer = new HoodieTimer().startTimer();
    String markerFileName = getMarkerFileName(dataFileName, type);
    Map<String, String> paramsMap = new HashMap<>();
    paramsMap.put(MARKER_DIR_PATH_PARAM, markerDirPath.toString());
    if (StringUtils.isNullOrEmpty(partitionPath)) {
        paramsMap.put(MARKER_NAME_PARAM, markerFileName);
    } else {
        paramsMap.put(MARKER_NAME_PARAM, partitionPath + "/" + markerFileName);
    }
    boolean success;
    try {
        success = executeRequestToTimelineServer(CREATE_MARKER_URL, paramsMap, new TypeReference<Boolean>() {
        }, RequestMethod.POST);
    } catch (IOException e) {
        throw new HoodieRemoteException("Failed to create marker file " + partitionPath + "/" + markerFileName, e);
    }
    LOG.info("[timeline-server-based] Created marker file " + partitionPath + "/" + markerFileName + " in " + timer.endTimer() + " ms");
    if (success) {
        return Option.of(new Path(FSUtils.getPartitionPath(markerDirPath, partitionPath), markerFileName));
    } else {
        return Option.empty();
    }
}
Also used : Path(org.apache.hadoop.fs.Path) HashMap(java.util.HashMap) HoodieTimer(org.apache.hudi.common.util.HoodieTimer) HoodieRemoteException(org.apache.hudi.exception.HoodieRemoteException) TypeReference(com.fasterxml.jackson.core.type.TypeReference) IOException(java.io.IOException)

Aggregations

TypeReference (com.fasterxml.jackson.core.type.TypeReference)1 IOException (java.io.IOException)1 HashMap (java.util.HashMap)1 Path (org.apache.hadoop.fs.Path)1 HoodieTimer (org.apache.hudi.common.util.HoodieTimer)1 HoodieRemoteException (org.apache.hudi.exception.HoodieRemoteException)1