Examples with DatasetSpecification co.cask.cdap.api.dataset.DatasetSpecification used on opensource projects

Example 31 with DatasetSpecification

use of co.cask.cdap.api.dataset.DatasetSpecification in project cdap by caskdata.

the class RemoteDatasetOpExecutor method create.

@Override
public DatasetSpecification create(DatasetId datasetInstanceId, DatasetTypeMeta typeMeta, DatasetProperties props) throws Exception {
    InternalDatasetCreationParams creationParams = new InternalDatasetCreationParams(typeMeta, props);
    HttpResponse response = doRequest(datasetInstanceId, "create", GSON.toJson(creationParams));
    return ObjectResponse.fromJsonBody(response, DatasetSpecification.class).getResponseObject();
}

Example 32 with DatasetSpecification

use of co.cask.cdap.api.dataset.DatasetSpecification in project cdap by caskdata.

the class CubeDatasetAdmin method update.

@Override
public void update(DatasetSpecification oldSpec) throws IOException {
    // update all existing resolution tables, create all new resolutions
    for (Map.Entry<String, DatasetSpecification> entry : spec.getSpecifications().entrySet()) {
        DatasetSpecification oldSubSpec = spec.getSpecification(entry.getKey());
        DatasetAdmin subAdmin = delegates.get(entry.getKey());
        if (oldSubSpec != null && subAdmin instanceof Updatable) {
            ((Updatable) subAdmin).update(oldSubSpec);
        } else if (oldSubSpec == null) {
            subAdmin.create();
        }
    }
// TODO (CDAP-6342) delete all resolutions that were removed as part of the update
}

Example 33 with DatasetSpecification

use of co.cask.cdap.api.dataset.DatasetSpecification in project cdap by caskdata.

the class DatasetSerDe method getDatasetSchema.

private void getDatasetSchema(Configuration conf, DatasetId datasetId) throws SerDeException {
    try (ContextManager.Context hiveContext = ContextManager.getContext(conf)) {
        // Because it calls initialize just to get the object inspector
        if (hiveContext == null) {
            LOG.info("Hive provided a null conf, will not be able to get dataset schema.");
            return;
        }
        // some datasets like Table and ObjectMappedTable have schema in the dataset properties
        try {
            DatasetSpecification datasetSpec = hiveContext.getDatasetSpec(datasetId);
            String schemaStr = datasetSpec.getProperty("schema");
            if (schemaStr != null) {
                schema = Schema.parseJson(schemaStr);
                return;
            }
        } catch (DatasetManagementException | ServiceUnavailableException e) {
            throw new SerDeException("Could not instantiate dataset " + datasetId, e);
        } catch (IOException e) {
            throw new SerDeException("Exception getting schema for dataset " + datasetId, e);
        }
        // other datasets must be instantiated to get their schema
        // conf is null if this is a query that writes to a dataset
        ClassLoader parentClassLoader = conf == null ? null : conf.getClassLoader();
        try (SystemDatasetInstantiator datasetInstantiator = hiveContext.createDatasetInstantiator(parentClassLoader)) {
            Dataset dataset = datasetInstantiator.getDataset(datasetId);
            if (dataset == null) {
                throw new SerDeException("Could not find dataset " + datasetId);
            }
            Type recordType;
            if (dataset instanceof RecordScannable) {
                recordType = ((RecordScannable) dataset).getRecordType();
            } else if (dataset instanceof RecordWritable) {
                recordType = ((RecordWritable) dataset).getRecordType();
            } else {
                throw new SerDeException("Dataset " + datasetId + " is not explorable.");
            }
            schema = schemaGenerator.generate(recordType);
        } catch (UnsupportedTypeException e) {
            throw new SerDeException("Dataset " + datasetId + " has an unsupported schema.", e);
        } catch (IOException e) {
            throw new SerDeException("Exception while trying to instantiate dataset " + datasetId, e);
        }
    } catch (IOException e) {
        throw new SerDeException("Could not get hive context from configuration.", e);
    }
}

Example 34 with DatasetSpecification

use of co.cask.cdap.api.dataset.DatasetSpecification in project cdap by caskdata.

the class ExploreExecutorHttpHandler method disableDataset.

/**
 * Disable ad-hoc exploration of a dataset instance.
 */
@POST
@Path("datasets/{dataset}/disable")
public void disableDataset(HttpRequest request, HttpResponder responder, @PathParam("namespace-id") String namespace, @PathParam("dataset") String datasetName) {
    final DatasetId datasetId = new DatasetId(namespace, datasetName);
    DatasetSpecification datasetSpec = retrieveDatasetSpec(responder, datasetId);
    if (datasetSpec == null) {
        // this means the spec could not be retrieved and retrievedDatasetSpec() already responded
        return;
    }
    disableDataset(responder, datasetId, datasetSpec);
}

Example 35 with DatasetSpecification

use of co.cask.cdap.api.dataset.DatasetSpecification in project cdap by caskdata.

the class ExploreExecutorHttpHandler method enableDataset.

/**
 * Enable ad-hoc exploration of a dataset instance.
 */
@POST
@Path("datasets/{dataset}/enable")
public void enableDataset(HttpRequest request, HttpResponder responder, @PathParam("namespace-id") String namespace, @PathParam("dataset") String datasetName) {
    DatasetId datasetId = new DatasetId(namespace, datasetName);
    DatasetSpecification datasetSpec = retrieveDatasetSpec(responder, datasetId);
    if (datasetSpec == null) {
        // this means the spec could not be retrieved and retrievedDatasetSpec() already responded
        return;
    }
    enableDataset(responder, datasetId, datasetSpec, false);
}