Search in sources :

Example 46 with ApiKeyRequired

use of io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired in project hopsworks by logicalclocks.

the class TrainingDatasetService method getPreparedStatements.

@ApiOperation(value = "Get prepared statements used to generate model serving vector from training dataset query", response = ServingPreparedStatementDTO.class)
@GET
@Path("/{trainingdatasetid}/preparedstatements")
@Produces(MediaType.APPLICATION_JSON)
@AllowedProjectRoles({ AllowedProjectRoles.DATA_OWNER, AllowedProjectRoles.DATA_SCIENTIST })
@JWTRequired(acceptedTokens = { Audience.API, Audience.JOB }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
@ApiKeyRequired(acceptedScopes = { ApiScope.FEATURESTORE }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
public Response getPreparedStatements(@Context SecurityContext sc, @Context UriInfo uriInfo, @ApiParam(value = "Id of the trainingdatasetid", required = true) @PathParam("trainingdatasetid") Integer trainingDatsetId, @ApiParam(value = "get batch serving vectors", example = "false") @QueryParam("batch") @DefaultValue("false") boolean batch) throws FeaturestoreException {
    verifyIdProvided(trainingDatsetId);
    Users user = jWTHelper.getUserPrincipal(sc);
    ServingPreparedStatementDTO servingPreparedStatementDTO = preparedStatementBuilder.build(uriInfo, new ResourceRequest(ResourceRequest.Name.PREPAREDSTATEMENTS), project, user, featurestore, trainingDatsetId, batch);
    return Response.ok().entity(servingPreparedStatementDTO).build();
}
Also used : ServingPreparedStatementDTO(io.hops.hopsworks.common.featurestore.query.ServingPreparedStatementDTO) Users(io.hops.hopsworks.persistence.entity.user.Users) ResourceRequest(io.hops.hopsworks.common.api.ResourceRequest) Path(javax.ws.rs.Path) DatasetPath(io.hops.hopsworks.common.dataset.util.DatasetPath) Produces(javax.ws.rs.Produces) GET(javax.ws.rs.GET) JWTRequired(io.hops.hopsworks.jwt.annotation.JWTRequired) ApiOperation(io.swagger.annotations.ApiOperation) ApiKeyRequired(io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired) AllowedProjectRoles(io.hops.hopsworks.api.filter.AllowedProjectRoles)

Example 47 with ApiKeyRequired

use of io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired in project hopsworks by logicalclocks.

the class TrainingDatasetService method getByName.

/**
 * Endpoint for getting a list of training dataset based on the name
 *
 * @param name name of the training dataset to get
 * @return return a JSON representation of the training dataset with the given id
 * @throws FeaturestoreException
 */
@GET
@Path("/{name: [a-z0-9_]*(?=[a-z])[a-z0-9_]+}")
@Produces(MediaType.APPLICATION_JSON)
@AllowedProjectRoles({ AllowedProjectRoles.DATA_OWNER, AllowedProjectRoles.DATA_SCIENTIST })
@JWTRequired(acceptedTokens = { Audience.API, Audience.JOB }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
@ApiKeyRequired(acceptedScopes = { ApiScope.FEATURESTORE }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
@ApiOperation(value = "Get a list of training datasets with a specific name, filter by version", response = List.class)
public Response getByName(@ApiParam(value = "Name of the training dataset", required = true) @PathParam("name") String name, @ApiParam(value = "Filter by a specific version") @QueryParam("version") Integer version, @Context SecurityContext sc) throws FeaturestoreException, ServiceException {
    verifyNameProvided(name);
    Users user = jWTHelper.getUserPrincipal(sc);
    List<TrainingDatasetDTO> trainingDatasetDTO;
    if (version == null) {
        trainingDatasetDTO = trainingDatasetController.getWithNameAndFeaturestore(user, project, featurestore, name);
    } else {
        trainingDatasetDTO = Arrays.asList(trainingDatasetController.getWithNameVersionAndFeaturestore(user, project, featurestore, name, version));
    }
    GenericEntity<List<TrainingDatasetDTO>> trainingDatasetGeneric = new GenericEntity<List<TrainingDatasetDTO>>(trainingDatasetDTO) {
    };
    return Response.ok().entity(trainingDatasetGeneric).build();
}
Also used : TrainingDatasetDTO(io.hops.hopsworks.common.featurestore.trainingdatasets.TrainingDatasetDTO) GenericEntity(javax.ws.rs.core.GenericEntity) List(java.util.List) Users(io.hops.hopsworks.persistence.entity.user.Users) Path(javax.ws.rs.Path) DatasetPath(io.hops.hopsworks.common.dataset.util.DatasetPath) Produces(javax.ws.rs.Produces) GET(javax.ws.rs.GET) JWTRequired(io.hops.hopsworks.jwt.annotation.JWTRequired) ApiOperation(io.swagger.annotations.ApiOperation) ApiKeyRequired(io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired) AllowedProjectRoles(io.hops.hopsworks.api.filter.AllowedProjectRoles)

Example 48 with ApiKeyRequired

use of io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired in project hopsworks by logicalclocks.

the class TrainingDatasetService method compute.

@POST
@Path("/{trainingDatasetId}/compute")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
@ApiOperation(value = "Setup a job to compute and write a training dataset", response = JobDTO.class)
@AllowedProjectRoles({ AllowedProjectRoles.DATA_OWNER, AllowedProjectRoles.DATA_SCIENTIST })
@JWTRequired(acceptedTokens = { Audience.API, Audience.JOB }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
@ApiKeyRequired(acceptedScopes = { ApiScope.DATASET_VIEW, ApiScope.FEATURESTORE }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
public Response compute(@Context UriInfo uriInfo, @Context SecurityContext sc, @PathParam("trainingDatasetId") Integer trainingDatasetId, TrainingDatasetJobConf trainingDatasetJobConf) throws FeaturestoreException, ServiceException, JobException, ProjectException, GenericException {
    verifyIdProvided(trainingDatasetId);
    Users user = jWTHelper.getUserPrincipal(sc);
    TrainingDataset trainingDataset = trainingDatasetController.getTrainingDatasetById(featurestore, trainingDatasetId);
    Map<String, String> writeOptions = null;
    if (trainingDatasetJobConf.getWriteOptions() != null) {
        writeOptions = trainingDatasetJobConf.getWriteOptions().stream().collect(Collectors.toMap(OptionDTO::getName, OptionDTO::getValue));
    }
    Jobs job = fsJobManagerController.setupTrainingDatasetJob(project, user, trainingDataset, trainingDatasetJobConf.getQuery(), trainingDatasetJobConf.getOverwrite(), writeOptions, trainingDatasetJobConf.getSparkJobConfiguration());
    JobDTO jobDTO = jobsBuilder.build(uriInfo, new ResourceRequest(ResourceRequest.Name.JOBS), job);
    return Response.created(jobDTO.getHref()).entity(jobDTO).build();
}
Also used : JobDTO(io.hops.hopsworks.api.jobs.JobDTO) Jobs(io.hops.hopsworks.persistence.entity.jobs.description.Jobs) TrainingDataset(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDataset) OptionDTO(io.hops.hopsworks.common.featurestore.OptionDTO) Users(io.hops.hopsworks.persistence.entity.user.Users) ResourceRequest(io.hops.hopsworks.common.api.ResourceRequest) Path(javax.ws.rs.Path) DatasetPath(io.hops.hopsworks.common.dataset.util.DatasetPath) POST(javax.ws.rs.POST) Consumes(javax.ws.rs.Consumes) Produces(javax.ws.rs.Produces) JWTRequired(io.hops.hopsworks.jwt.annotation.JWTRequired) ApiOperation(io.swagger.annotations.ApiOperation) ApiKeyRequired(io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired) AllowedProjectRoles(io.hops.hopsworks.api.filter.AllowedProjectRoles)

Example 49 with ApiKeyRequired

use of io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired in project hopsworks by logicalclocks.

the class TrainingDatasetService method getAll.

/**
 * Endpoint for getting a list of all training datasets in the feature store.
 *
 * @return a JSON representation of the training datasets in the features store
 */
@GET
@Produces(MediaType.APPLICATION_JSON)
@AllowedProjectRoles({ AllowedProjectRoles.DATA_OWNER, AllowedProjectRoles.DATA_SCIENTIST })
@JWTRequired(acceptedTokens = { Audience.API }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
@ApiKeyRequired(acceptedScopes = { ApiScope.FEATURESTORE }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
@ApiOperation(value = "Get the list of training datasets for a featurestore", response = TrainingDatasetDTO.class, responseContainer = "List")
public Response getAll(@Context SecurityContext sc) throws ServiceException, FeaturestoreException {
    Users user = jWTHelper.getUserPrincipal(sc);
    List<TrainingDatasetDTO> trainingDatasetDTOs = trainingDatasetController.getTrainingDatasetsForFeaturestore(user, project, featurestore);
    GenericEntity<List<TrainingDatasetDTO>> trainingDatasetsGeneric = new GenericEntity<List<TrainingDatasetDTO>>(trainingDatasetDTOs) {
    };
    return noCacheResponse.getNoCacheResponseBuilder(Response.Status.OK).entity(trainingDatasetsGeneric).build();
}
Also used : TrainingDatasetDTO(io.hops.hopsworks.common.featurestore.trainingdatasets.TrainingDatasetDTO) GenericEntity(javax.ws.rs.core.GenericEntity) List(java.util.List) Users(io.hops.hopsworks.persistence.entity.user.Users) Produces(javax.ws.rs.Produces) GET(javax.ws.rs.GET) JWTRequired(io.hops.hopsworks.jwt.annotation.JWTRequired) ApiOperation(io.swagger.annotations.ApiOperation) ApiKeyRequired(io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired) AllowedProjectRoles(io.hops.hopsworks.api.filter.AllowedProjectRoles)

Example 50 with ApiKeyRequired

use of io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired in project hopsworks by logicalclocks.

the class TrainingDatasetService method getTag.

@ApiOperation(value = "Get tag attached to a training dataset", response = TagsDTO.class)
@GET
@Path("/{trainingDatasetId}/tags/{name}")
@Produces(MediaType.APPLICATION_JSON)
@AllowedProjectRoles({ AllowedProjectRoles.DATA_SCIENTIST, AllowedProjectRoles.DATA_OWNER })
@JWTRequired(acceptedTokens = { Audience.API, Audience.JOB }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
@ApiKeyRequired(acceptedScopes = { ApiScope.FEATURESTORE }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
public Response getTag(@Context SecurityContext sc, @Context UriInfo uriInfo, @ApiParam(value = "Id of the training dataset", required = true) @PathParam("trainingDatasetId") Integer trainingDatasetId, @ApiParam(value = "Name of the tag", required = true) @PathParam("name") String name, @BeanParam TagsExpansionBeanParam tagsExpansionBeanParam) throws DatasetException, MetadataException, FeaturestoreException, SchematizedTagException {
    verifyIdProvided(trainingDatasetId);
    Users user = jWTHelper.getUserPrincipal(sc);
    TrainingDataset trainingDataset = trainingDatasetController.getTrainingDatasetById(featurestore, trainingDatasetId);
    Map<String, String> result = new HashMap<>();
    result.put(name, tagController.get(project, user, featurestore, trainingDataset, name));
    ResourceRequest resourceRequest = new ResourceRequest(ResourceRequest.Name.TAGS);
    resourceRequest.setExpansions(tagsExpansionBeanParam.getResources());
    TagsDTO dto = tagBuilder.build(uriInfo, resourceRequest, project, featurestore.getId(), ResourceRequest.Name.TRAININGDATASETS.name(), trainingDatasetId, result);
    return Response.status(Response.Status.OK).entity(dto).build();
}
Also used : TagsDTO(io.hops.hopsworks.common.tags.TagsDTO) HashMap(java.util.HashMap) TrainingDataset(io.hops.hopsworks.persistence.entity.featurestore.trainingdataset.TrainingDataset) Users(io.hops.hopsworks.persistence.entity.user.Users) ResourceRequest(io.hops.hopsworks.common.api.ResourceRequest) Path(javax.ws.rs.Path) DatasetPath(io.hops.hopsworks.common.dataset.util.DatasetPath) Produces(javax.ws.rs.Produces) GET(javax.ws.rs.GET) JWTRequired(io.hops.hopsworks.jwt.annotation.JWTRequired) ApiOperation(io.swagger.annotations.ApiOperation) ApiKeyRequired(io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired) AllowedProjectRoles(io.hops.hopsworks.api.filter.AllowedProjectRoles)

Aggregations

ApiKeyRequired (io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired)175 JWTRequired (io.hops.hopsworks.jwt.annotation.JWTRequired)171 ApiOperation (io.swagger.annotations.ApiOperation)169 Produces (javax.ws.rs.Produces)165 AllowedProjectRoles (io.hops.hopsworks.api.filter.AllowedProjectRoles)159 Path (javax.ws.rs.Path)124 Users (io.hops.hopsworks.persistence.entity.user.Users)116 GET (javax.ws.rs.GET)96 ResourceRequest (io.hops.hopsworks.common.api.ResourceRequest)94 Consumes (javax.ws.rs.Consumes)45 DatasetPath (io.hops.hopsworks.common.dataset.util.DatasetPath)38 POST (javax.ws.rs.POST)32 DELETE (javax.ws.rs.DELETE)26 GenericEntity (javax.ws.rs.core.GenericEntity)25 PUT (javax.ws.rs.PUT)23 TagsDTO (io.hops.hopsworks.common.tags.TagsDTO)15 Featuregroup (io.hops.hopsworks.persistence.entity.featurestore.featuregroup.Featuregroup)11 UriBuilder (javax.ws.rs.core.UriBuilder)11 Project (io.hops.hopsworks.persistence.entity.project.Project)10 FeaturestoreException (io.hops.hopsworks.exceptions.FeaturestoreException)9