Search in sources :

Example 6 with SchemaException

use of io.hops.hopsworks.exceptions.SchemaException in project hopsworks by logicalclocks.

the class SubjectsController method deleteSubject.

public List<Integer> deleteSubject(Project project, String subject) throws SchemaException, KafkaException {
    validateSubject(subject, false);
    if (!projectTopicsFacade.findTopicsBySubject(project, subject).isEmpty()) {
        throw new KafkaException(RESTCodes.KafkaErrorCode.SCHEMA_IN_USE, Level.FINE, "project=" + project.getName() + ", subject=" + subject);
    }
    List<Integer> versions = getSubjectVersions(project, subject);
    Integer deleted = subjectsFacade.deleteSubject(project, subject);
    if (versions.size() != deleted) {
        throw new SchemaException(RESTCodes.SchemaRegistryErrorCode.INTERNAL_SERVER_ERROR, Level.FINE, "error deleting " + "subject. versions=" + Arrays.toString(versions.toArray()) + ", but deleted " + deleted + " items.");
    }
    subjectsCompatibilityFacade.getSubjectCompatibility(project, subject).ifPresent(sc -> subjectsCompatibilityFacade.remove(sc));
    return versions;
}
Also used : BigInteger(java.math.BigInteger) SchemaException(io.hops.hopsworks.exceptions.SchemaException) KafkaException(io.hops.hopsworks.exceptions.KafkaException)

Example 7 with SchemaException

use of io.hops.hopsworks.exceptions.SchemaException in project hopsworks by logicalclocks.

the class SubjectsController method isCompatible.

private boolean isCompatible(Project project, String subject, Schema schema) throws SchemaException {
    SchemaCompatibility sc = getSubjectOrProjectCompatibility(project, subject);
    if (sc.equals(SchemaCompatibility.NONE)) {
        return true;
    }
    SchemaValidator validator = getSchemaValidator(sc);
    List<Schema> previousSchemas = subjectsFacade.findSubjectByName(project, subject).stream().sorted(Comparator.comparing(Subjects::getVersion).reversed()).map(s -> new Schema.Parser().parse(s.getSchema().getSchema())).collect(Collectors.toList());
    try {
        validator.validate(schema, previousSchemas);
    } catch (SchemaValidationException e) {
        return false;
    }
    return true;
}
Also used : Arrays(java.util.Arrays) SubjectDTO(io.hops.hopsworks.common.dao.kafka.schemas.SubjectDTO) Schemas(io.hops.hopsworks.persistence.entity.kafka.schemas.Schemas) CompatibilityCheck(io.hops.hopsworks.common.dao.kafka.schemas.CompatibilityCheck) Project(io.hops.hopsworks.persistence.entity.project.Project) Level(java.util.logging.Level) ProjectTopicsFacade(io.hops.hopsworks.common.dao.kafka.ProjectTopicsFacade) SubjectsCompatibility(io.hops.hopsworks.persistence.entity.kafka.schemas.SubjectsCompatibility) Settings(io.hops.hopsworks.common.util.Settings) TransactionAttributeType(javax.ejb.TransactionAttributeType) TransactionAttribute(javax.ejb.TransactionAttribute) SubjectsFacade(io.hops.hopsworks.common.dao.kafka.schemas.SubjectsFacade) SubjectsCompatibilityFacade(io.hops.hopsworks.common.dao.kafka.schemas.SubjectsCompatibilityFacade) BigInteger(java.math.BigInteger) EJB(javax.ejb.EJB) SchemaCompatibility(io.hops.hopsworks.persistence.entity.kafka.schemas.SchemaCompatibility) Stateless(javax.ejb.Stateless) Schema(org.apache.avro.Schema) SchemaValidator(org.apache.avro.SchemaValidator) SchemaValidationException(org.apache.avro.SchemaValidationException) SchemaException(io.hops.hopsworks.exceptions.SchemaException) RESTCodes(io.hops.hopsworks.restutils.RESTCodes) Logger(java.util.logging.Logger) Collectors(java.util.stream.Collectors) List(java.util.List) Subjects(io.hops.hopsworks.persistence.entity.kafka.schemas.Subjects) SchemaValidatorBuilder(org.apache.avro.SchemaValidatorBuilder) Optional(java.util.Optional) SchemaParseException(org.apache.avro.SchemaParseException) Comparator(java.util.Comparator) KafkaException(io.hops.hopsworks.exceptions.KafkaException) Collections(java.util.Collections) SchemaValidationException(org.apache.avro.SchemaValidationException) SchemaCompatibility(io.hops.hopsworks.persistence.entity.kafka.schemas.SchemaCompatibility) Schema(org.apache.avro.Schema) SchemaValidator(org.apache.avro.SchemaValidator) Subjects(io.hops.hopsworks.persistence.entity.kafka.schemas.Subjects)

Example 8 with SchemaException

use of io.hops.hopsworks.exceptions.SchemaException in project hopsworks by logicalclocks.

the class KafkaResource method getSubjectVersions.

@ApiOperation(value = "Get list of versions of a registered subject.")
@GET
@Path("/subjects/{subject}/versions")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.TEXT_PLAIN)
@AllowedProjectRoles({ AllowedProjectRoles.DATA_OWNER, AllowedProjectRoles.DATA_SCIENTIST })
@JWTRequired(acceptedTokens = { Audience.API }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
@ApiKeyRequired(acceptedScopes = { ApiScope.KAFKA }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
public Response getSubjectVersions(@PathParam("subject") String subject, @Context SecurityContext sc) {
    try {
        List<Integer> versions = subjectsController.getSubjectVersions(project, subject);
        String array = Arrays.toString(versions.toArray());
        GenericEntity<String> entity = new GenericEntity<String>(array) {
        };
        return Response.ok().entity(entity).build();
    } catch (SchemaException e) {
        SchemaRegistryError error = new SchemaRegistryError(e.getErrorCode().getCode(), e.getErrorCode().getMessage());
        return Response.status(e.getErrorCode().getRespStatus()).entity(error).build();
    }
}
Also used : SchemaException(io.hops.hopsworks.exceptions.SchemaException) GenericEntity(javax.ws.rs.core.GenericEntity) SchemaRegistryError(io.hops.hopsworks.common.dao.kafka.schemas.SchemaRegistryError) Path(javax.ws.rs.Path) Consumes(javax.ws.rs.Consumes) Produces(javax.ws.rs.Produces) GET(javax.ws.rs.GET) JWTRequired(io.hops.hopsworks.jwt.annotation.JWTRequired) ApiOperation(io.swagger.annotations.ApiOperation) ApiKeyRequired(io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired) AllowedProjectRoles(io.hops.hopsworks.api.filter.AllowedProjectRoles)

Example 9 with SchemaException

use of io.hops.hopsworks.exceptions.SchemaException in project hopsworks by logicalclocks.

the class KafkaResource method deleteSubject.

@ApiOperation(value = "Delete a subject with all its versions and its compatibility if exists.")
@DELETE
@Path("/subjects/{subject}")
@Produces({ MediaType.TEXT_PLAIN, MediaType.APPLICATION_JSON })
@AllowedProjectRoles({ AllowedProjectRoles.DATA_OWNER, AllowedProjectRoles.DATA_SCIENTIST })
@JWTRequired(acceptedTokens = { Audience.API }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
@ApiKeyRequired(acceptedScopes = { ApiScope.KAFKA }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
public Response deleteSubject(@PathParam("subject") String subject, @Context SecurityContext sc) throws KafkaException {
    try {
        List<Integer> versions = subjectsController.deleteSubject(project, subject);
        String array = Arrays.toString(versions.toArray());
        GenericEntity<String> entity = new GenericEntity<String>(array) {
        };
        return Response.ok().entity(entity).build();
    } catch (SchemaException e) {
        SchemaRegistryError error = new SchemaRegistryError(e.getErrorCode().getCode(), e.getErrorCode().getMessage());
        return Response.status(e.getErrorCode().getRespStatus()).entity(error).build();
    }
}
Also used : SchemaException(io.hops.hopsworks.exceptions.SchemaException) GenericEntity(javax.ws.rs.core.GenericEntity) SchemaRegistryError(io.hops.hopsworks.common.dao.kafka.schemas.SchemaRegistryError) Path(javax.ws.rs.Path) DELETE(javax.ws.rs.DELETE) Produces(javax.ws.rs.Produces) JWTRequired(io.hops.hopsworks.jwt.annotation.JWTRequired) ApiOperation(io.swagger.annotations.ApiOperation) ApiKeyRequired(io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired) AllowedProjectRoles(io.hops.hopsworks.api.filter.AllowedProjectRoles)

Example 10 with SchemaException

use of io.hops.hopsworks.exceptions.SchemaException in project hopsworks by logicalclocks.

the class KafkaResource method getSchema.

@ApiOperation(value = "Get avro schema of a subject and version.")
@GET
@Path("/subjects/{subject}/versions/{version}/schema")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
@AllowedProjectRoles({ AllowedProjectRoles.DATA_OWNER, AllowedProjectRoles.DATA_SCIENTIST })
@JWTRequired(acceptedTokens = { Audience.API }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
@ApiKeyRequired(acceptedScopes = { ApiScope.KAFKA }, allowedUserRoles = { "HOPS_ADMIN", "HOPS_USER" })
public Response getSchema(@PathParam("subject") String subject, @PathParam("version") String version, @Context SecurityContext sc) {
    try {
        SubjectDTO dto = subjectsController.getSubjectDetails(project, subject, version);
        GenericEntity<String> entity = new GenericEntity<String>(dto.getSchema()) {
        };
        return Response.ok().entity(entity).build();
    } catch (SchemaException e) {
        SchemaRegistryError error = new SchemaRegistryError(e.getErrorCode().getCode(), e.getErrorCode().getMessage());
        return Response.status(e.getErrorCode().getRespStatus()).entity(error).build();
    }
}
Also used : SchemaException(io.hops.hopsworks.exceptions.SchemaException) SubjectDTO(io.hops.hopsworks.common.dao.kafka.schemas.SubjectDTO) GenericEntity(javax.ws.rs.core.GenericEntity) SchemaRegistryError(io.hops.hopsworks.common.dao.kafka.schemas.SchemaRegistryError) Path(javax.ws.rs.Path) Consumes(javax.ws.rs.Consumes) Produces(javax.ws.rs.Produces) GET(javax.ws.rs.GET) JWTRequired(io.hops.hopsworks.jwt.annotation.JWTRequired) ApiOperation(io.swagger.annotations.ApiOperation) ApiKeyRequired(io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired) AllowedProjectRoles(io.hops.hopsworks.api.filter.AllowedProjectRoles)

Aggregations

SchemaException (io.hops.hopsworks.exceptions.SchemaException)15 Subjects (io.hops.hopsworks.persistence.entity.kafka.schemas.Subjects)7 KafkaException (io.hops.hopsworks.exceptions.KafkaException)6 Path (javax.ws.rs.Path)5 Produces (javax.ws.rs.Produces)5 AllowedProjectRoles (io.hops.hopsworks.api.filter.AllowedProjectRoles)4 ApiKeyRequired (io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired)4 SchemaRegistryError (io.hops.hopsworks.common.dao.kafka.schemas.SchemaRegistryError)4 SubjectDTO (io.hops.hopsworks.common.dao.kafka.schemas.SubjectDTO)4 JWTRequired (io.hops.hopsworks.jwt.annotation.JWTRequired)4 ApiOperation (io.swagger.annotations.ApiOperation)4 BigInteger (java.math.BigInteger)4 GenericEntity (javax.ws.rs.core.GenericEntity)4 SchemaCompatibility (io.hops.hopsworks.persistence.entity.kafka.schemas.SchemaCompatibility)3 Project (io.hops.hopsworks.persistence.entity.project.Project)3 Schema (org.apache.avro.Schema)3 SchemaParseException (org.apache.avro.SchemaParseException)3 DistributedFileSystemOps (io.hops.hopsworks.common.hdfs.DistributedFileSystemOps)2 ProvTypeDTO (io.hops.hopsworks.common.provenance.core.dto.ProvTypeDTO)2 DatasetException (io.hops.hopsworks.exceptions.DatasetException)2