Search in sources :

Example 1 with SchemaCompatibility

use of io.hops.hopsworks.persistence.entity.kafka.schemas.SchemaCompatibility in project hopsworks by logicalclocks.

the class SubjectsCompatibilityController method setSubjectCompatibility.

public Compatibility setSubjectCompatibility(Project project, String subject, SchemaCompatibility sc) throws SchemaException {
    if (sc == null) {
        throw new SchemaException(RESTCodes.SchemaRegistryErrorCode.INVALID_COMPATIBILITY, Level.WARNING, "Compatibility cannot be null");
    }
    if (subject == null || subject.equals(Settings.PROJECT_COMPATIBILITY_SUBJECT) || subjectsFacade.findSubjectByName(project, subject).isEmpty()) {
        throw new SchemaException(RESTCodes.SchemaRegistryErrorCode.SUBJECT_NOT_FOUND, Level.WARNING, "Incorrect subject");
    }
    subjectsCompatibilityFacade.updateSubjectCompatibility(project, subject, sc);
    CompatibilityLevel levelDto = getSubjectCompatibility(project, subject);
    return new Compatibility(levelDto.getCompatibilityLevel());
}
Also used : SchemaException(io.hops.hopsworks.exceptions.SchemaException) CompatibilityLevel(io.hops.hopsworks.common.dao.kafka.schemas.CompatibilityLevel) SchemaCompatibility(io.hops.hopsworks.persistence.entity.kafka.schemas.SchemaCompatibility) SubjectsCompatibility(io.hops.hopsworks.persistence.entity.kafka.schemas.SubjectsCompatibility) Compatibility(io.hops.hopsworks.common.dao.kafka.schemas.Compatibility)

Example 2 with SchemaCompatibility

use of io.hops.hopsworks.persistence.entity.kafka.schemas.SchemaCompatibility in project hopsworks by logicalclocks.

the class SubjectsController method isCompatible.

private boolean isCompatible(Project project, String subject, Schema schema) throws SchemaException {
    SchemaCompatibility sc = getSubjectOrProjectCompatibility(project, subject);
    if (sc.equals(SchemaCompatibility.NONE)) {
        return true;
    }
    SchemaValidator validator = getSchemaValidator(sc);
    List<Schema> previousSchemas = subjectsFacade.findSubjectByName(project, subject).stream().sorted(Comparator.comparing(Subjects::getVersion).reversed()).map(s -> new Schema.Parser().parse(s.getSchema().getSchema())).collect(Collectors.toList());
    try {
        validator.validate(schema, previousSchemas);
    } catch (SchemaValidationException e) {
        return false;
    }
    return true;
}
Also used : Arrays(java.util.Arrays) SubjectDTO(io.hops.hopsworks.common.dao.kafka.schemas.SubjectDTO) Schemas(io.hops.hopsworks.persistence.entity.kafka.schemas.Schemas) CompatibilityCheck(io.hops.hopsworks.common.dao.kafka.schemas.CompatibilityCheck) Project(io.hops.hopsworks.persistence.entity.project.Project) Level(java.util.logging.Level) ProjectTopicsFacade(io.hops.hopsworks.common.dao.kafka.ProjectTopicsFacade) SubjectsCompatibility(io.hops.hopsworks.persistence.entity.kafka.schemas.SubjectsCompatibility) Settings(io.hops.hopsworks.common.util.Settings) TransactionAttributeType(javax.ejb.TransactionAttributeType) TransactionAttribute(javax.ejb.TransactionAttribute) SubjectsFacade(io.hops.hopsworks.common.dao.kafka.schemas.SubjectsFacade) SubjectsCompatibilityFacade(io.hops.hopsworks.common.dao.kafka.schemas.SubjectsCompatibilityFacade) BigInteger(java.math.BigInteger) EJB(javax.ejb.EJB) SchemaCompatibility(io.hops.hopsworks.persistence.entity.kafka.schemas.SchemaCompatibility) Stateless(javax.ejb.Stateless) Schema(org.apache.avro.Schema) SchemaValidator(org.apache.avro.SchemaValidator) SchemaValidationException(org.apache.avro.SchemaValidationException) SchemaException(io.hops.hopsworks.exceptions.SchemaException) RESTCodes(io.hops.hopsworks.restutils.RESTCodes) Logger(java.util.logging.Logger) Collectors(java.util.stream.Collectors) List(java.util.List) Subjects(io.hops.hopsworks.persistence.entity.kafka.schemas.Subjects) SchemaValidatorBuilder(org.apache.avro.SchemaValidatorBuilder) Optional(java.util.Optional) SchemaParseException(org.apache.avro.SchemaParseException) Comparator(java.util.Comparator) KafkaException(io.hops.hopsworks.exceptions.KafkaException) Collections(java.util.Collections) SchemaValidationException(org.apache.avro.SchemaValidationException) SchemaCompatibility(io.hops.hopsworks.persistence.entity.kafka.schemas.SchemaCompatibility) Schema(org.apache.avro.Schema) SchemaValidator(org.apache.avro.SchemaValidator) Subjects(io.hops.hopsworks.persistence.entity.kafka.schemas.Subjects)

Example 3 with SchemaCompatibility

use of io.hops.hopsworks.persistence.entity.kafka.schemas.SchemaCompatibility in project hopsworks by logicalclocks.

the class SubjectsController method checkIfSchemaCompatible.

public CompatibilityCheck checkIfSchemaCompatible(Project project, String subject, String version, String schemaToTest) throws SchemaException {
    validateVersion(version);
    Schema schema;
    try {
        schema = new Schema.Parser().parse(schemaToTest);
    } catch (SchemaParseException e) {
        throw new SchemaException(RESTCodes.SchemaRegistryErrorCode.INVALID_AVRO_SCHEMA, Level.FINE, "schema=" + schemaToTest);
    }
    if (!subjectsFacade.getListOfSubjects(project).contains(subject)) {
        throw new SchemaException(RESTCodes.SchemaRegistryErrorCode.SUBJECT_NOT_FOUND, Level.FINE, "subject=" + subject);
    }
    SchemaCompatibility sc = getSubjectOrProjectCompatibility(project, subject);
    Optional<Subjects> optional;
    if (version.equals("latest")) {
        optional = subjectsFacade.findSubjectLatestVersion(project, subject);
    } else {
        optional = subjectsFacade.findSubjectByNameAndVersion(project, subject, Integer.valueOf(version));
    }
    if (!optional.isPresent()) {
        throw new SchemaException(RESTCodes.SchemaRegistryErrorCode.VERSION_NOT_FOUND, Level.FINE, "project=" + project.getName() + ", subject=" + subject + ", version=" + version);
    }
    boolean isCompatible = isCompatible(new Schema.Parser().parse(optional.get().getSchema().getSchema()), schema, sc);
    return new CompatibilityCheck(isCompatible);
}
Also used : CompatibilityCheck(io.hops.hopsworks.common.dao.kafka.schemas.CompatibilityCheck) SchemaException(io.hops.hopsworks.exceptions.SchemaException) SchemaParseException(org.apache.avro.SchemaParseException) SchemaCompatibility(io.hops.hopsworks.persistence.entity.kafka.schemas.SchemaCompatibility) Schema(org.apache.avro.Schema) Subjects(io.hops.hopsworks.persistence.entity.kafka.schemas.Subjects)

Aggregations

SchemaException (io.hops.hopsworks.exceptions.SchemaException)3 SchemaCompatibility (io.hops.hopsworks.persistence.entity.kafka.schemas.SchemaCompatibility)3 CompatibilityCheck (io.hops.hopsworks.common.dao.kafka.schemas.CompatibilityCheck)2 Subjects (io.hops.hopsworks.persistence.entity.kafka.schemas.Subjects)2 SubjectsCompatibility (io.hops.hopsworks.persistence.entity.kafka.schemas.SubjectsCompatibility)2 Schema (org.apache.avro.Schema)2 SchemaParseException (org.apache.avro.SchemaParseException)2 ProjectTopicsFacade (io.hops.hopsworks.common.dao.kafka.ProjectTopicsFacade)1 Compatibility (io.hops.hopsworks.common.dao.kafka.schemas.Compatibility)1 CompatibilityLevel (io.hops.hopsworks.common.dao.kafka.schemas.CompatibilityLevel)1 SubjectDTO (io.hops.hopsworks.common.dao.kafka.schemas.SubjectDTO)1 SubjectsCompatibilityFacade (io.hops.hopsworks.common.dao.kafka.schemas.SubjectsCompatibilityFacade)1 SubjectsFacade (io.hops.hopsworks.common.dao.kafka.schemas.SubjectsFacade)1 Settings (io.hops.hopsworks.common.util.Settings)1 KafkaException (io.hops.hopsworks.exceptions.KafkaException)1 Schemas (io.hops.hopsworks.persistence.entity.kafka.schemas.Schemas)1 Project (io.hops.hopsworks.persistence.entity.project.Project)1 RESTCodes (io.hops.hopsworks.restutils.RESTCodes)1 BigInteger (java.math.BigInteger)1 Arrays (java.util.Arrays)1