Search in sources :

Example 41 with DataSet

use of nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSet in project timbuctoo by HuygensING.

the class Import method importData.

@POST
@Produces("application/json")
public Response importData(@HeaderParam("Authorization") String authorization, @QueryParam("forceCreation") boolean forceCreation, ImportData importData) throws DataStoreCreationException {
    final Either<Response, Response> responses = authCheck.getOrCreate(authorization, importData.userId, importData.dataSetId, forceCreation).flatMap(userAndDs -> authCheck.hasAdminAccess(userAndDs.getLeft(), userAndDs.getRight())).map(userAndDs -> {
        final DataSet dataSet = userAndDs.getRight();
        ImportManager importManager = dataSet.getImportManager();
        try {
            LOG.info("Loading files");
            Iterator<RemoteFile> files = resourceSyncFileLoader.loadFiles(importData.source.toString()).iterator();
            LOG.info("Found files '{}'", files.hasNext());
            ResourceSyncResport resourceSyncResport = new ResourceSyncResport();
            while (files.hasNext()) {
                RemoteFile file = files.next();
                MediaType parsedMediatype = MediaType.APPLICATION_OCTET_STREAM_TYPE;
                try {
                    parsedMediatype = MediaType.valueOf(file.getMimeType());
                } catch (IllegalArgumentException e) {
                    LOG.error("Failed to get mediatype", e);
                }
                if (importManager.isRdfTypeSupported(parsedMediatype)) {
                    resourceSyncResport.importedFiles.add(file.getUrl());
                    importManager.addLog(dataSet.getMetadata().getBaseUri(), dataSet.getMetadata().getBaseUri(), file.getUrl().substring(file.getUrl().lastIndexOf('/') + 1), file.getData(), Optional.of(Charsets.UTF_8), parsedMediatype);
                } else {
                    resourceSyncResport.ignoredFiles.add(file.getUrl());
                    importManager.addFile(file.getData(), file.getUrl(), parsedMediatype);
                }
            }
            return Response.ok(resourceSyncResport).build();
        } catch (Exception e) {
            LOG.error("Could not read files to import", e);
            return Response.serverError().build();
        }
    });
    if (responses.isLeft()) {
        return responses.getLeft();
    } else {
        return responses.get();
    }
}
Also used : Response(javax.ws.rs.core.Response) Charsets(com.google.common.base.Charsets) Either(javaslang.control.Either) Logger(org.slf4j.Logger) POST(javax.ws.rs.POST) ImportManager(nl.knaw.huygens.timbuctoo.v5.dataset.ImportManager) Produces(javax.ws.rs.Produces) Iterator(java.util.Iterator) ResourceSyncFileLoader(nl.knaw.huygens.timbuctoo.remote.rs.download.ResourceSyncFileLoader) DataStoreCreationException(nl.knaw.huygens.timbuctoo.v5.dataset.exceptions.DataStoreCreationException) LoggerFactory(org.slf4j.LoggerFactory) Path(javax.ws.rs.Path) Set(java.util.Set) RemoteFile(nl.knaw.huygens.timbuctoo.remote.rs.download.RemoteFile) Sets(com.google.common.collect.Sets) MediaType(javax.ws.rs.core.MediaType) QueryParam(javax.ws.rs.QueryParam) Response(javax.ws.rs.core.Response) HeaderParam(javax.ws.rs.HeaderParam) Optional(java.util.Optional) URI(java.net.URI) DataSet(nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSet) AuthCheck(nl.knaw.huygens.timbuctoo.v5.dropwizard.endpoints.auth.AuthCheck) ImportManager(nl.knaw.huygens.timbuctoo.v5.dataset.ImportManager) DataSet(nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSet) MediaType(javax.ws.rs.core.MediaType) RemoteFile(nl.knaw.huygens.timbuctoo.remote.rs.download.RemoteFile) DataStoreCreationException(nl.knaw.huygens.timbuctoo.v5.dataset.exceptions.DataStoreCreationException) POST(javax.ws.rs.POST) Produces(javax.ws.rs.Produces)

Example 42 with DataSet

use of nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSet in project timbuctoo by HuygensING.

the class PermissionBasedFieldVisibility method getFieldDefinition.

@Override
public GraphQLFieldDefinition getFieldDefinition(GraphQLFieldsContainer fieldsContainer, String fieldName) {
    Collection<DataSet> dataSets = dataSetRepository.getDataSets();
    Set<String> dataSetNamesWithOutReadPermission = dataSets.stream().filter(dataSet -> !userPermissionCheck.getPermissions(dataSet.getMetadata()).contains(Permission.READ)).map(dataSet -> dataSet.getMetadata().getCombinedId()).collect(Collectors.toSet());
    Iterator<GraphQLFieldDefinition> graphQlFieldDefinitionIterator = fieldsContainer.getFieldDefinitions().iterator();
    while (graphQlFieldDefinitionIterator.hasNext()) {
        GraphQLFieldDefinition graphQlFieldDefinition = graphQlFieldDefinitionIterator.next();
        if (!dataSetNamesWithOutReadPermission.contains(graphQlFieldDefinition.getName()) && graphQlFieldDefinition.getName().equals(fieldName)) {
            return graphQlFieldDefinition;
        }
    }
    return null;
}
Also used : List(java.util.List) Iterator(java.util.Iterator) DataSetRepository(nl.knaw.huygens.timbuctoo.v5.dataset.DataSetRepository) GraphqlFieldVisibility(graphql.schema.visibility.GraphqlFieldVisibility) Collection(java.util.Collection) GraphQLFieldDefinition(graphql.schema.GraphQLFieldDefinition) Set(java.util.Set) GraphQLFieldsContainer(graphql.schema.GraphQLFieldsContainer) DataSet(nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSet) Collectors(java.util.stream.Collectors) Permission(nl.knaw.huygens.timbuctoo.v5.security.dto.Permission) ArrayList(java.util.ArrayList) DataSet(nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSet) GraphQLFieldDefinition(graphql.schema.GraphQLFieldDefinition)

Example 43 with DataSet

use of nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSet in project timbuctoo by HuygensING.

the class PermissionBasedFieldVisibility method getFieldDefinitions.

@Override
public List<GraphQLFieldDefinition> getFieldDefinitions(GraphQLFieldsContainer fieldsContainer) {
    List<GraphQLFieldDefinition> graphQlFieldDefinitions = new ArrayList<>();
    Collection<DataSet> dataSets = dataSetRepository.getDataSets();
    Set<String> dataSetNamesWithOutReadPermission = dataSets.stream().filter(dataSet -> !userPermissionCheck.getPermissions(dataSet.getMetadata()).contains(Permission.READ)).map(dataSet -> dataSet.getMetadata().getCombinedId()).collect(Collectors.toSet());
    Iterator<GraphQLFieldDefinition> graphQlFieldDefinitionIterator = fieldsContainer.getFieldDefinitions().iterator();
    while (graphQlFieldDefinitionIterator.hasNext()) {
        GraphQLFieldDefinition graphQlFieldDefinition = graphQlFieldDefinitionIterator.next();
        if (!dataSetNamesWithOutReadPermission.contains(graphQlFieldDefinition.getName())) {
            graphQlFieldDefinitions.add(graphQlFieldDefinition);
        }
    }
    return graphQlFieldDefinitions;
}
Also used : List(java.util.List) Iterator(java.util.Iterator) DataSetRepository(nl.knaw.huygens.timbuctoo.v5.dataset.DataSetRepository) GraphqlFieldVisibility(graphql.schema.visibility.GraphqlFieldVisibility) Collection(java.util.Collection) GraphQLFieldDefinition(graphql.schema.GraphQLFieldDefinition) Set(java.util.Set) GraphQLFieldsContainer(graphql.schema.GraphQLFieldsContainer) DataSet(nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSet) Collectors(java.util.stream.Collectors) Permission(nl.knaw.huygens.timbuctoo.v5.security.dto.Permission) ArrayList(java.util.ArrayList) DataSet(nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSet) ArrayList(java.util.ArrayList) GraphQLFieldDefinition(graphql.schema.GraphQLFieldDefinition)

Example 44 with DataSet

use of nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSet in project timbuctoo by HuygensING.

the class RootQuery method getCollections.

public CollectionMetadataList getCollections(DataSetMetaData input, Optional<User> userOpt) {
    final User user = userOpt.orElse(null);
    final DataSet dataSet = dataSetRepository.getDataSet(user, input.getOwnerId(), input.getDataSetId()).get();
    final TypeNameStore typeNameStore = dataSet.getTypeNameStore();
    final List<CollectionMetadata> colls = dataSet.getSchemaStore().getStableTypes().values().stream().map(x -> {
        return getCollection(dataSet, typeNameStore, x);
    }).collect(Collectors.toList());
    return ImmutableCollectionMetadataList.builder().nextCursor(Optional.empty()).prevCursor(Optional.empty()).items(colls).build();
}
Also used : CursorQuad(nl.knaw.huygens.timbuctoo.v5.datastores.quadstore.dto.CursorQuad) DataSetRepository(nl.knaw.huygens.timbuctoo.v5.dataset.DataSetRepository) ImmutableProperty(nl.knaw.huygens.timbuctoo.v5.graphql.rootquery.dataproviders.ImmutableProperty) ExplicitType(nl.knaw.huygens.timbuctoo.v5.datastores.schemastore.dto.ExplicitType) Type(nl.knaw.huygens.timbuctoo.v5.datastores.schemastore.dto.Type) MakePublicMutation(nl.knaw.huygens.timbuctoo.v5.graphql.mutations.MakePublicMutation) LoggerFactory(org.slf4j.LoggerFactory) ExtendSchemaMutation(nl.knaw.huygens.timbuctoo.v5.graphql.mutations.ExtendSchemaMutation) DerivedSchemaTypeGenerator(nl.knaw.huygens.timbuctoo.v5.graphql.derivedschema.DerivedSchemaTypeGenerator) DeleteDataSetMutation(nl.knaw.huygens.timbuctoo.v5.graphql.mutations.DeleteDataSetMutation) SubjectReference(nl.knaw.huygens.timbuctoo.v5.graphql.datafetchers.dto.SubjectReference) Map(java.util.Map) SummaryPropsMutation(nl.knaw.huygens.timbuctoo.v5.graphql.mutations.SummaryPropsMutation) ImmutableCollectionMetadataList(nl.knaw.huygens.timbuctoo.v5.graphql.rootquery.dataproviders.ImmutableCollectionMetadataList) TypeNameStore(nl.knaw.huygens.timbuctoo.v5.datastores.prefixstore.TypeNameStore) ImmutablePropertyList(nl.knaw.huygens.timbuctoo.v5.graphql.rootquery.dataproviders.ImmutablePropertyList) ViewConfigFetcher(nl.knaw.huygens.timbuctoo.v5.graphql.rootquery.dataproviders.ViewConfigFetcher) CreateDataSetMutation(nl.knaw.huygens.timbuctoo.v5.graphql.mutations.CreateDataSetMutation) RootData(nl.knaw.huygens.timbuctoo.v5.graphql.datafetchers.dto.RootData) TypeDefinitionRegistry(graphql.schema.idl.TypeDefinitionRegistry) Collectors(java.util.stream.Collectors) Resources.getResource(com.google.common.io.Resources.getResource) List(java.util.List) Stream(java.util.stream.Stream) DataSetMetaData(nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSetMetaData) ExplicitField(nl.knaw.huygens.timbuctoo.v5.datastores.schemastore.dto.ExplicitField) Optional(java.util.Optional) Direction(nl.knaw.huygens.timbuctoo.v5.datastores.quadstore.dto.Direction) RdfWiringFactory(nl.knaw.huygens.timbuctoo.v5.graphql.datafetchers.RdfWiringFactory) ImportStatus(nl.knaw.huygens.timbuctoo.v5.dataset.ImportStatus) CollectionMetadata(nl.knaw.huygens.timbuctoo.v5.graphql.rootquery.dataproviders.CollectionMetadata) SupportedExportFormats(nl.knaw.huygens.timbuctoo.v5.dropwizard.SupportedExportFormats) CollectionMetadataList(nl.knaw.huygens.timbuctoo.v5.graphql.rootquery.dataproviders.CollectionMetadataList) DataSetImportStatus(nl.knaw.huygens.timbuctoo.v5.dataset.DataSetImportStatus) Tuple(nl.knaw.huygens.timbuctoo.util.Tuple) HashMap(java.util.HashMap) QuadStore(nl.knaw.huygens.timbuctoo.v5.datastores.quadstore.QuadStore) Supplier(java.util.function.Supplier) ContextData(nl.knaw.huygens.timbuctoo.v5.graphql.datafetchers.dto.ContextData) ArrayList(java.util.ArrayList) User(nl.knaw.huygens.timbuctoo.v5.security.dto.User) SchemaParser(graphql.schema.idl.SchemaParser) ImmutableStringList(nl.knaw.huygens.timbuctoo.v5.graphql.rootquery.dataproviders.ImmutableStringList) GraphQLSchema(graphql.schema.GraphQLSchema) ImmutableCollectionMetadata(nl.knaw.huygens.timbuctoo.v5.graphql.rootquery.dataproviders.ImmutableCollectionMetadata) Charsets(com.google.common.base.Charsets) ViewConfigMutation(nl.knaw.huygens.timbuctoo.v5.graphql.mutations.ViewConfigMutation) Logger(org.slf4j.Logger) UserPermissionCheck(nl.knaw.huygens.timbuctoo.v5.graphql.security.UserPermissionCheck) Resources(com.google.common.io.Resources) MimeTypeDescription(nl.knaw.huygens.timbuctoo.v5.graphql.rootquery.dataproviders.MimeTypeDescription) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) MergeSchemas(nl.knaw.huygens.timbuctoo.v5.graphql.customschema.MergeSchemas) Property(nl.knaw.huygens.timbuctoo.v5.graphql.rootquery.dataproviders.Property) IOException(java.io.IOException) DataSetWithDatabase(nl.knaw.huygens.timbuctoo.v5.graphql.datafetchers.dto.DataSetWithDatabase) EntryImportStatus(nl.knaw.huygens.timbuctoo.v5.dataset.dto.EntryImportStatus) TIM_HASINDEXERCONFIG(nl.knaw.huygens.timbuctoo.v5.util.RdfConstants.TIM_HASINDEXERCONFIG) Permission(nl.knaw.huygens.timbuctoo.v5.security.dto.Permission) IndexConfigMutation(nl.knaw.huygens.timbuctoo.v5.graphql.mutations.IndexConfigMutation) RuntimeWiring(graphql.schema.idl.RuntimeWiring) DataSetMetadataMutation(nl.knaw.huygens.timbuctoo.v5.graphql.mutations.DataSetMetadataMutation) CollectionMetadataMutation(nl.knaw.huygens.timbuctoo.v5.graphql.mutations.CollectionMetadataMutation) SchemaGenerator(graphql.schema.idl.SchemaGenerator) DataSet(nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSet) Collections(java.util.Collections) CollectionMetadata(nl.knaw.huygens.timbuctoo.v5.graphql.rootquery.dataproviders.CollectionMetadata) ImmutableCollectionMetadata(nl.knaw.huygens.timbuctoo.v5.graphql.rootquery.dataproviders.ImmutableCollectionMetadata) User(nl.knaw.huygens.timbuctoo.v5.security.dto.User) DataSet(nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSet) TypeNameStore(nl.knaw.huygens.timbuctoo.v5.datastores.prefixstore.TypeNameStore)

Example 45 with DataSet

use of nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSet in project timbuctoo by HuygensING.

the class DataSetRepositoryTest method removeDataSetRemovesTheDataSetsAuthorizations.

@Test
public void removeDataSetRemovesTheDataSetsAuthorizations() throws Exception {
    User user = User.create(null, "user");
    final DataSet dataSet = dataSetRepository.createDataSet(user, "dataset");
    DataSetMetaData metadata = dataSet.getMetadata();
    String owner = metadata.getOwnerId();
    given(permissionFetcher.getPermissions(user, metadata)).willReturn(Sets.newHashSet(Permission.ADMIN));
    dataSetRepository.removeDataSet(owner, "dataset", user);
    verify(permissionFetcher).removeAuthorizations(metadata.getCombinedId());
}
Also used : User(nl.knaw.huygens.timbuctoo.v5.security.dto.User) DataSet(nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSet) DataSetMetaData(nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSetMetaData) Test(org.junit.Test)

Aggregations

DataSet (nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSet)44 DataSetRepository (nl.knaw.huygens.timbuctoo.v5.dataset.DataSetRepository)12 DataSetMetaData (nl.knaw.huygens.timbuctoo.v5.dataset.dto.DataSetMetaData)12 User (nl.knaw.huygens.timbuctoo.v5.security.dto.User)12 Test (org.junit.Test)12 IOException (java.io.IOException)11 LogStorageFailedException (nl.knaw.huygens.timbuctoo.v5.filestorage.exceptions.LogStorageFailedException)11 Map (java.util.Map)10 GraphQLFieldDefinition (graphql.schema.GraphQLFieldDefinition)7 GraphQLFieldsContainer (graphql.schema.GraphQLFieldsContainer)7 List (java.util.List)7 Optional (java.util.Optional)7 ExecutionException (java.util.concurrent.ExecutionException)7 Logger (org.slf4j.Logger)7 LoggerFactory (org.slf4j.LoggerFactory)7 HashMap (java.util.HashMap)6 Collectors (java.util.stream.Collectors)6 ImportManager (nl.knaw.huygens.timbuctoo.v5.dataset.ImportManager)6 ImportStatus (nl.knaw.huygens.timbuctoo.v5.dataset.ImportStatus)6 DataStoreCreationException (nl.knaw.huygens.timbuctoo.v5.dataset.exceptions.DataStoreCreationException)6