Search in sources :

Example 1 with BlobStore

use of com.baremaps.blob.BlobStore in project baremaps by baremaps.

the class Diff method call.

@Override
public Integer call() throws Exception {
    BlobStore blobStore = options.blobStore();
    DataSource datasource = PostgresUtils.datasource(database);
    LongDataMap<Coordinate> coordinates = new PostgresCoordinateMap(datasource);
    LongDataMap<List<Long>> references = new PostgresReferenceMap(datasource);
    HeaderRepository headerRepository = new PostgresHeaderRepository(datasource);
    Repository<Long, Node> nodeRepository = new PostgresNodeRepository(datasource);
    Repository<Long, Way> wayRepository = new PostgresWayRepository(datasource);
    Repository<Long, Relation> relationRepository = new PostgresRelationRepository(datasource);
    logger.info("Saving diff");
    Path tmpTiles = Files.createFile(Paths.get("diff.tmp"));
    try (PrintWriter printWriter = new PrintWriter(Files.newBufferedWriter(tmpTiles))) {
        new DiffService(blobStore, coordinates, references, headerRepository, nodeRepository, wayRepository, relationRepository, srid, zoom).call();
    }
    blobStore.put(this.tiles, Blob.builder().withContentLength(Files.size(tmpTiles)).withInputStream(Files.newInputStream(tmpTiles)).build());
    Files.deleteIfExists(tmpTiles);
    logger.info("Done");
    return 0;
}
Also used : Path(java.nio.file.Path) PostgresHeaderRepository(com.baremaps.osm.postgres.PostgresHeaderRepository) HeaderRepository(com.baremaps.osm.repository.HeaderRepository) PostgresWayRepository(com.baremaps.osm.postgres.PostgresWayRepository) PostgresReferenceMap(com.baremaps.osm.postgres.PostgresReferenceMap) Node(com.baremaps.osm.domain.Node) DiffService(com.baremaps.osm.repository.DiffService) PostgresCoordinateMap(com.baremaps.osm.postgres.PostgresCoordinateMap) Way(com.baremaps.osm.domain.Way) DataSource(javax.sql.DataSource) Relation(com.baremaps.osm.domain.Relation) Coordinate(org.locationtech.jts.geom.Coordinate) PostgresNodeRepository(com.baremaps.osm.postgres.PostgresNodeRepository) List(java.util.List) PostgresRelationRepository(com.baremaps.osm.postgres.PostgresRelationRepository) BlobStore(com.baremaps.blob.BlobStore) PostgresHeaderRepository(com.baremaps.osm.postgres.PostgresHeaderRepository) PrintWriter(java.io.PrintWriter)

Example 2 with BlobStore

use of com.baremaps.blob.BlobStore in project baremaps by baremaps.

the class Execute method call.

@Override
public Integer call() throws Exception {
    DataSource datasource = PostgresUtils.datasource(database);
    BlobStore blobStore = options.blobStore();
    for (URI file : files) {
        logger.info("Execute {}", file);
        String blob = new String(blobStore.get(file).getInputStream().readAllBytes(), StandardCharsets.UTF_8);
        blob = interpolate(System.getenv(), blob);
        StreamUtils.batch(Splitter.on(";").splitToStream(blob), 1).forEach(query -> {
            try (Connection connection = datasource.getConnection();
                Statement statement = connection.createStatement()) {
                statement.execute(query);
            } catch (SQLException e) {
                throw new StreamException(e);
            }
        });
    }
    logger.info("Done");
    return 0;
}
Also used : SQLException(java.sql.SQLException) Statement(java.sql.Statement) Connection(java.sql.Connection) URI(java.net.URI) BlobStore(com.baremaps.blob.BlobStore) DataSource(javax.sql.DataSource) StreamException(com.baremaps.stream.StreamException)

Example 3 with BlobStore

use of com.baremaps.blob.BlobStore in project baremaps by baremaps.

the class Import method call.

@Override
public Integer call() throws Exception {
    BlobStore blobStore = options.blobStore();
    DataSource datasource = PostgresUtils.datasource(database);
    HeaderRepository headerRepository = new PostgresHeaderRepository(datasource);
    Repository<Long, Node> nodeRepository = new PostgresNodeRepository(datasource);
    Repository<Long, Way> wayRepository = new PostgresWayRepository(datasource);
    Repository<Long, Relation> relationRepository = new PostgresRelationRepository(datasource);
    Path directory = Files.createTempDirectory(Paths.get("."), "baremaps_");
    Path nodes = Files.createDirectories(directory.resolve("nodes"));
    Path referencesKeys = Files.createDirectories(directory.resolve("references_keys"));
    Path referencesValues = Files.createDirectories(directory.resolve("references_values"));
    LongDataMap<Coordinate> coordinates = new LongAlignedDataDenseMap<>(new LonLatDataType(), new OnDiskMemory(nodes));
    LongDataMap<List<Long>> references = new LongDataSortedMap<>(new AlignedDataList<>(new PairDataType<>(new LongDataType(), new LongDataType()), new OnDiskMemory(referencesKeys)), new DataStore<>(new LongListDataType(), new OnDiskMemory(referencesValues)));
    logger.info("Importing data");
    new ImportService(file, blobStore, coordinates, references, headerRepository, nodeRepository, wayRepository, relationRepository, srid).call();
    logger.info("Done");
    return 0;
}
Also used : PostgresHeaderRepository(com.baremaps.osm.postgres.PostgresHeaderRepository) HeaderRepository(com.baremaps.osm.repository.HeaderRepository) LongListDataType(com.baremaps.store.type.LongListDataType) Node(com.baremaps.osm.domain.Node) Way(com.baremaps.osm.domain.Way) Relation(com.baremaps.osm.domain.Relation) ImportService(com.baremaps.osm.repository.ImportService) LongDataSortedMap(com.baremaps.store.LongDataSortedMap) LongAlignedDataDenseMap(com.baremaps.store.LongAlignedDataDenseMap) AlignedDataList(com.baremaps.store.AlignedDataList) List(java.util.List) BlobStore(com.baremaps.blob.BlobStore) PairDataType(com.baremaps.store.type.PairDataType) Path(java.nio.file.Path) PostgresWayRepository(com.baremaps.osm.postgres.PostgresWayRepository) OnDiskMemory(com.baremaps.store.memory.OnDiskMemory) LonLatDataType(com.baremaps.store.type.LonLatDataType) DataSource(javax.sql.DataSource) Coordinate(org.locationtech.jts.geom.Coordinate) PostgresNodeRepository(com.baremaps.osm.postgres.PostgresNodeRepository) PostgresRelationRepository(com.baremaps.osm.postgres.PostgresRelationRepository) LongDataType(com.baremaps.store.type.LongDataType) PostgresHeaderRepository(com.baremaps.osm.postgres.PostgresHeaderRepository)

Example 4 with BlobStore

use of com.baremaps.blob.BlobStore in project baremaps by baremaps.

the class Init method call.

@Override
public Integer call() throws BlobStoreException, IOException {
    BlobStore blobStore = options.blobStore();
    ObjectMapper mapper = defaultObjectMapper();
    if (style != null) {
        MbStyle styleObject = new MbStyle();
        styleObject.setName("Baremaps");
        MbStyleSources sources = new MbStyleSources();
        sources.setType("vector");
        sources.setUrl("http://localhost:9000/tiles.json");
        styleObject.setSources(Map.of("baremaps", sources));
        blobStore.put(style, Blob.builder().withByteArray(mapper.writerWithDefaultPrettyPrinter().writeValueAsBytes(styleObject)).build());
        logger.info("Style initialized: {}", style);
    }
    if (tileset != null) {
        TileJSON tilesetObject = new TileJSON();
        tilesetObject.setTilejson("2.2.0");
        tilesetObject.setName("Baremaps");
        tilesetObject.setTiles(Arrays.asList("http://localhost:9000/tiles.json"));
        blobStore.put(tileset, Blob.builder().withByteArray(mapper.writerWithDefaultPrettyPrinter().writeValueAsBytes(tilesetObject)).build());
        logger.info("Tileset initialized: {}", tileset);
    }
    return 0;
}
Also used : MbStyleSources(com.baremaps.model.MbStyleSources) TileJSON(com.baremaps.model.TileJSON) BlobStore(com.baremaps.blob.BlobStore) DefaultObjectMapper.defaultObjectMapper(com.baremaps.server.common.DefaultObjectMapper.defaultObjectMapper) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) MbStyle(com.baremaps.model.MbStyle)

Example 5 with BlobStore

use of com.baremaps.blob.BlobStore in project baremaps by baremaps.

the class View method call.

@Override
public Integer call() throws Exception {
    ObjectMapper objectMapper = defaultObjectMapper();
    BlobStore blobStore = options.blobStore();
    TileJSON tileJSON = objectMapper.readValue(blobStore.get(this.tileset).getInputStream(), TileJSON.class);
    CaffeineSpec caffeineSpec = CaffeineSpec.parse(cache);
    DataSource datasource = PostgresUtils.datasource(database);
    List<PostgresQuery> queries = asPostgresQuery(tileJSON);
    TileStore tileStore = new PostgresTileStore(datasource, queries);
    TileStore tileCache = new TileCache(tileStore, caffeineSpec);
    // Configure the application
    ResourceConfig application = new ResourceConfig().register(CorsFilter.class).register(ViewerResources.class).register(contextResolverFor(objectMapper)).register(new AbstractBinder() {

        @Override
        protected void configure() {
            bind(tileset).to(URI.class).named("tileset");
            bind(style).to(URI.class).named("style");
            bind(blobStore).to(BlobStore.class);
            bind(tileCache).to(TileStore.class);
        }
    });
    BlockingStreamingHttpService httpService = new HttpJerseyRouterBuilder().buildBlockingStreaming(application);
    ServerContext serverContext = HttpServers.forPort(port).listenBlockingStreamingAndAwait(httpService);
    logger.info("Listening on {}", serverContext.listenAddress());
    serverContext.awaitShutdown();
    return 0;
}
Also used : CorsFilter(com.baremaps.server.common.CorsFilter) TileStore(com.baremaps.tile.TileStore) PostgresTileStore(com.baremaps.tile.postgres.PostgresTileStore) Conversions.asPostgresQuery(com.baremaps.server.ogcapi.Conversions.asPostgresQuery) PostgresQuery(com.baremaps.tile.postgres.PostgresQuery) AbstractBinder(org.glassfish.hk2.utilities.binding.AbstractBinder) CaffeineSpec(com.github.benmanes.caffeine.cache.CaffeineSpec) DataSource(javax.sql.DataSource) HttpJerseyRouterBuilder(io.servicetalk.http.router.jersey.HttpJerseyRouterBuilder) BlockingStreamingHttpService(io.servicetalk.http.api.BlockingStreamingHttpService) ServerContext(io.servicetalk.transport.api.ServerContext) TileJSON(com.baremaps.model.TileJSON) PostgresTileStore(com.baremaps.tile.postgres.PostgresTileStore) ResourceConfig(org.glassfish.jersey.server.ResourceConfig) TileCache(com.baremaps.tile.TileCache) DefaultObjectMapper.defaultObjectMapper(com.baremaps.server.common.DefaultObjectMapper.defaultObjectMapper) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) BlobStore(com.baremaps.blob.BlobStore)

Aggregations

BlobStore (com.baremaps.blob.BlobStore)9 DataSource (javax.sql.DataSource)7 DefaultObjectMapper.defaultObjectMapper (com.baremaps.server.common.DefaultObjectMapper.defaultObjectMapper)4 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)4 List (java.util.List)4 TileJSON (com.baremaps.model.TileJSON)3 Node (com.baremaps.osm.domain.Node)3 Relation (com.baremaps.osm.domain.Relation)3 Way (com.baremaps.osm.domain.Way)3 PostgresHeaderRepository (com.baremaps.osm.postgres.PostgresHeaderRepository)3 PostgresNodeRepository (com.baremaps.osm.postgres.PostgresNodeRepository)3 PostgresRelationRepository (com.baremaps.osm.postgres.PostgresRelationRepository)3 PostgresWayRepository (com.baremaps.osm.postgres.PostgresWayRepository)3 HeaderRepository (com.baremaps.osm.repository.HeaderRepository)3 BlobStoreException (com.baremaps.blob.BlobStoreException)2 PostgresCoordinateMap (com.baremaps.osm.postgres.PostgresCoordinateMap)2 PostgresReferenceMap (com.baremaps.osm.postgres.PostgresReferenceMap)2 CorsFilter (com.baremaps.server.common.CorsFilter)2 Conversions.asPostgresQuery (com.baremaps.server.ogcapi.Conversions.asPostgresQuery)2 TileStore (com.baremaps.tile.TileStore)2