Search in sources :

Example 1 with ODirectFileInputStream

use of herddb.utils.ODirectFileInputStream in project herddb by diennea.

the class FileDataStorageManager method readIndexPage.

@Override
public <X> X readIndexPage(String tableSpace, String indexName, Long pageId, DataReader<X> reader) throws DataStorageManagerException {
    Path tableDir = getIndexDirectory(tableSpace, indexName);
    Path pageFile = getPageFile(tableDir, pageId);
    long _start = System.currentTimeMillis();
    X result;
    try {
        if (indexodirect) {
            try (ODirectFileInputStream odirect = new ODirectFileInputStream(pageFile, O_DIRECT_BLOCK_BATCH)) {
                result = readIndexPage(reader, pageFile, odirect);
            }
        } else {
            try (InputStream input = Files.newInputStream(pageFile);
                BufferedInputStream buffer = new BufferedInputStream(input, COPY_BUFFERS_SIZE)) {
                result = readIndexPage(reader, pageFile, buffer);
            }
        }
    } catch (IOException err) {
        throw new DataStorageManagerException(err);
    }
    long _stop = System.currentTimeMillis();
    long delta = _stop - _start;
    LOGGER.log(Level.FINE, "readIndexPage {0}.{1} {2} ms", new Object[] { tableSpace, indexName, delta + "" });
    indexPageReads.registerSuccessfulEvent(delta, TimeUnit.MILLISECONDS);
    return result;
}
Also used : Path(java.nio.file.Path) DataStorageManagerException(herddb.storage.DataStorageManagerException) BufferedInputStream(java.io.BufferedInputStream) BufferedInputStream(java.io.BufferedInputStream) ODirectFileInputStream(herddb.utils.ODirectFileInputStream) ExtendedDataInputStream(herddb.utils.ExtendedDataInputStream) SimpleByteArrayInputStream(herddb.utils.SimpleByteArrayInputStream) InputStream(java.io.InputStream) ODirectFileInputStream(herddb.utils.ODirectFileInputStream) IOException(java.io.IOException)

Example 2 with ODirectFileInputStream

use of herddb.utils.ODirectFileInputStream in project herddb by diennea.

the class FileDataStorageManager method rawReadDataPage.

public static List<Record> rawReadDataPage(Path pageFile) throws DataStorageManagerException, IOException {
    List<Record> result;
    long hashFromFile;
    long hashFromDigest;
    try (ODirectFileInputStream odirect = new ODirectFileInputStream(pageFile, O_DIRECT_BLOCK_BATCH);
        XXHash64Utils.HashingStream hash = new XXHash64Utils.HashingStream(odirect);
        ExtendedDataInputStream dataIn = new ExtendedDataInputStream(hash)) {
        // version
        long version = dataIn.readVLong();
        // flags for future implementations
        long flags = dataIn.readVLong();
        if (version != 1 || flags != 0) {
            throw new DataStorageManagerException("corrupted data file " + pageFile.toAbsolutePath());
        }
        int numRecords = dataIn.readInt();
        result = new ArrayList<>(numRecords);
        for (int i = 0; i < numRecords; i++) {
            Bytes key = dataIn.readBytes();
            Bytes value = dataIn.readBytes();
            result.add(new Record(key, value));
        }
        hashFromDigest = hash.hash();
        hashFromFile = dataIn.readLong();
    }
    if (hashFromFile != NO_HASH_PRESENT && hashFromDigest != hashFromFile) {
        throw new DataStorageManagerException("Corrupted datafile " + pageFile + ". Bad hash " + hashFromFile + " <> " + hashFromDigest);
    }
    return result;
}
Also used : Bytes(herddb.utils.Bytes) ExtendedDataInputStream(herddb.utils.ExtendedDataInputStream) DataStorageManagerException(herddb.storage.DataStorageManagerException) ODirectFileInputStream(herddb.utils.ODirectFileInputStream) Record(herddb.model.Record) XXHash64Utils(herddb.utils.XXHash64Utils)

Example 3 with ODirectFileInputStream

use of herddb.utils.ODirectFileInputStream in project herddb by diennea.

the class FileDataStorageManager method readPage.

@Override
public List<Record> readPage(String tableSpace, String tableName, Long pageId) throws DataStorageManagerException {
    long _start = System.currentTimeMillis();
    Path tableDir = getTableDirectory(tableSpace, tableName);
    Path pageFile = getPageFile(tableDir, pageId);
    List<Record> result;
    try {
        if (pageodirect) {
            try (ODirectFileInputStream odirect = new ODirectFileInputStream(pageFile, O_DIRECT_BLOCK_BATCH)) {
                result = rawReadDataPage(pageFile, odirect);
            }
        } else {
            try (InputStream input = Files.newInputStream(pageFile);
                BufferedInputStream buffer = new BufferedInputStream(input, COPY_BUFFERS_SIZE)) {
                result = rawReadDataPage(pageFile, buffer);
            }
        }
    } catch (NoSuchFileException nsfe) {
        throw new DataPageDoesNotExistException("No such page: " + tableSpace + "_" + tableName + "." + pageId, nsfe);
    } catch (IOException err) {
        throw new DataStorageManagerException("error reading data page: " + tableSpace + "_" + tableName + "." + pageId, err);
    }
    long _stop = System.currentTimeMillis();
    long delta = _stop - _start;
    LOGGER.log(Level.FINE, "readPage {0}.{1} {2} ms", new Object[] { tableSpace, tableName, delta + "" });
    dataPageReads.registerSuccessfulEvent(delta, TimeUnit.MILLISECONDS);
    return result;
}
Also used : Path(java.nio.file.Path) DataStorageManagerException(herddb.storage.DataStorageManagerException) BufferedInputStream(java.io.BufferedInputStream) BufferedInputStream(java.io.BufferedInputStream) ODirectFileInputStream(herddb.utils.ODirectFileInputStream) ExtendedDataInputStream(herddb.utils.ExtendedDataInputStream) SimpleByteArrayInputStream(herddb.utils.SimpleByteArrayInputStream) InputStream(java.io.InputStream) ODirectFileInputStream(herddb.utils.ODirectFileInputStream) NoSuchFileException(java.nio.file.NoSuchFileException) Record(herddb.model.Record) IOException(java.io.IOException) DataPageDoesNotExistException(herddb.storage.DataPageDoesNotExistException)

Aggregations

DataStorageManagerException (herddb.storage.DataStorageManagerException)3 ExtendedDataInputStream (herddb.utils.ExtendedDataInputStream)3 ODirectFileInputStream (herddb.utils.ODirectFileInputStream)3 Record (herddb.model.Record)2 SimpleByteArrayInputStream (herddb.utils.SimpleByteArrayInputStream)2 BufferedInputStream (java.io.BufferedInputStream)2 IOException (java.io.IOException)2 InputStream (java.io.InputStream)2 Path (java.nio.file.Path)2 DataPageDoesNotExistException (herddb.storage.DataPageDoesNotExistException)1 Bytes (herddb.utils.Bytes)1 XXHash64Utils (herddb.utils.XXHash64Utils)1 NoSuchFileException (java.nio.file.NoSuchFileException)1