Search in sources :

Example 11 with RecordWriter

use of com.revolsys.record.io.RecordWriter in project com.revolsys.open by revolsys.

the class Records method copyRecords.

static void copyRecords(final RecordStore sourceRecordStore, final String sourceTableName, final RecordStore targetRecordStore, final String targetTableName, final BiConsumer<Record, Record> recordEditor) {
    final Query query = new Query(sourceTableName);
    try (RecordReader reader = sourceRecordStore.getRecords(query);
        RecordWriter writer = targetRecordStore.newRecordWriter()) {
        final RecordDefinition recordDefinition = targetRecordStore.getRecordDefinition(targetTableName);
        for (final Record record : reader) {
            final Record newRecord = recordDefinition.newRecord();
            newRecord.setValuesAll(record);
            recordEditor.accept(record, newRecord);
            writer.write(newRecord);
        }
    }
}
Also used : RecordWriter(com.revolsys.record.io.RecordWriter) Query(com.revolsys.record.query.Query) RecordReader(com.revolsys.record.io.RecordReader) RecordDefinition(com.revolsys.record.schema.RecordDefinition)

Example 12 with RecordWriter

use of com.revolsys.record.io.RecordWriter in project com.revolsys.open by revolsys.

the class DirectoryRecordStore method insertRecord.

@Override
public synchronized void insertRecord(final Record record) {
    final RecordDefinition recordDefinition = record.getRecordDefinition();
    final String typePath = recordDefinition.getPath();
    RecordWriter writer = this.writers.get(typePath);
    if (writer == null) {
        final String schemaName = PathUtil.getPath(typePath);
        final File subDirectory = FileUtil.getDirectory(getDirectory(), schemaName);
        final String fileExtension = getFileExtension();
        final File file = new File(subDirectory, recordDefinition.getName() + "." + fileExtension);
        final Resource resource = new PathResource(file);
        writer = RecordWriter.newRecordWriter(recordDefinition, resource);
        if (writer == null) {
            throw new RuntimeException("Cannot create writer for: " + typePath);
        } else if (writer instanceof ObjectWithProperties) {
            final ObjectWithProperties properties = writer;
            properties.setProperties(getProperties());
        }
        this.writers.put(typePath, writer);
    }
    writer.write(record);
    addStatistic("Insert", record);
}
Also used : RecordWriter(com.revolsys.record.io.RecordWriter) PathResource(com.revolsys.spring.resource.PathResource) ObjectWithProperties(com.revolsys.properties.ObjectWithProperties) PathResource(com.revolsys.spring.resource.PathResource) Resource(com.revolsys.spring.resource.Resource) File(java.io.File) RecordDefinition(com.revolsys.record.schema.RecordDefinition)

Example 13 with RecordWriter

use of com.revolsys.record.io.RecordWriter in project com.revolsys.open by revolsys.

the class CategoryLabelCountMap method writeCounts.

public void writeCounts(final Object target, final String labelTitle, final Iterable<String> categoryNames) {
    final RecordDefinitionBuilder recordDefinitionBuilder = new RecordDefinitionBuilder("Counts");
    recordDefinitionBuilder.addField(labelTitle, DataTypes.STRING, 50);
    final Set<String> allLabels = new TreeSet<>();
    final List<String> matchedCategoryNames = new ArrayList<>();
    for (final String categoryName : categoryNames) {
        final LabelCountMap labelCountMap = this.labelCountMapByCategory.get(categoryName);
        if (labelCountMap != null) {
            matchedCategoryNames.add(categoryName);
            recordDefinitionBuilder.addField(categoryName, DataTypes.LONG, 10);
            final Set<String> labels = labelCountMap.getLabels();
            allLabels.addAll(labels);
        }
    }
    final RecordDefinition recordDefinition = recordDefinitionBuilder.getRecordDefinition();
    try (RecordWriter recordWriter = RecordWriter.newRecordWriter(recordDefinition, target)) {
        final List<Object> row = new ArrayList<>(matchedCategoryNames.size() + 1);
        for (final String label : allLabels) {
            row.clear();
            row.add(label);
            for (final String categoryName : matchedCategoryNames) {
                final Long count = getCount(categoryName, label);
                row.add(count);
            }
            recordWriter.write(row);
        }
    }
}
Also used : RecordDefinitionBuilder(com.revolsys.record.schema.RecordDefinitionBuilder) RecordWriter(com.revolsys.record.io.RecordWriter) TreeSet(java.util.TreeSet) ArrayList(java.util.ArrayList) RecordDefinition(com.revolsys.record.schema.RecordDefinition)

Aggregations

RecordWriter (com.revolsys.record.io.RecordWriter)13 RecordDefinition (com.revolsys.record.schema.RecordDefinition)6 RecordReader (com.revolsys.record.io.RecordReader)3 Query (com.revolsys.record.query.Query)3 Record (com.revolsys.record.Record)2 RecordDefinitionBuilder (com.revolsys.record.schema.RecordDefinitionBuilder)2 PathResource (com.revolsys.spring.resource.PathResource)2 Resource (com.revolsys.spring.resource.Resource)2 File (java.io.File)2 Identifier (com.revolsys.identifier.Identifier)1 ObjectWithProperties (com.revolsys.properties.ObjectWithProperties)1 ArrayRecord (com.revolsys.record.ArrayRecord)1 ZipRecordWriter (com.revolsys.record.io.format.zip.ZipRecordWriter)1 Transaction (com.revolsys.transaction.Transaction)1 ArrayList (java.util.ArrayList)1 TreeSet (java.util.TreeSet)1