use of org.talend.components.common.dataset.DatasetProperties in project components by Talend.
the class RuntimeControllerImpl method useDatasetRuntime.
private <//
T> //
T useDatasetRuntime(//
final DatasetDefinition<DatasetProperties<DatastoreProperties>> datasetDefinition, //
DatasetProperties datasetProperties, Function<DatasetRuntime<DatasetProperties<DatastoreProperties>>, T> consumer) {
try (SandboxedInstance instance = RuntimeUtil.createRuntimeClass(datasetDefinition.getRuntimeInfo(datasetProperties), datasetProperties.getClass().getClassLoader())) {
DatasetRuntime<DatasetProperties<DatastoreProperties>> datasetRuntimeInstance = (DatasetRuntime<DatasetProperties<DatastoreProperties>>) instance.getInstance();
datasetRuntimeInstance.initialize(null, datasetProperties);
return consumer.apply(datasetRuntimeInstance);
}
}
use of org.talend.components.common.dataset.DatasetProperties in project components by Talend.
the class RuntimeControllerImpl method writeData.
@Override
public void writeData(InputStream rawPayload) throws IOException {
// 1) Read payload (with data as a stream of course)
DatasetWritePayload payload = DatasetWritePayload.readData(rawPayload, mapper);
String definitionName = payload.getConfiguration().getDefinitionName();
// 2) Create properties
Properties properties = propertiesHelpers.propertiesFromDto(payload.getConfiguration());
if (properties instanceof ComponentProperties) {
ComponentProperties componentProperties = (ComponentProperties) properties;
// 3) Retrieve component definition to be able to create the runtime
final ComponentDefinition definition = propertiesHelpers.getDefinition(ComponentDefinition.class, definitionName);
// 4) Get the execution engine
ExecutionEngine executionEngine;
if (definition.isSupportingExecutionEngines(DI)) {
executionEngine = DI;
// 5) Create the sandbox
try (SandboxedInstance instance = RuntimeUtil.createRuntimeClass(definition.getRuntimeInfo(executionEngine, componentProperties, INCOMING), definition.getClass().getClassLoader())) {
Sink datasetRuntimeInstance = (Sink) instance.getInstance();
datasetRuntimeInstance.initialize(null, componentProperties);
Iterator<IndexedRecord> data = payload.getData();
WriteOperation writeOperation = datasetRuntimeInstance.createWriteOperation();
// Supplier return null to signify end of data stream => see WriterDataSupplier.writeData
WriterDataSupplier<?, IndexedRecord> stringWriterDataSupplier = new WriterDataSupplier<Object, IndexedRecord>(writeOperation, () -> data.hasNext() ? data.next() : null, null);
stringWriterDataSupplier.writeData();
}
} else if (definition.isSupportingExecutionEngines(BEAM)) {
throw new UnsupportedOperationException("Beam runtime is not available for dataset write through HTTP API.");
} else {
throw new TalendRuntimeException(CommonErrorCodes.UNREGISTERED_DEFINITION);
}
} else if (properties instanceof DatasetProperties) {
throw new UnsupportedOperationException("HTTP API is only able to write using component implementations. Not " + properties.getClass());
}
}
use of org.talend.components.common.dataset.DatasetProperties in project components by Talend.
the class GoogleDriveDatastoreDefinitionTest method testCreateDatasetProperties.
@Test
public void testCreateDatasetProperties() throws Exception {
GoogleDriveDatastoreProperties dsp = new GoogleDriveDatastoreProperties("test");
dsp.setupProperties();
dsp.setupLayout();
DatasetProperties ds = def.createDatasetProperties(dsp);
assertNotNull(ds);
assertEquals(dsp, ds.getDatastoreProperties());
}
use of org.talend.components.common.dataset.DatasetProperties in project components by Talend.
the class JDBCDatastoreDefinitionTest method testCreateDatasetProperties.
/**
* Run the DatasetProperties createDatasetProperties(JDBCDatastoreProperties) method test.
*
* @throws Exception
*
* @generatedBy CodePro at 17-6-23 PM2:14
*/
@Test
public void testCreateDatasetProperties() throws Exception {
JDBCDatastoreDefinition fixture = new JDBCDatastoreDefinition();
JDBCDatastoreProperties storeProp = new JDBCDatastoreProperties("datastore");
DatasetProperties result = fixture.createDatasetProperties(storeProp);
assertNotNull(result);
assertEquals("properties.dataset.displayName", result.getDisplayName());
assertEquals("dataset", result.getName());
assertEquals("dataset", result.getTitle());
}
use of org.talend.components.common.dataset.DatasetProperties in project components by Talend.
the class KafkaDatastoreDefinitionTest method createDatasetProperties.
@Test
public void createDatasetProperties() throws Exception {
KafkaDatastoreProperties properties = new KafkaDatastoreProperties("");
DatasetProperties datasetProperties = definition.createDatasetProperties(properties);
assertNotNull(datasetProperties);
assertEquals(properties, datasetProperties.getDatastoreProperties());
}
Aggregations