Search in sources :

Example 1 with ConnectorConfigurer

use of io.cdap.cdap.etl.api.connector.ConnectorConfigurer in project cdap by caskdata.

the class RemoteConnectionSpecTask method execute.

@Override
public String execute(SystemAppTaskContext systemAppContext, RemoteConnectionRequest request) throws Exception {
    String namespace = request.getNamespace();
    Connection connection = request.getConnection();
    // Plugin selector and configurer
    TrackedPluginSelector pluginSelector = new TrackedPluginSelector(new ArtifactSelectorProvider().getPluginSelector(connection.getPlugin().getArtifact()));
    ServicePluginConfigurer pluginConfigurer = systemAppContext.createServicePluginConfigurer(namespace);
    ConnectorConfigurer connectorConfigurer = new DefaultConnectorConfigurer(pluginConfigurer);
    ConnectorContext connectorContext = ConnectionUtils.getConnectorContext(pluginConfigurer);
    SpecGenerationRequest specRequest = GSON.fromJson(request.getRequest(), SpecGenerationRequest.class);
    try (Connector connector = getConnector(systemAppContext, pluginConfigurer, connection.getPlugin(), namespace, pluginSelector)) {
        connector.configure(connectorConfigurer);
        ConnectorSpecRequest connectorSpecRequest = ConnectorSpecRequest.builder().setPath(specRequest.getPath()).setConnection(connection.getName()).setProperties(specRequest.getProperties()).build();
        ConnectorSpec spec = connector.generateSpec(connectorContext, connectorSpecRequest);
        ConnectorSpec newSpec = ConnectionUtils.filterSpecWithPluginNameAndType(spec, specRequest.getPluginName(), specRequest.getPluginType());
        return GSON.toJson(ConnectionUtils.getConnectorDetail(pluginSelector.getSelectedArtifact(), newSpec));
    }
}
Also used : Connector(io.cdap.cdap.etl.api.connector.Connector) ArtifactSelectorProvider(io.cdap.cdap.etl.common.ArtifactSelectorProvider) TrackedPluginSelector(io.cdap.cdap.etl.spec.TrackedPluginSelector) Connection(io.cdap.cdap.etl.proto.connection.Connection) ConnectorContext(io.cdap.cdap.etl.api.connector.ConnectorContext) SpecGenerationRequest(io.cdap.cdap.etl.proto.connection.SpecGenerationRequest) ConnectorSpecRequest(io.cdap.cdap.etl.api.connector.ConnectorSpecRequest) ConnectorSpec(io.cdap.cdap.etl.api.connector.ConnectorSpec) DefaultConnectorConfigurer(io.cdap.cdap.datapipeline.connection.DefaultConnectorConfigurer) DefaultConnectorConfigurer(io.cdap.cdap.datapipeline.connection.DefaultConnectorConfigurer) ConnectorConfigurer(io.cdap.cdap.etl.api.connector.ConnectorConfigurer) ServicePluginConfigurer(io.cdap.cdap.api.service.http.ServicePluginConfigurer)

Example 2 with ConnectorConfigurer

use of io.cdap.cdap.etl.api.connector.ConnectorConfigurer in project cdap by caskdata.

the class RemoteConnectionTestTask method execute.

@Override
public String execute(SystemAppTaskContext systemAppContext, RemoteConnectionRequest request) throws Exception {
    String namespace = request.getNamespace();
    ConnectionCreationRequest connectionCreationRequest = GSON.fromJson(request.getRequest(), ConnectionCreationRequest.class);
    ServicePluginConfigurer pluginConfigurer = systemAppContext.createServicePluginConfigurer(namespace);
    ConnectorConfigurer connectorConfigurer = new DefaultConnectorConfigurer(pluginConfigurer);
    SimpleFailureCollector failureCollector = new SimpleFailureCollector();
    ConnectorContext connectorContext = new DefaultConnectorContext(failureCollector, pluginConfigurer);
    TrackedPluginSelector pluginSelector = new TrackedPluginSelector(new ArtifactSelectorProvider().getPluginSelector(connectionCreationRequest.getPlugin().getArtifact()));
    try (Connector connector = getConnector(systemAppContext, pluginConfigurer, connectionCreationRequest.getPlugin(), namespace, pluginSelector)) {
        connector.configure(connectorConfigurer);
        try {
            connector.test(connectorContext);
            failureCollector.getOrThrowException();
            return "";
        } catch (ValidationException e) {
            return GSON.toJson(e.getFailures());
        }
    }
}
Also used : Connector(io.cdap.cdap.etl.api.connector.Connector) SimpleFailureCollector(io.cdap.cdap.etl.proto.validation.SimpleFailureCollector) ArtifactSelectorProvider(io.cdap.cdap.etl.common.ArtifactSelectorProvider) DefaultConnectorContext(io.cdap.cdap.datapipeline.connection.DefaultConnectorContext) TrackedPluginSelector(io.cdap.cdap.etl.spec.TrackedPluginSelector) ValidationException(io.cdap.cdap.etl.api.validation.ValidationException) DefaultConnectorContext(io.cdap.cdap.datapipeline.connection.DefaultConnectorContext) ConnectorContext(io.cdap.cdap.etl.api.connector.ConnectorContext) ConnectionCreationRequest(io.cdap.cdap.etl.proto.connection.ConnectionCreationRequest) DefaultConnectorConfigurer(io.cdap.cdap.datapipeline.connection.DefaultConnectorConfigurer) DefaultConnectorConfigurer(io.cdap.cdap.datapipeline.connection.DefaultConnectorConfigurer) ConnectorConfigurer(io.cdap.cdap.etl.api.connector.ConnectorConfigurer) ServicePluginConfigurer(io.cdap.cdap.api.service.http.ServicePluginConfigurer)

Example 3 with ConnectorConfigurer

use of io.cdap.cdap.etl.api.connector.ConnectorConfigurer in project cdap by caskdata.

the class ConnectionHandler method sampleLocally.

private void sampleLocally(String namespace, String sampleRequestString, Connection conn, HttpServiceResponder responder) throws IOException {
    SampleRequest sampleRequest = GSON.fromJson(sampleRequestString, SampleRequest.class);
    ServicePluginConfigurer pluginConfigurer = getContext().createServicePluginConfigurer(namespace);
    ConnectorConfigurer connectorConfigurer = new DefaultConnectorConfigurer(pluginConfigurer);
    ConnectorContext connectorContext = new DefaultConnectorContext(new SimpleFailureCollector(), pluginConfigurer);
    PluginInfo plugin = conn.getPlugin();
    // use tracked selector to get exact plugin version that gets selected since the passed version can be null
    TrackedPluginSelector pluginSelector = new TrackedPluginSelector(new ArtifactSelectorProvider().getPluginSelector(plugin.getArtifact()));
    try (Connector connector = getConnector(pluginConfigurer, plugin, namespace, pluginSelector)) {
        connector.configure(connectorConfigurer);
        ConnectorSpecRequest specRequest = ConnectorSpecRequest.builder().setPath(sampleRequest.getPath()).setConnection(conn.getName()).setProperties(sampleRequest.getProperties()).build();
        ConnectorSpec spec = connector.generateSpec(connectorContext, specRequest);
        ConnectorDetail detail = ConnectionUtils.getConnectorDetail(pluginSelector.getSelectedArtifact(), spec);
        try {
            SampleResponse sampleResponse = ConnectionUtils.getSampleResponse(connector, connectorContext, sampleRequest, detail, pluginConfigurer);
            responder.sendString(GSON.toJson(sampleResponse));
        } catch (ConnectionBadRequestException e) {
            // should not happen
            responder.sendError(HttpURLConnection.HTTP_BAD_REQUEST, e.getMessage());
        }
    }
}
Also used : Connector(io.cdap.cdap.etl.api.connector.Connector) SimpleFailureCollector(io.cdap.cdap.etl.proto.validation.SimpleFailureCollector) ArtifactSelectorProvider(io.cdap.cdap.etl.common.ArtifactSelectorProvider) SampleResponse(io.cdap.cdap.etl.proto.connection.SampleResponse) ConnectorDetail(io.cdap.cdap.etl.proto.connection.ConnectorDetail) ConnectorSpec(io.cdap.cdap.etl.api.connector.ConnectorSpec) SampleRequest(io.cdap.cdap.etl.api.connector.SampleRequest) DefaultConnectorConfigurer(io.cdap.cdap.datapipeline.connection.DefaultConnectorConfigurer) ConnectionBadRequestException(io.cdap.cdap.etl.proto.connection.ConnectionBadRequestException) DefaultConnectorContext(io.cdap.cdap.datapipeline.connection.DefaultConnectorContext) TrackedPluginSelector(io.cdap.cdap.etl.spec.TrackedPluginSelector) ConnectorContext(io.cdap.cdap.etl.api.connector.ConnectorContext) DefaultConnectorContext(io.cdap.cdap.datapipeline.connection.DefaultConnectorContext) PluginInfo(io.cdap.cdap.etl.proto.connection.PluginInfo) ConnectorSpecRequest(io.cdap.cdap.etl.api.connector.ConnectorSpecRequest) DefaultConnectorConfigurer(io.cdap.cdap.datapipeline.connection.DefaultConnectorConfigurer) ConnectorConfigurer(io.cdap.cdap.etl.api.connector.ConnectorConfigurer) ServicePluginConfigurer(io.cdap.cdap.api.service.http.ServicePluginConfigurer)

Example 4 with ConnectorConfigurer

use of io.cdap.cdap.etl.api.connector.ConnectorConfigurer in project cdap by caskdata.

the class ConnectionHandler method testLocally.

private void testLocally(String namespace, ConnectionCreationRequest creationRequest, HttpServiceResponder responder) throws IOException {
    ServicePluginConfigurer pluginConfigurer = getContext().createServicePluginConfigurer(namespace);
    ConnectorConfigurer connectorConfigurer = new DefaultConnectorConfigurer(pluginConfigurer);
    SimpleFailureCollector failureCollector = new SimpleFailureCollector();
    ConnectorContext connectorContext = new DefaultConnectorContext(failureCollector, pluginConfigurer);
    TrackedPluginSelector pluginSelector = new TrackedPluginSelector(new ArtifactSelectorProvider().getPluginSelector(creationRequest.getPlugin().getArtifact()));
    try (Connector connector = getConnector(pluginConfigurer, creationRequest.getPlugin(), namespace, pluginSelector)) {
        connector.configure(connectorConfigurer);
        try {
            connector.test(connectorContext);
            failureCollector.getOrThrowException();
        } catch (ValidationException e) {
            responder.sendJson(e.getFailures());
            return;
        }
    }
    responder.sendStatus(HttpURLConnection.HTTP_OK);
}
Also used : Connector(io.cdap.cdap.etl.api.connector.Connector) SimpleFailureCollector(io.cdap.cdap.etl.proto.validation.SimpleFailureCollector) ArtifactSelectorProvider(io.cdap.cdap.etl.common.ArtifactSelectorProvider) DefaultConnectorContext(io.cdap.cdap.datapipeline.connection.DefaultConnectorContext) TrackedPluginSelector(io.cdap.cdap.etl.spec.TrackedPluginSelector) ValidationException(io.cdap.cdap.etl.api.validation.ValidationException) ConnectorContext(io.cdap.cdap.etl.api.connector.ConnectorContext) DefaultConnectorContext(io.cdap.cdap.datapipeline.connection.DefaultConnectorContext) DefaultConnectorConfigurer(io.cdap.cdap.datapipeline.connection.DefaultConnectorConfigurer) DefaultConnectorConfigurer(io.cdap.cdap.datapipeline.connection.DefaultConnectorConfigurer) ConnectorConfigurer(io.cdap.cdap.etl.api.connector.ConnectorConfigurer) ServicePluginConfigurer(io.cdap.cdap.api.service.http.ServicePluginConfigurer)

Example 5 with ConnectorConfigurer

use of io.cdap.cdap.etl.api.connector.ConnectorConfigurer in project cdap by caskdata.

the class ConnectionHandler method specGenerationLocally.

private void specGenerationLocally(String namespace, SpecGenerationRequest specRequest, Connection conn, HttpServiceResponder responder) throws IOException {
    ServicePluginConfigurer pluginConfigurer = getContext().createServicePluginConfigurer(namespace);
    ConnectorConfigurer connectorConfigurer = new DefaultConnectorConfigurer(pluginConfigurer);
    ConnectorContext connectorContext = new DefaultConnectorContext(new SimpleFailureCollector(), pluginConfigurer);
    // use tracked selector to get exact plugin version that gets selected since the passed version can be null
    TrackedPluginSelector pluginSelector = new TrackedPluginSelector(new ArtifactSelectorProvider().getPluginSelector(conn.getPlugin().getArtifact()));
    try (Connector connector = getConnector(pluginConfigurer, conn.getPlugin(), namespace, pluginSelector)) {
        connector.configure(connectorConfigurer);
        ConnectorSpecRequest connectorSpecRequest = ConnectorSpecRequest.builder().setPath(specRequest.getPath()).setConnection(conn.getName()).setProperties(specRequest.getProperties()).build();
        ConnectorSpec spec = connector.generateSpec(connectorContext, connectorSpecRequest);
        ConnectorSpec newSpec = ConnectionUtils.filterSpecWithPluginNameAndType(spec, specRequest.getPluginName(), specRequest.getPluginType());
        responder.sendString(GSON.toJson(ConnectionUtils.getConnectorDetail(pluginSelector.getSelectedArtifact(), newSpec)));
    }
}
Also used : Connector(io.cdap.cdap.etl.api.connector.Connector) SimpleFailureCollector(io.cdap.cdap.etl.proto.validation.SimpleFailureCollector) ArtifactSelectorProvider(io.cdap.cdap.etl.common.ArtifactSelectorProvider) DefaultConnectorContext(io.cdap.cdap.datapipeline.connection.DefaultConnectorContext) TrackedPluginSelector(io.cdap.cdap.etl.spec.TrackedPluginSelector) ConnectorContext(io.cdap.cdap.etl.api.connector.ConnectorContext) DefaultConnectorContext(io.cdap.cdap.datapipeline.connection.DefaultConnectorContext) ConnectorSpecRequest(io.cdap.cdap.etl.api.connector.ConnectorSpecRequest) ConnectorSpec(io.cdap.cdap.etl.api.connector.ConnectorSpec) DefaultConnectorConfigurer(io.cdap.cdap.datapipeline.connection.DefaultConnectorConfigurer) DefaultConnectorConfigurer(io.cdap.cdap.datapipeline.connection.DefaultConnectorConfigurer) ConnectorConfigurer(io.cdap.cdap.etl.api.connector.ConnectorConfigurer) ServicePluginConfigurer(io.cdap.cdap.api.service.http.ServicePluginConfigurer)

Aggregations

ServicePluginConfigurer (io.cdap.cdap.api.service.http.ServicePluginConfigurer)6 DefaultConnectorConfigurer (io.cdap.cdap.datapipeline.connection.DefaultConnectorConfigurer)6 Connector (io.cdap.cdap.etl.api.connector.Connector)6 ConnectorConfigurer (io.cdap.cdap.etl.api.connector.ConnectorConfigurer)6 ConnectorContext (io.cdap.cdap.etl.api.connector.ConnectorContext)6 ArtifactSelectorProvider (io.cdap.cdap.etl.common.ArtifactSelectorProvider)6 TrackedPluginSelector (io.cdap.cdap.etl.spec.TrackedPluginSelector)6 DefaultConnectorContext (io.cdap.cdap.datapipeline.connection.DefaultConnectorContext)5 SimpleFailureCollector (io.cdap.cdap.etl.proto.validation.SimpleFailureCollector)5 ConnectorSpec (io.cdap.cdap.etl.api.connector.ConnectorSpec)3 ConnectorSpecRequest (io.cdap.cdap.etl.api.connector.ConnectorSpecRequest)3 ValidationException (io.cdap.cdap.etl.api.validation.ValidationException)2 SampleRequest (io.cdap.cdap.etl.api.connector.SampleRequest)1 Connection (io.cdap.cdap.etl.proto.connection.Connection)1 ConnectionBadRequestException (io.cdap.cdap.etl.proto.connection.ConnectionBadRequestException)1 ConnectionCreationRequest (io.cdap.cdap.etl.proto.connection.ConnectionCreationRequest)1 ConnectorDetail (io.cdap.cdap.etl.proto.connection.ConnectorDetail)1 PluginInfo (io.cdap.cdap.etl.proto.connection.PluginInfo)1 SampleResponse (io.cdap.cdap.etl.proto.connection.SampleResponse)1 SpecGenerationRequest (io.cdap.cdap.etl.proto.connection.SpecGenerationRequest)1