Search in sources :

Example 1 with CsvInspectionResponse

use of io.atlasmap.csv.v2.CsvInspectionResponse in project atlasmap by atlasmap.

the class CsvService method inspect.

/**
 * Inspect a CSV instance and return a Document object.
 * @param request request
 * @param format format
 * @param delimiter delimiter
 * @param firstRecordAsHeader first record as header
 * @param skipHeaderRecord skip header record
 * @param headers headers
 * @param commentMarker comment marker
 * @param escape escape
 * @param ignoreEmptyLines ignore empty lines
 * @param ignoreHeaderCase ignore header case
 * @param ignoreSurroundingSpaces ignore surrounding spaces
 * @param nullString null string
 * @param quote quote
 * @param allowDuplicateHeaderNames allow duplicate header names
 * @param allowMissingColumnNames allow missing column names
 * @return {@link CsvInspectionResponse}
 * @throws IOException unexpected error
 */
@POST
@Consumes({ MediaType.APPLICATION_JSON })
@Produces({ MediaType.APPLICATION_JSON })
@Path("/inspect")
@Operation(summary = "Inspect CSV", description = "Inspect a CSV instance and return a Document object")
@RequestBody(description = "Csv", content = @Content(mediaType = "text/csv", schema = @Schema(implementation = String.class)))
@ApiResponses(@ApiResponse(responseCode = "200", content = @Content(schema = @Schema(implementation = CsvInspectionResponse.class)), description = "Return a Document object"))
public Response inspect(InputStream request, @QueryParam("format") String format, @QueryParam("delimiter") String delimiter, @QueryParam("firstRecordAsHeader") Boolean firstRecordAsHeader, @QueryParam("skipRecordHeader") Boolean skipHeaderRecord, @QueryParam("headers") String headers, @QueryParam("commentMarker") String commentMarker, @QueryParam("escape") String escape, @QueryParam("ignoreEmptyLines") Boolean ignoreEmptyLines, @QueryParam("ignoreHeaderCase") Boolean ignoreHeaderCase, @QueryParam("ignoreSurroundingSpaces") Boolean ignoreSurroundingSpaces, @QueryParam("nullString") String nullString, @QueryParam("quote") String quote, @QueryParam("allowDuplicateHeaderNames") Boolean allowDuplicateHeaderNames, @QueryParam("allowMissingColumnNames") Boolean allowMissingColumnNames) throws IOException {
    long startTime = System.currentTimeMillis();
    CsvInspectionResponse response = new CsvInspectionResponse();
    try {
        if (LOG.isDebugEnabled()) {
            LOG.debug("Options: delimiter={}, firstRecordAsHeader={}", delimiter, firstRecordAsHeader);
        }
        CsvConfig csvConfig = new CsvConfig(format);
        if (delimiter != null) {
            csvConfig.setDelimiter(delimiter.charAt(0));
        }
        csvConfig.setFirstRecordAsHeader(firstRecordAsHeader);
        csvConfig.setSkipHeaderRecord(skipHeaderRecord);
        csvConfig.setHeaders(headers);
        if (commentMarker != null) {
            csvConfig.setCommentMarker(commentMarker.charAt(0));
        }
        if (escape != null) {
            csvConfig.setEscape(escape.charAt(0));
        }
        csvConfig.setIgnoreEmptyLines(ignoreEmptyLines);
        csvConfig.setIgnoreHeaderCase(ignoreHeaderCase);
        csvConfig.setIgnoreSurroundingSpaces(ignoreSurroundingSpaces);
        csvConfig.setNullString(nullString);
        if (quote != null) {
            csvConfig.setQuote(quote.charAt(0));
        }
        csvConfig.setAllowDuplicateHeaderNames(allowDuplicateHeaderNames);
        csvConfig.setAllowMissingColumnNames(allowMissingColumnNames);
        CsvFieldReader csvFieldReader = new CsvFieldReader(csvConfig);
        csvFieldReader.setDocument(request);
        Document document = csvFieldReader.readSchema();
        response.setCsvDocument(document);
        request.close();
    } catch (Exception e) {
        LOG.error("Error inspecting CSV: " + e.getMessage(), e);
        response.setErrorMessage(e.getMessage());
    } finally {
        request.close();
        ;
        response.setExecutionTime(System.currentTimeMillis() - startTime);
    }
    if (LOG.isDebugEnabled()) {
        LOG.debug(("Response: {}" + new ObjectMapper().writeValueAsString(response)));
    }
    return Response.ok().entity(toJson(response)).build();
}
Also used : CsvInspectionResponse(io.atlasmap.csv.v2.CsvInspectionResponse) CsvConfig(io.atlasmap.csv.core.CsvConfig) Document(io.atlasmap.v2.Document) CsvFieldReader(io.atlasmap.csv.core.CsvFieldReader) IOException(java.io.IOException) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException) WebApplicationException(javax.ws.rs.WebApplicationException) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Path(javax.ws.rs.Path) POST(javax.ws.rs.POST) Consumes(javax.ws.rs.Consumes) Produces(javax.ws.rs.Produces) Operation(io.swagger.v3.oas.annotations.Operation) ApiResponses(io.swagger.v3.oas.annotations.responses.ApiResponses) RequestBody(io.swagger.v3.oas.annotations.parameters.RequestBody)

Example 2 with CsvInspectionResponse

use of io.atlasmap.csv.v2.CsvInspectionResponse in project atlasmap by atlasmap.

the class CsvServiceTest method testSchemaNoParametersSpecified.

@Test
public void testSchemaNoParametersSpecified() throws Exception {
    final String source = "l1r1,l1r2,l1r3\n" + "l2r1,l2r2,l2r3\n" + "l3r1,l3r2,l3r3\n";
    InputStream inputStream = new ByteArrayInputStream(source.getBytes());
    Response res = csvService.inspect(inputStream, null, null, null, null, null, null, null, null, null, null, null, null, null, null);
    Object entity = res.getEntity();
    assertEquals(byte[].class, entity.getClass());
    CsvInspectionResponse csvInspectionResponse = Json.mapper().readValue((byte[]) entity, CsvInspectionResponse.class);
    CsvComplexType complexType = (CsvComplexType) csvInspectionResponse.getCsvDocument().getFields().getField().get(0);
    List<CsvField> fields = complexType.getCsvFields().getCsvField();
    assertEquals("0", fields.get(0).getName());
    assertEquals("1", fields.get(1).getName());
    assertEquals("2", fields.get(2).getName());
}
Also used : CsvInspectionResponse(io.atlasmap.csv.v2.CsvInspectionResponse) Response(javax.ws.rs.core.Response) CsvInspectionResponse(io.atlasmap.csv.v2.CsvInspectionResponse) ByteArrayInputStream(java.io.ByteArrayInputStream) CsvComplexType(io.atlasmap.csv.v2.CsvComplexType) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) CsvField(io.atlasmap.csv.v2.CsvField) Test(org.junit.jupiter.api.Test)

Example 3 with CsvInspectionResponse

use of io.atlasmap.csv.v2.CsvInspectionResponse in project atlasmap by atlasmap.

the class CsvServiceTest method testSchemaFile.

@Test
public void testSchemaFile() throws Exception {
    InputStream inputStream = Thread.currentThread().getContextClassLoader().getResourceAsStream("test.csv");
    Response res = csvService.inspect(inputStream, null, ",", true, null, null, null, null, null, null, null, null, null, null, null);
    Object entity = res.getEntity();
    assertEquals(byte[].class, entity.getClass());
    CsvInspectionResponse csvInspectionResponse = Json.mapper().readValue((byte[]) entity, CsvInspectionResponse.class);
    CsvComplexType complexType = (CsvComplexType) csvInspectionResponse.getCsvDocument().getFields().getField().get(0);
    List<CsvField> fields = complexType.getCsvFields().getCsvField();
    assertEquals(5, fields.size());
    assertEquals("sourceCsvString", fields.get(0).getName());
    assertEquals("sourceCsvNumber", fields.get(1).getName());
    assertEquals("sourceCsvDecimal", fields.get(2).getName());
    assertEquals("sourceCsvDate", fields.get(3).getName());
    assertEquals("sourceCsvBoolean", fields.get(4).getName());
}
Also used : CsvInspectionResponse(io.atlasmap.csv.v2.CsvInspectionResponse) Response(javax.ws.rs.core.Response) CsvInspectionResponse(io.atlasmap.csv.v2.CsvInspectionResponse) CsvComplexType(io.atlasmap.csv.v2.CsvComplexType) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) CsvField(io.atlasmap.csv.v2.CsvField) Test(org.junit.jupiter.api.Test)

Example 4 with CsvInspectionResponse

use of io.atlasmap.csv.v2.CsvInspectionResponse in project atlasmap by atlasmap.

the class CsvServiceTest method testSchema.

@Test
public void testSchema() throws Exception {
    final String source = "header1,header2,header3\n" + "l1r1,l1r2,l1r3\n" + "l2r1,l2r2,l2r3\n" + "l3r1,l3r2,l3r3\n";
    InputStream inputStream = new ByteArrayInputStream(source.getBytes());
    Response res = csvService.inspect(inputStream, null, ",", true, null, null, null, null, null, null, null, null, null, null, null);
    Object entity = res.getEntity();
    assertEquals(byte[].class, entity.getClass());
    CsvInspectionResponse csvInspectionResponse = Json.mapper().readValue((byte[]) entity, CsvInspectionResponse.class);
    CsvComplexType complexType = (CsvComplexType) csvInspectionResponse.getCsvDocument().getFields().getField().get(0);
    List<CsvField> fields = complexType.getCsvFields().getCsvField();
    assertEquals("header1", fields.get(0).getName());
    assertEquals("header2", fields.get(1).getName());
    assertEquals("header3", fields.get(2).getName());
}
Also used : CsvInspectionResponse(io.atlasmap.csv.v2.CsvInspectionResponse) Response(javax.ws.rs.core.Response) CsvInspectionResponse(io.atlasmap.csv.v2.CsvInspectionResponse) ByteArrayInputStream(java.io.ByteArrayInputStream) CsvComplexType(io.atlasmap.csv.v2.CsvComplexType) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) CsvField(io.atlasmap.csv.v2.CsvField) Test(org.junit.jupiter.api.Test)

Aggregations

CsvInspectionResponse (io.atlasmap.csv.v2.CsvInspectionResponse)4 CsvComplexType (io.atlasmap.csv.v2.CsvComplexType)3 CsvField (io.atlasmap.csv.v2.CsvField)3 ByteArrayInputStream (java.io.ByteArrayInputStream)3 InputStream (java.io.InputStream)3 Response (javax.ws.rs.core.Response)3 Test (org.junit.jupiter.api.Test)3 JsonProcessingException (com.fasterxml.jackson.core.JsonProcessingException)1 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1 CsvConfig (io.atlasmap.csv.core.CsvConfig)1 CsvFieldReader (io.atlasmap.csv.core.CsvFieldReader)1 Document (io.atlasmap.v2.Document)1 Operation (io.swagger.v3.oas.annotations.Operation)1 RequestBody (io.swagger.v3.oas.annotations.parameters.RequestBody)1 ApiResponses (io.swagger.v3.oas.annotations.responses.ApiResponses)1 IOException (java.io.IOException)1 Consumes (javax.ws.rs.Consumes)1 POST (javax.ws.rs.POST)1 Path (javax.ws.rs.Path)1 Produces (javax.ws.rs.Produces)1