Search in sources :

Example 1 with ServerSideTest

use of uk.co.automatictester.lightning.tests.ServerSideTest in project lightning by automatictester.

the class LightningTask method runTests.

void runTests() {
    long testSetExecStart = System.currentTimeMillis();
    LightningXMLFileReader xmlFileReader = new LightningXMLFileReader();
    xmlFileReader.readTests(extension.getTestSetXml());
    List<ClientSideTest> clientSideTests = xmlFileReader.getClientSideTests();
    List<ServerSideTest> serverSideTests = xmlFileReader.getServerSideTests();
    testSet = new TestSet(clientSideTests, serverSideTests);
    jmeterTransactions = new JMeterCSVFileReader().getTransactions(extension.getJmeterCsv());
    if (extension.getPerfmonCsv() != null) {
        PerfMonDataEntries perfMonDataEntries = new PerfMonDataReader().getDataEntires(extension.getPerfmonCsv());
        testSet.executeServerSideTests(perfMonDataEntries);
    }
    testSet.executeClientSideTests(jmeterTransactions);
    log(testSet.getTestExecutionReport());
    log(new TestSetReporter(testSet).getTestSetExecutionSummaryReport());
    long testSetExecEnd = System.currentTimeMillis();
    long testExecTime = testSetExecEnd - testSetExecStart;
    log(String.format("Execution time:    %dms", testExecTime));
    if (testSet.getFailCount() + testSet.getErrorCount() != 0) {
        exitCode = 1;
    }
}
Also used : ClientSideTest(uk.co.automatictester.lightning.tests.ClientSideTest) ServerSideTest(uk.co.automatictester.lightning.tests.ServerSideTest) TestSetReporter(uk.co.automatictester.lightning.reporters.TestSetReporter) JMeterCSVFileReader(uk.co.automatictester.lightning.readers.JMeterCSVFileReader) PerfMonDataReader(uk.co.automatictester.lightning.readers.PerfMonDataReader) PerfMonDataEntries(uk.co.automatictester.lightning.data.PerfMonDataEntries) LightningXMLFileReader(uk.co.automatictester.lightning.readers.LightningXMLFileReader) TestSet(uk.co.automatictester.lightning.TestSet)

Example 2 with ServerSideTest

use of uk.co.automatictester.lightning.tests.ServerSideTest in project lightning by automatictester.

the class TeamCityReporterTest method testPrintTeamCityVerifyStatistics.

@Test
public void testPrintTeamCityVerifyStatistics() {
    final PassedTransactionsTest clientTest = mock(PassedTransactionsTest.class);
    when(clientTest.getName()).thenReturn("Failed transactions");
    when(clientTest.getActualResult()).thenReturn(1);
    final ServerSideTest serverTest = mock(ServerSideTest.class);
    when(serverTest.getName()).thenReturn("Memory utilization");
    when(serverTest.getActualResult()).thenReturn(45);
    TestSet testSet = mock(TestSet.class);
    when(testSet.getClientSideTests()).thenReturn(new ArrayList<ClientSideTest>() {

        {
            add(clientTest);
        }
    });
    when(testSet.getServerSideTests()).thenReturn(new ArrayList<ServerSideTest>() {

        {
            add(serverTest);
        }
    });
    String output = new TeamCityReporter(testSet).getTeamCityVerifyStatistics();
    assertThat(output, containsString("##teamcity[buildStatisticValue key='Failed transactions' value='1']"));
    assertThat(output, containsString("##teamcity[buildStatisticValue key='Memory utilization' value='45']"));
}
Also used : ServerSideTest(uk.co.automatictester.lightning.tests.ServerSideTest) ClientSideTest(uk.co.automatictester.lightning.tests.ClientSideTest) PassedTransactionsTest(uk.co.automatictester.lightning.tests.PassedTransactionsTest) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) TestSet(uk.co.automatictester.lightning.TestSet) Test(org.testng.annotations.Test) PassedTransactionsTest(uk.co.automatictester.lightning.tests.PassedTransactionsTest) ClientSideTest(uk.co.automatictester.lightning.tests.ClientSideTest) ServerSideTest(uk.co.automatictester.lightning.tests.ServerSideTest)

Example 3 with ServerSideTest

use of uk.co.automatictester.lightning.tests.ServerSideTest in project lightning by automatictester.

the class LightningHandler method runTests.

private void runTests() {
    long testSetExecStart = System.currentTimeMillis();
    LightningXMLS3ObjectReader xmlFileReader = new LightningXMLS3ObjectReader(region, bucket);
    xmlFileReader.readTests(xml);
    List<ClientSideTest> clientSideTests = xmlFileReader.getClientSideTests();
    List<ServerSideTest> serverSideTests = xmlFileReader.getServerSideTests();
    testSet = new TestSet(clientSideTests, serverSideTests);
    jmeterTransactions = new JMeterCSVS3ObjectReader(region, bucket).getTransactions(jmeterCsv);
    if (perfmonCsv != null) {
        PerfMonDataEntries perfMonDataEntries = new PerfMonS3ObjectDataReader(region, bucket).getDataEntires(perfmonCsv);
        testSet.executeServerSideTests(perfMonDataEntries);
    }
    testSet.executeClientSideTests(jmeterTransactions);
    String testExecutionReport = testSet.getTestExecutionReport();
    String testSetExecutionSummaryReport = new TestSetReporter(testSet).getTestSetExecutionSummaryReport();
    String combinedTestReport = String.format("\n%s%s\n", testExecutionReport, testSetExecutionSummaryReport);
    String combinedTestReportS3Path = s3Client.putS3Object("output/verify.log", combinedTestReport);
    response.setCombinedTestReport(combinedTestReportS3Path);
    log.info(combinedTestReport);
    long testSetExecEnd = System.currentTimeMillis();
    long testExecTime = testSetExecEnd - testSetExecStart;
    log.info("Total verify stage execution time:    {}ms", testExecTime);
    if (testSet.getFailCount() + testSet.getErrorCount() != 0) {
        response.setExitCode(1);
    } else {
        response.setExitCode(0);
    }
}
Also used : ClientSideTest(uk.co.automatictester.lightning.tests.ClientSideTest) ServerSideTest(uk.co.automatictester.lightning.tests.ServerSideTest) TestSetReporter(uk.co.automatictester.lightning.reporters.TestSetReporter) JMeterCSVS3ObjectReader(uk.co.automatictester.lightning.lambda.readers.JMeterCSVS3ObjectReader) LightningXMLS3ObjectReader(uk.co.automatictester.lightning.lambda.readers.LightningXMLS3ObjectReader) PerfMonS3ObjectDataReader(uk.co.automatictester.lightning.lambda.readers.PerfMonS3ObjectDataReader) PerfMonDataEntries(uk.co.automatictester.lightning.data.PerfMonDataEntries) TestSet(uk.co.automatictester.lightning.TestSet)

Example 4 with ServerSideTest

use of uk.co.automatictester.lightning.tests.ServerSideTest in project lightning by automatictester.

the class LightningMojo method runTests.

private void runTests() {
    long testSetExecStart = System.currentTimeMillis();
    LightningXMLFileReader xmlFileReader = new LightningXMLFileReader();
    xmlFileReader.readTests(testSetXml);
    List<ClientSideTest> clientSideTests = xmlFileReader.getClientSideTests();
    List<ServerSideTest> serverSideTests = xmlFileReader.getServerSideTests();
    testSet = new TestSet(clientSideTests, serverSideTests);
    jmeterTransactions = new JMeterCSVFileReader().getTransactions(jmeterCsv);
    if (perfmonCsv != null) {
        PerfMonDataEntries perfMonDataEntries = new PerfMonDataReader().getDataEntires(perfmonCsv);
        testSet.executeServerSideTests(perfMonDataEntries);
    }
    testSet.executeClientSideTests(jmeterTransactions);
    log(testSet.getTestExecutionReport());
    log(new TestSetReporter(testSet).getTestSetExecutionSummaryReport());
    long testSetExecEnd = System.currentTimeMillis();
    long testExecTime = testSetExecEnd - testSetExecStart;
    log(String.format("Execution time:    %dms", testExecTime));
    if (testSet.getFailCount() + testSet.getErrorCount() != 0) {
        exitCode = 1;
    }
}
Also used : ClientSideTest(uk.co.automatictester.lightning.tests.ClientSideTest) ServerSideTest(uk.co.automatictester.lightning.tests.ServerSideTest) TestSetReporter(uk.co.automatictester.lightning.reporters.TestSetReporter) JMeterCSVFileReader(uk.co.automatictester.lightning.readers.JMeterCSVFileReader) PerfMonDataReader(uk.co.automatictester.lightning.readers.PerfMonDataReader) PerfMonDataEntries(uk.co.automatictester.lightning.data.PerfMonDataEntries) LightningXMLFileReader(uk.co.automatictester.lightning.readers.LightningXMLFileReader) TestSet(uk.co.automatictester.lightning.TestSet)

Example 5 with ServerSideTest

use of uk.co.automatictester.lightning.tests.ServerSideTest in project lightning by automatictester.

the class TestSet method executeServerSideTests.

public void executeServerSideTests(PerfMonDataEntries dataEntires) {
    StringBuilder output = new StringBuilder();
    for (ServerSideTest test : getServerSideTests()) {
        test.execute(dataEntires);
        setCounts(test);
        output.append(test.getTestExecutionReport()).append(System.lineSeparator());
    }
    testExecutionReport += output;
}
Also used : ServerSideTest(uk.co.automatictester.lightning.tests.ServerSideTest)

Aggregations

ServerSideTest (uk.co.automatictester.lightning.tests.ServerSideTest)7 ClientSideTest (uk.co.automatictester.lightning.tests.ClientSideTest)6 TestSet (uk.co.automatictester.lightning.TestSet)5 PerfMonDataEntries (uk.co.automatictester.lightning.data.PerfMonDataEntries)4 TestSetReporter (uk.co.automatictester.lightning.reporters.TestSetReporter)4 JMeterCSVFileReader (uk.co.automatictester.lightning.readers.JMeterCSVFileReader)3 LightningXMLFileReader (uk.co.automatictester.lightning.readers.LightningXMLFileReader)3 PerfMonDataReader (uk.co.automatictester.lightning.readers.PerfMonDataReader)3 Test (org.testng.annotations.Test)2 PassedTransactionsTest (uk.co.automatictester.lightning.tests.PassedTransactionsTest)2 File (java.io.File)1 ArrayList (java.util.ArrayList)1 CoreMatchers.containsString (org.hamcrest.CoreMatchers.containsString)1 JMeterCSVS3ObjectReader (uk.co.automatictester.lightning.lambda.readers.JMeterCSVS3ObjectReader)1 LightningXMLS3ObjectReader (uk.co.automatictester.lightning.lambda.readers.LightningXMLS3ObjectReader)1 PerfMonS3ObjectDataReader (uk.co.automatictester.lightning.lambda.readers.PerfMonS3ObjectDataReader)1 RespTimeAvgTest (uk.co.automatictester.lightning.tests.RespTimeAvgTest)1