Search in sources :

Example 1 with TestSet

use of uk.co.automatictester.lightning.TestSet in project lightning by automatictester.

the class LightningTask method runTests.

void runTests() {
    long testSetExecStart = System.currentTimeMillis();
    LightningXMLFileReader xmlFileReader = new LightningXMLFileReader();
    xmlFileReader.readTests(extension.getTestSetXml());
    List<ClientSideTest> clientSideTests = xmlFileReader.getClientSideTests();
    List<ServerSideTest> serverSideTests = xmlFileReader.getServerSideTests();
    testSet = new TestSet(clientSideTests, serverSideTests);
    jmeterTransactions = new JMeterCSVFileReader().getTransactions(extension.getJmeterCsv());
    if (extension.getPerfmonCsv() != null) {
        PerfMonDataEntries perfMonDataEntries = new PerfMonDataReader().getDataEntires(extension.getPerfmonCsv());
        testSet.executeServerSideTests(perfMonDataEntries);
    }
    testSet.executeClientSideTests(jmeterTransactions);
    log(testSet.getTestExecutionReport());
    log(new TestSetReporter(testSet).getTestSetExecutionSummaryReport());
    long testSetExecEnd = System.currentTimeMillis();
    long testExecTime = testSetExecEnd - testSetExecStart;
    log(String.format("Execution time:    %dms", testExecTime));
    if (testSet.getFailCount() + testSet.getErrorCount() != 0) {
        exitCode = 1;
    }
}
Also used : ClientSideTest(uk.co.automatictester.lightning.tests.ClientSideTest) ServerSideTest(uk.co.automatictester.lightning.tests.ServerSideTest) TestSetReporter(uk.co.automatictester.lightning.reporters.TestSetReporter) JMeterCSVFileReader(uk.co.automatictester.lightning.readers.JMeterCSVFileReader) PerfMonDataReader(uk.co.automatictester.lightning.readers.PerfMonDataReader) PerfMonDataEntries(uk.co.automatictester.lightning.data.PerfMonDataEntries) LightningXMLFileReader(uk.co.automatictester.lightning.readers.LightningXMLFileReader) TestSet(uk.co.automatictester.lightning.TestSet)

Example 2 with TestSet

use of uk.co.automatictester.lightning.TestSet in project lightning by automatictester.

the class TeamCityReporterTest method testPrintTeamCityVerifyStatistics.

@Test
public void testPrintTeamCityVerifyStatistics() {
    final PassedTransactionsTest clientTest = mock(PassedTransactionsTest.class);
    when(clientTest.getName()).thenReturn("Failed transactions");
    when(clientTest.getActualResult()).thenReturn(1);
    final ServerSideTest serverTest = mock(ServerSideTest.class);
    when(serverTest.getName()).thenReturn("Memory utilization");
    when(serverTest.getActualResult()).thenReturn(45);
    TestSet testSet = mock(TestSet.class);
    when(testSet.getClientSideTests()).thenReturn(new ArrayList<ClientSideTest>() {

        {
            add(clientTest);
        }
    });
    when(testSet.getServerSideTests()).thenReturn(new ArrayList<ServerSideTest>() {

        {
            add(serverTest);
        }
    });
    String output = new TeamCityReporter(testSet).getTeamCityVerifyStatistics();
    assertThat(output, containsString("##teamcity[buildStatisticValue key='Failed transactions' value='1']"));
    assertThat(output, containsString("##teamcity[buildStatisticValue key='Memory utilization' value='45']"));
}
Also used : ServerSideTest(uk.co.automatictester.lightning.tests.ServerSideTest) ClientSideTest(uk.co.automatictester.lightning.tests.ClientSideTest) PassedTransactionsTest(uk.co.automatictester.lightning.tests.PassedTransactionsTest) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) TestSet(uk.co.automatictester.lightning.TestSet) Test(org.testng.annotations.Test) PassedTransactionsTest(uk.co.automatictester.lightning.tests.PassedTransactionsTest) ClientSideTest(uk.co.automatictester.lightning.tests.ClientSideTest) ServerSideTest(uk.co.automatictester.lightning.tests.ServerSideTest)

Example 3 with TestSet

use of uk.co.automatictester.lightning.TestSet in project lightning by automatictester.

the class JenkinsReporterTest method testSetJenkinsBuildName_verify.

@Test
public void testSetJenkinsBuildName_verify() throws FileNotFoundException {
    TestSet testSet = mock(TestSet.class);
    when(testSet.getTestCount()).thenReturn(3);
    when(testSet.getFailCount()).thenReturn(1);
    when(testSet.getErrorCount()).thenReturn(1);
    new JenkinsReporter(testSet).setJenkinsBuildName();
    File lightningFile = new File("lightning-jenkins.properties");
    String text = new Scanner(lightningFile).useDelimiter("\\A").next();
    lightningFile.delete();
    assertThat(text, containsString("In Jenkins Build Name Setter Plugin, define build name as: ${BUILD_NUMBER} - ${PROPFILE,file=\"lightning-jenkins.properties\",property=\"result.string\"}"));
    assertThat(text, containsString("result.string=Tests executed\\: 3, failed\\: 2"));
}
Also used : Scanner(java.util.Scanner) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) File(java.io.File) TestSet(uk.co.automatictester.lightning.TestSet) Test(org.testng.annotations.Test)

Example 4 with TestSet

use of uk.co.automatictester.lightning.TestSet in project lightning by automatictester.

the class LightningHandler method runTests.

private void runTests() {
    long testSetExecStart = System.currentTimeMillis();
    LightningXMLS3ObjectReader xmlFileReader = new LightningXMLS3ObjectReader(region, bucket);
    xmlFileReader.readTests(xml);
    List<ClientSideTest> clientSideTests = xmlFileReader.getClientSideTests();
    List<ServerSideTest> serverSideTests = xmlFileReader.getServerSideTests();
    testSet = new TestSet(clientSideTests, serverSideTests);
    jmeterTransactions = new JMeterCSVS3ObjectReader(region, bucket).getTransactions(jmeterCsv);
    if (perfmonCsv != null) {
        PerfMonDataEntries perfMonDataEntries = new PerfMonS3ObjectDataReader(region, bucket).getDataEntires(perfmonCsv);
        testSet.executeServerSideTests(perfMonDataEntries);
    }
    testSet.executeClientSideTests(jmeterTransactions);
    String testExecutionReport = testSet.getTestExecutionReport();
    String testSetExecutionSummaryReport = new TestSetReporter(testSet).getTestSetExecutionSummaryReport();
    String combinedTestReport = String.format("\n%s%s\n", testExecutionReport, testSetExecutionSummaryReport);
    String combinedTestReportS3Path = s3Client.putS3Object("output/verify.log", combinedTestReport);
    response.setCombinedTestReport(combinedTestReportS3Path);
    log.info(combinedTestReport);
    long testSetExecEnd = System.currentTimeMillis();
    long testExecTime = testSetExecEnd - testSetExecStart;
    log.info("Total verify stage execution time:    {}ms", testExecTime);
    if (testSet.getFailCount() + testSet.getErrorCount() != 0) {
        response.setExitCode(1);
    } else {
        response.setExitCode(0);
    }
}
Also used : ClientSideTest(uk.co.automatictester.lightning.tests.ClientSideTest) ServerSideTest(uk.co.automatictester.lightning.tests.ServerSideTest) TestSetReporter(uk.co.automatictester.lightning.reporters.TestSetReporter) JMeterCSVS3ObjectReader(uk.co.automatictester.lightning.lambda.readers.JMeterCSVS3ObjectReader) LightningXMLS3ObjectReader(uk.co.automatictester.lightning.lambda.readers.LightningXMLS3ObjectReader) PerfMonS3ObjectDataReader(uk.co.automatictester.lightning.lambda.readers.PerfMonS3ObjectDataReader) PerfMonDataEntries(uk.co.automatictester.lightning.data.PerfMonDataEntries) TestSet(uk.co.automatictester.lightning.TestSet)

Example 5 with TestSet

use of uk.co.automatictester.lightning.TestSet in project lightning by automatictester.

the class LightningMojo method runTests.

private void runTests() {
    long testSetExecStart = System.currentTimeMillis();
    LightningXMLFileReader xmlFileReader = new LightningXMLFileReader();
    xmlFileReader.readTests(testSetXml);
    List<ClientSideTest> clientSideTests = xmlFileReader.getClientSideTests();
    List<ServerSideTest> serverSideTests = xmlFileReader.getServerSideTests();
    testSet = new TestSet(clientSideTests, serverSideTests);
    jmeterTransactions = new JMeterCSVFileReader().getTransactions(jmeterCsv);
    if (perfmonCsv != null) {
        PerfMonDataEntries perfMonDataEntries = new PerfMonDataReader().getDataEntires(perfmonCsv);
        testSet.executeServerSideTests(perfMonDataEntries);
    }
    testSet.executeClientSideTests(jmeterTransactions);
    log(testSet.getTestExecutionReport());
    log(new TestSetReporter(testSet).getTestSetExecutionSummaryReport());
    long testSetExecEnd = System.currentTimeMillis();
    long testExecTime = testSetExecEnd - testSetExecStart;
    log(String.format("Execution time:    %dms", testExecTime));
    if (testSet.getFailCount() + testSet.getErrorCount() != 0) {
        exitCode = 1;
    }
}
Also used : ClientSideTest(uk.co.automatictester.lightning.tests.ClientSideTest) ServerSideTest(uk.co.automatictester.lightning.tests.ServerSideTest) TestSetReporter(uk.co.automatictester.lightning.reporters.TestSetReporter) JMeterCSVFileReader(uk.co.automatictester.lightning.readers.JMeterCSVFileReader) PerfMonDataReader(uk.co.automatictester.lightning.readers.PerfMonDataReader) PerfMonDataEntries(uk.co.automatictester.lightning.data.PerfMonDataEntries) LightningXMLFileReader(uk.co.automatictester.lightning.readers.LightningXMLFileReader) TestSet(uk.co.automatictester.lightning.TestSet)

Aggregations

TestSet (uk.co.automatictester.lightning.TestSet)10 Test (org.testng.annotations.Test)6 CoreMatchers.containsString (org.hamcrest.CoreMatchers.containsString)5 ClientSideTest (uk.co.automatictester.lightning.tests.ClientSideTest)5 ServerSideTest (uk.co.automatictester.lightning.tests.ServerSideTest)5 TestSetReporter (uk.co.automatictester.lightning.reporters.TestSetReporter)4 PerfMonDataEntries (uk.co.automatictester.lightning.data.PerfMonDataEntries)3 JMeterCSVFileReader (uk.co.automatictester.lightning.readers.JMeterCSVFileReader)3 LightningXMLFileReader (uk.co.automatictester.lightning.readers.LightningXMLFileReader)3 PerfMonDataReader (uk.co.automatictester.lightning.readers.PerfMonDataReader)3 File (java.io.File)2 Scanner (java.util.Scanner)1 Element (org.w3c.dom.Element)1 JUnitReporter (uk.co.automatictester.lightning.ci.JUnitReporter)1 JMeterCSVS3ObjectReader (uk.co.automatictester.lightning.lambda.readers.JMeterCSVS3ObjectReader)1 LightningXMLS3ObjectReader (uk.co.automatictester.lightning.lambda.readers.LightningXMLS3ObjectReader)1 PerfMonS3ObjectDataReader (uk.co.automatictester.lightning.lambda.readers.PerfMonS3ObjectDataReader)1 LightningTest (uk.co.automatictester.lightning.tests.LightningTest)1 PassedTransactionsTest (uk.co.automatictester.lightning.tests.PassedTransactionsTest)1