serenity-bdd/serenity-core

Generate serenity reports takes longer when use @singleBrowser #2679

ikallala posted onGitHub

Hi, I noticed that the generation of serenity report takes longer when we use @singleBrowser for the feature file (to use the same session for all scenarios in the same feature file) This is the duration of the report generation for the same feature report generation with tag takes 52 secondes report generation without tag take 22secondes


Tags can lead to more reports being generated, which leads to longer processing time. Try using @singlebrowser, which should not be included as a tag.

posted by wakaleo almost 3 years ago

how can we use @singlebrowser without including the tag ?

posted by ikallala almost 3 years ago

If you use "@singlebrowser" (in lowercase) it will automatically be excluded from the reporting.

posted by wakaleo almost 3 years ago

I already used it in lowercase, but it was included in the report with first letter uppercase image

posted by ikallala almost 3 years ago

what version are you using?

posted by wakaleo almost 3 years ago

I am using 3.1.1

posted by ikallala almost 3 years ago

Try with 3.1.16

posted by wakaleo almost 3 years ago

I tried with 3.1.16, the tag @singlebrowser was excluded but the difference in time duration of report generation persist

posted by ikallala almost 3 years ago

Can you provide a sample project that reproduces the issue?

posted by wakaleo almost 3 years ago

Please find the sample project here. Let me know if you need anything else. https://github.com/ikallala/SampleSerenityTest/tree/master I noticed that the report generation time increase from 3 s to 5 s when adding the tag

posted by ikallala almost 3 years ago

I can't get that test suite to run. But 3-5 seconds is a normal range. Also, cucumber-jvm-parallel-plugin is no longer supported (see https://github.com/temyers/cucumber-jvm-parallel-plugin). Can you provide a sample project that reproduces the 52 second delay you mentioned above, and that can run in any environment?

posted by wakaleo almost 3 years ago

Time report generation issue fixed after upgrade serenity to 3.1.20. But I got these issues when running the test suite (contains more than 1000 scenarios):

  • during the test execution :GC overhead limit exceeded

  • when generate report : Failed to generate report for Requirements report 5b5c1f00b946bcef89861afdc089e9c5bebdb5a2fffec0ed61367caa30eff821.html - java.util.concurrent.ExecutionException: java.lang.NullPointerException net.thucydides.core.requirements.reports.cucumber.FeatureFileScenarioOutcomes.scenarioOutcomeFrom(FeatureFileScenarioOutcomes.java:163) java.util.concurrent.ExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask.report (FutureTask.java:122) at java.util.concurrent.FutureTask.get (FutureTask.java:206) at net.thucydides.core.reports.html.Reporter.generateReports (Reporter.java:56) at net.thucydides.core.reports.html.Reporter.generateReportsFor (Reporter.java:32) at net.thucydides.core.reports.html.HtmlAggregateStoryReporter.generateReportsForTestResultsIn

posted by ikallala almost 3 years ago

"during the test execution :GC overhead limit exceeded" - your JVM has run out of memory. If this happens during the test execution, it is probably not directly related to Serenity, but to your test suite more generally. How you add that memory will depend on your project setup, but you might need to do something like this:

<build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.19</version>
        <configuration>
            <argLine>-Xmx1024m</argLine>
        </configuration>
      </plugin>
    </plugins>
  </build>

If the tests run out of memory, the generated test reports will be incomplete, so you will get errors during report generation.

posted by wakaleo almost 3 years ago

same issue , it disappears only when downgrade to 3.1.1

posted by ikallala almost 3 years ago

Can you check if the number of files in the target directory changes when you use the @singleBrowser tag?

posted by wakaleo almost 3 years ago

Closing as not reproducible with 3.2.0

posted by wakaleo almost 3 years ago

Fund this Issue

$0.00
Funded

Pull requests