Very few requests in report when testing over HTTPs

Hi,

When running XLT load tests against a web app using HTTPs, it seems the report only shows very few requests. When running the same tests over unencrypted HTTP, all requests will show up in the report.

Is this a limitation, or am I missing something here?

Regards,
Benjamin

There are no limits. Have you checked that the calls are really made? You can check the CSV files in theresult data to see if the requests are made and recorded.

Besides that, examples would be nice.

Hi,

Thanks for your quick reply.
I’ve verified the behavior again. Running a simple test against the app, using HTTP a reasonable amount of requests shows up in the report and requests are distributed over the whole time of the run. Running the same test against the same instance of the app, using HTTPS instead, only few requests show up, and also they are limited to the very beginning of the run.
In both cases the test case passes, we see the same amount and distribution over time of actions and also the action timings are pretty the same.

Some facts:
XLT version: 5.4.0

HTTP scenario:
Transaction: 1; Users: 1; Measurement Period: 60s
Actions: 16 (distributed over the complete runtime, looking reasonable)
Requests: 182 (distributed over the complete runtime, looking reasonable)

HTTPS scenario:
Transaction: 1; Users: 1; Measurement Period: 60s
Actions: 16 (distributed over the complete runtime, looking reasonable)
Requests: 77 (only at the beginning within the first ~2s of the run)

The report matches exactly the amount and timing of the CSV result files.

As in both cases the test passes, the amount and timings of the actions match and we don’t see any errors on executing the actions, I suspect the requests are made in both cases. In the HTTPS scenario, besides the initial requests, they are just not recorded.

Any ideas?

Regards,
Benjamin

XLT obeys the caching headers of the server. Could it be that your HTTPS is cacheable?

Run a test case in Eclipse or any IDE as Junit test and turn info logging on (log4j-dev.properties) and see what XLT logs. In case of caching it will likely state that.

Hi,

Thanks for the hint! Caching could indeed be the problem here. The responses seem to include cache-control: private, max-age=864000, immutable for HTTPS, but no cache-control directive for HTTP. While I’ll try to “fix” this on the server side, I’d also like to try switching off caching on the XLT client side.
Would com.xceptance.xlt.staticContentCache = false switch off client side caching completely? Or is there anything else? Because a quick test having this setting does not seem to do the trick, still seeing only limited requests when running over HTTPS.

Thanks for your help.

Regards,
Benjamin

Hi Benjamin,

That should do the trick.

Did you run your quick test inside your IDE (or via Maven)? If so, please make sure that you put this setting into dev.properties. Otherwise, it should go into test.properties or project.properties.

Thanks,
Hartmut

Hi Hartmut,

I ran it in the IDE. Originally I set it in default.properties, and now reran having it in dev.properties. In the output I can see [11:21:01,185] DEBUG [main] - | com.xceptance.xlt.staticContentCache = false being logged, so unless there is some other mechanism that takes precedence, caching should be off.
Nevertheless, it seems to have no effect. Using HTTPS I still see only 77 requests (all within first ~2s of the run) in the xlt.log, while with HTTP there are 181 (distributed over the runtime of ~40s).

Regards,
Benjamin

Could you please share the full log? Please send it to support@xceptance.com.

Hi,

I’ve sent in two logs, one for each of a HTTP and HTTPS run.

Regards,
Benjamin

Thanks for that! According to the logs, you perform your tests with a real Chrome browser (via XltChromeDriver). In this case, the setting com.xceptance.xlt.staticContentCache has no effect. It would apply only if you would use the emulated browser (HtmlUnit, via XltDriver).

So I just switched to xlt as driver. Now the testcase fails right at the beginning when it tries to find an element by it’s xpath, using isElementPresent(locator).
Does switching the driver from chrome to xlt come with the need to re-implement the tests?

Nevertheless, the main question remains why in the HTTPs case fewer requests are recorded! Disabling client-side caching would only be a workaround.

Because when the browser caches the resource, it is not fetched anymore and hence there is not data provided to XLT we could record. No call, no measurement. If you have a different idea, let us know.

In regards to the HtmlUnit: This is not a full browser and the JS and CSS it runs is off by a lot from what modern browsers do, hence your data might just not be there at all. The real browser path makes sense when you page is really heavy in terms of client side logic. When you have to run a lot of traffic and the client side is not logic heavy, you can run Htmlunit, but have to live with an almost request only approach (html is parsed and built, but JS logic has to be emulated). See the advanced test suite for that with its demonstration how to use HtmlUnit and no JS.

Regarding the caching:
I would agree, if it was the same when using HTTP only.
But, when looking through the logs, I can see 55 responses having cache-control: no-store in the HTTP case, but only 1 in the HTTPs case. Where are they?
When accessing the WebApp using a browser from my desktop, checking the requests/responses using the WebDevTools, I can see a lot more using HTTPs than XLT does record. Just by opening the first page, I get 4 responses having cache-control: no-store, while XLT only records 1 during the complete test!

I’ll continue debugging…

Regarding the driver to use:
I guess I’ve to stick with a “real” browser than, and HTMLUnit is not an option.

Do we have a chance to see the application in action and check? We only record what Chrome issues as part of its performance timing API.

Would need to check if we can arrange this and how. I’ll get back to you.