Siesta 4.2.0 – Improved Recorder + Faster Testing
We’ve just released Siesta 4.2.0 – an important Siesta milestone and in this blog post we’ll highlight its most important features.
Introducing Ariadne – The New Query Finder
The internals of the Siesta event recorder have been completely rewritten (more specifically, the target finding algorithm). In previous versions, the recorder sometimes found either a very long CSS query or it could not figure out a stable CSS query at all. We decided that this feature was important enough to warrant a new implementation.
In this new release, we have moved the query finding functionality to a new project called Ariadne. Ariadne finds the “optimal” CSS query for any given DOM element and it handles Ext JS component queries just as well. Let’s check the key requirements we had for this task and design choices made during the development.
First of all, how does one define the “optimal” query, seen from the event recorder’s perspective?
The “optimal” query has to be stable – it should survive reasonable markup changes. This means the recorder should try to avoid all types of positional selectors such as: “>” (direct child), “nth-child()”, “nth-of-type()”. Those are very fragile and will break as soon as new element is added to the DOM, or when the position of an element changes. In the same time there are situations when it is not possible to build a unique query, without using those selectors. In such cases Ariadne falls back to a positional selector, but it uses them as little as possible. Ariadne prefers “nth-of-type(1)” selector over the “nth-child()”, since it survives addition of element with another tag name.
The “optimal” query also has to be short and readable. This means only specific CSS classes (or DOM attributes) should be included in the query, and as few as possible of them. This requirement also improves the stability of the query, since the bigger query is – the higher risk that a markup change will break the query.
And lastly of course, the finding query should be fast.
We have implemented a quite sophisticated algorithm to achieve all these requirements and here are some examples from ExtJS kitchensink (action targets are marked with red cross):
Example 1
Old recorder:
Previous implementation finds some huge, unreadable CSS query:
CSS query:
#content-panel .kitchensink-example .x-panel.x-panel-default-framed.x-resizable.x-panel-resizable.x-panel-default-framed-resizable.x-border-box .x-panel-body.x-panel-body-default-framed.x-column-layout-ct.x-resizable.x-panel-body-resizable.x-panel-body-default-framed-resizable .x-container.x-column .x-form-layout-wrap.x-form-layout-auto-label .x-field.x-form-item.x-form-type-text.x-form-form-item .x-form-item-body.x-form-text-field-body .x-form-trigger-wrap .x-form-text-wrap .x-form-text
Component query:
>>#content-panel form-multicolumn[title=Multi Column Form] textfield[inputType=text]
Composite query:
#content-panel form-multicolumn[title=Multi Column Form] textfield[inputType=text] => .x-form-text
Ariadne:
Now the same recording repeated with Ariadne – resulting in a much shorter, more stable and more readable query:
CSS query:
#content-panel .x-container:nth-of-type(1) .x-field:nth-of-type(1) .x-form-text
Component query:
>>#content-panel textfield[fieldLabel=First Name]
Composite query:
#content-panel textfield[fieldLabel=First Name] => .x-form-text
Example 2
Old recorder:
Previous implementation was not able to find any query and only generated coordinate: [ 546, 411 ]
Ariadne:
Ariadne demonstrates stable results:
CSS query:
#content-panel .x-container:nth-of-type(1) .x-field:nth-of-type(2) .x-form-text
Component query:
>>#content-panel textfield[fieldLabel=Last Name]
Composite query:
#content-panel textfield[fieldLabel=Last Name] => .x-form-text
Example 3
Old recorder:
Previous implementation only found a component query: >>#content-panel layout-cardtabs tabbar tab[text=Tab 2]
Ariadne:
Ariadne finds all 3 types of queries, and the component query is shorter:
CSS query:
#content-panel .x-tab-inner:textEquals(Tab 2)
Component query:
>>#content-panel tab[text=Tab 2]
Composite query:
#content-panel tab[text=Tab 2] => .x-tab-inner
In general you should notice a significant improvement in the quality of generated queries.
Playback improvements
While improving the quality of the queries, we’ve also put some effort into improving the quality of the playback, so that recorded actions could be replayed 100% reliably. Notably Siesta now respects the caret position in text fields, supports HOME/END key caret navigation, properly simulates “click” events in certain edge cases. All these minor improvements should allow you to successfully replay what you’ve recorded.
SlimerJS upgrade
As you know Siesta includes SlimerJS as a headless test launcher. However as of version 0.10.0, SlimerJS is no longer headless and it requires Firefox to be installed in the OS along with a graphical server (xvfb or similar). We did not upgrade SlimerJS for a long time because of this reason, but now the Firefox version it implements is too old and outdated (39 and current is 49). So we had to upgrade SlimerJS for our users to get consistent results with the real Firefox browser. If you are using SlimerJS for running your tests in a server environment you will need to install `xvfb` and `Firefox`. It should be just a matter of running `apt-get install firefox xvfb` on Debian-like systems.
The good news is that once you have xvfb installed on your machine, you can significantly increase the test suite execution speed (see next section)
Xvfb – running several browser sessions simultaneously on local machine
As your test suite grows, execution time can become a bottleneck. The solution for that is parallelization – running your test suite in several parallel “threads” or “workers” (as it’s called in Siesta). Since version 4, Siesta supports the `–max-workers` command line config. However, when several browser windows are opened simultaneously on the same desktop, they start “focus fighting” – because focus can only be in one place by definition. Because of that, previously the primary use case for `–max-workers` config was in cloud testing providers, since on local machine it didn’t work reliably.
Starting from this release, Siesta supports a new command line switch `–xvfb`. It requires the presence of “xvfb” in the host system, which is basically a virtual desktop server. With this switch, Siesta will wrap every launched browser process with a virtual desktop, meaning the “focus fighting” problem goes away and one can safely specify reasonably big values for `–max-workers` config (limited by CPU mostly). Values between 5-10 works great for us:
Execution time for `–xvfb –max-workers 1` ~ 13 min:
Execution time for `–xvfb –max-workers 7` ~ 3 min:
This option is supported on Linux only for now. Support for MacOS may be added in future releases. Unfortunately we couldn’t find any xvfb-similar tool for Windows (those that exists don’t have command line interface), so Windows support is currently not feasible.
Conclusion
We encourage you to give this release a try and use the new features and improvements to speed up your test suite. Please also share your feedback in our forums.
Enjoy the Siesta!