Get help with testing, discuss unit testing strategies etc.


Post by zombeerose »

I'm looking for suggestions how to utilize Siesta for testing different build profiles. Within my ExtJS app.json file, I have a "classic" and "modern" profile, which yield different production packages for desktop and mobile platforms. Within my code base, files are organized like the following:
\includes\app (universal, common MVC application code)
\includes\lib (universal, common library code)
\includes\classic\src\app
\includes\classic\src\lib
\includes\modern\src\app
\includes\modern\src\lib

The folder structure for my unit tests are very similarly organized. Currently, the harness specifies a pageUrl instead of individual preloads, and I call harness.startFromUrl to load the list of tests, which is over 1000 files and growing.

What is the recommended way to "test" the different production builds (classic vs modern)?

Post by nickolay »

And what is the problem? You just specify different "pageUrl" for different bulids?

Post by zombeerose »

I would consider the problem is that I would have to create a separate index page, a separate harness to reference each index page, and manually identify the tests to use per profile. This makes it really difficult to do in a completely automated fashion. I basically can't expect to run one instance of Siesta and have it run everything.

Post by nickolay »

Do you run the same set of tests for classic and modern profiles?

Post by zombeerose »

The "universal" tests should be tested against each profile.

Post by nickolay »

A bit more magical harness configuration can handle that. Something like (just a source for inspiration):
var getUniversalTests = function (profile) {
    var baseUrl = profile == 'modern' ? 'modern/index.html' : 'classic/index.html'

    return {
        group       : 'Universal tests for profile: ' + profile,

        pageUrl     : baseUrl + '/something.html',

        items       : [

        ]
    }
}


harness.start([
    getUniversalTests('classic'),
    getUniversalTests('modern')
])

Post by zombeerose »

That's the sort of inspiration I was looking for! I didn't realize I could pass multiple definitions to the start() method, which could include separate pageUrl configs.

Now my last issue - is there anything similar to the startFromUrl() method that will fetch all the test items, which I can use within the test descriptor? For example, instead of an array for items: [], can I specify a url?

Thanks!

Post by zombeerose »

For example:
harness.start({
    group: 'classic',
    items: '/tests/discover?classic',
    pageUrl: '/tests/index?classic'
},{
    group: 'modern',
    items: '/tests/discover?modern',
    pageUrl: '/tests/index?modern'
});

Post by nickolay »

That is not supported unfortunately.. Should be doable in the user-space pretty easily though.

Post by zombeerose »

Thank you Nickolay for pointing me in the right direction. I have been able to update my harness configuration to address my needs. However, in doing so, I have encountered a quirk with the test tree in which branches with the same name can't be expanded at the same time. I have included a video link to demonstrate: https://drive.google.com/file/d/1Sm1tlg ... s2jCK/view

I believe it's an issue because I have the same universal tests included under different groups. I realized that in order to truly test the "universal" code, I need to test it against each build package.

My test group structure:
- classic package
- - universal
- - - app
- - - lib
- - classic
- - - app
- - - lib
- modern package
- - universal
- - - app
- - - lib
- - modern
- - - app
- - - lib

Please let me know if you have any suggestions. Thanks!

Post Reply