Creating class level xunit report files with QtTestLib

Working on my new project SynPlayer – an controller to the Synology AudioStation based on C++ and Qt – it becomes evident to set up a testing infrastructure. Keeping dependency small and simple the decision was to facilitate QtTestLib for testing purposes.
To control test parameters and enable result reporting you may set the environment variable TESTARGS, for example to write the tests to the file xunit-result.xml in XUnit XML format pass:

TESTARGS="-o xunit-result.xml,xunitxml"

Unfortunately running multiple test cases will cause overwriting the report file with the result for each test, ending up eventually having only the last result.

The test case invocation is done by one of those methods:

QTest::qExec(QObject *test, QStringList args)
QTest::qExec(QObject *test, int argc, char **argv)

whereas the command line arguments taken from the TESTARGS variable are passed right into the tests. Supervising the passed parameters let raise the idea of manipulating the output parameters right before the test execution, preferably append the name of the test to the result file name.

Every test (case) should be derived from the QObject class. Thanks to the QObjects infrastructure the concrete class name can be easily obtained from the QMetaObjects method className().

For the first try the code sticks to statically replacing all output options with one option enforcing the results to be stored in XUnit XML format in a file named like xunit-result-TestClassName.xml ending up in the following code:

#include <QTest>
#include<QDebug>
#include<QStringList>
#include<QObject>

#include "logincommandtest.h"
#include "remoteplayertest.h"



#ifdef X_BUILD_TEST_CODE
/**
 * @brief getArgumentsWithTestName add name of the executing test to the test result file. Force output as xunit xml
 * @param arguments arguments to keep
 * @param test test to be executed
 * @return cmd line arguments enriched by the output flag
 */
QStringList getArgumentsWithTestName(QStringList arguments, QObject* test);

/**
 * @brief mangleArguments removes all output arguments from the cmd line
 * @param argc number of cmd line arguments
 * @param argv the cmd args itselfs
 * @return cmd line without output arguments
 */
QStringList mangleArguments(int argc, char** argv);

/**
 * @brief runTest simple wrapper enforcing the setting of the desired xunit result file name
 * @param test test to run
 * @param arguments cmd to pass to test execution
 * @return  result of executed test
 */
int runTest(QObject *test, QStringList arguments);

/**
 * @brief main Test main method. Execute tests
 * @param argc number of cmd line parameters (including executable)
 * @param argv cmd line parameter values (including executable)
 * @return aggregated test result
 */
int main(int argc, char **argv) {
    // remove all output options from cmd line
    QStringList defaultArguments = mangleArguments(argc,argv);

    // define tests to run
    LoginCommandTest loginCmdTest;
    RemotePlayerTest remotePlayerTest;

    // run tests
    int result = runTest(&loginCmdTest, defaultArguments);
    result |= runTest(&remotePlayerTest, defaultArguments);

    return result;
}


int runTest(QObject* test, QStringList arguments) {
    QStringList targs = getArgumentsWithTestName(arguments,test);
    qDebug()<<"Executing with &#91;"<<targs<<"&#93;";
    return QTest::qExec(test,targs);
}

QStringList getArgumentsWithTestName(QStringList arguments, QObject* testClass) {
    QStringList newArgs = QStringList(arguments);
    // append output flag
    newArgs.append("-o");
    // append filename build from constant value and name of the class
    newArgs.append(QString("xunit-result-%1.xml,xunitxml").arg(testClass->metaObject()->className()));

    return newArgs;
}

QStringList mangleArguments(int argc, char** argv) {
    // cleaned up cmd line
    QStringList newArgs;

    // iterate over the input parameter
    for(int i = 0; i < argc; i++) {
        // current arg
        QString arg(argv&#91;i&#93;);
        // previous ard
        QString prev = (i > 0) ? QString(argv[i-1]) : QString();

        if(prev == "-o"  || arg == "-o") {
            // remove arguments
        }
        else {
            // append to new cmd line
            newArgs.append(arg);
        }
    }
    return newArgs;
}

#endif //X_BUILD_TEST_CODE

which yields the result files xunit-result-LoginCommandTest.xml and xunit-result-RemotePlayerTest.xml.