Improving the quality of Jami

Unit-tests

  • It is harder to make unit-test on Jami project because of the race conditions on multi-level dependance.

  • We use lcov for the coverage, you can find the lcov’s configuration in the daemon’s Makefile.am. Also, the coverage can be found at https://docs.jami.net/coverage/ and https://dl.jami.net/docs/dhtnet/coverage

  • A system needs to be implemented to start convincing the team to make a unit-test for new code before merging

  • You can launch them by doing ‘make check’ in the daemon folder or separately in the unit-test folder with gdb: ‘gdb ut_media_encoder’

  • The environment needs to be set with ‘–disable-shared’ during the ’./configure’ command

  • Clients also have tests (cf jami-client-qt/tests for Desktop, jami-client-android/jami-android/app/src/androidTest for Android)

Integration tests

  • Each commit goes through integration tests in dockers on the build machines you can find the details at: jenkins.jami.net

  • Code-review is made by a fellow developer, sometimes the code is reviewed by the same developer, this should be avoided to emphasize Linus’ law. The ‘Jenkins verified’ label is sometimes discarded and replaced by +1 from a developer, this should also be avoided.

  • Sonarqube lets Jenkins build Jami and verify linting. You can find filters and results at: sonar- jami.savoirfairelinux.net Sonar uses clang-tidy as a preprocessor linting compilator, you can find clang’s filters in .clang-tidy file in the daemon folder.

  • On SFLVault sonarqube can be found at service m#2637 and admin logins at service s#7169

Doc and feedback:

  • You can find all the documentation on docs.jami.net

  • Issues are made by developers or users on git.jami.net

Agent

Every night, hundred of calls are tested via 2 Agents and a message is posted every morning in the chat with the result (if all call succeeded or not)

Smoke tests

Before each releases every clients MUST past a list of scenarios.

Scenarios are described here: Jami smoke tests

They are reviewed by QA dpt. before sending it to the developers if needed.

If a release contains a network commit that has been merged, the QA dept. Should be able to automate the different connectivity tests (as descibed below in Calls configurations)

Calls configurations.

This is the list of network configurations that need to be tested:

(IPv4 | IPv6) + (TURN | !TURN) + (STUN | !STUN) + (UPnP | !UPnP) for both sides.

If both sides are IPv4 only without TURN/STUN/UPnP, the call should be only local.

What needs to be done

  • Establish a system within the team to assure maintenance and creation of unit-tests.

  • Each major functionality should be tested as whole by adding a test (i.e. making sure a message was received, the call was ended well on both side, etc…)

  • Each new functionality should be tested on each platform before merging it to reduce regression

  • Integrate sonarqube on each client

  • Automate the testing of Jami’s behavior on network compatibility