Python unit-tests are passing even when it fails an assertion

After changing the way of that I run multiple testcases, I noticed that make test doesn't detect a failure even when an assertion fails.

These are the changes for running multiple testcase:

if __name__ == '__main__':
    test_classes_to_run = [StateVectorTest, StateMultibodyTest]
    loader = unittest.TestLoader()
    suites_list = []
    for test_class in test_classes_to_run:
        suite = loader.loadTestsFromTestCase(test_class)
        suites_list.append(suite)
    big_suite = unittest.TestSuite(suites_list)
    runner = unittest.TextTestRunner()
    results = runner.run(big_suite)

and when I added an intentional error. Let's say, I modified this line https://gepgitlab.laas.fr/loco-3d/crocoddyl/blob/cpp_devel/unittest/bindings/test_states.py#L32 to:

        if self.STATE.__class__ == crocoddyl.libcrocoddyl_pywrap.StateMultibody:
            self.assertTrue(np.allclose(2*self.STATE.diff(x0, x1)[3:], self.STATE_DER.diff(x0, x1)[3:], atol=1e-9),
                            "state.diff() function doesn't agree with Python bindings.")

Then running make test doesn't detect the problem, i.e.

      Start 20: pybinds-states
20/24 Test #20: pybinds-states ...................   Passed    0.60 sec
      Start 21: pybinds-actions
21/24 Test #21: pybinds-actions ..................   Passed    0.59 sec
      Start 22: pybinds-shooting
22/24 Test #22: pybinds-shooting .................   Passed    0.59 sec
      Start 23: pybinds-solvers
23/24 Test #23: pybinds-solvers ..................   Passed    0.82 sec
      Start 24: pybinds-costs
24/24 Test #24: pybinds-costs ....................   Passed    0.67 sec

100% tests passed, 0 tests failed out of 24

but running ctest -V let us know what happens, i.e.:

20: Test command: /usr/bin/python "/devel/crocoddyl/unittest/bindings/test_states.py"
20: Environment variables: 
20:  PYTHONPATH=/devel/crocoddyl/build/bindings/python:/opt/ros/dls-distro/lib/python2.7/dist-packages:/opt/ros/kinetic/lib/python2.7/dist-packages:/local/users/cmastall/lib/python2.7/dist-packages:/local/users/cmastall/lib/python2.7/site-packages:/opt/openrobots/lib/python2.7/dist-packages:/opt/openrobots/lib/python2.7/site-packages:/home/cmastall/.local/lib/python2.7/dist-packages:/home/cmastall/.local/lib/python2.7/site-packages
20: Test timeout computed to be: 1500
20: .......F..
20: ======================================================================
20: FAIL: test_python_derived_diff (__main__.StateMultibodyTest)
20: ----------------------------------------------------------------------
20: Traceback (most recent call last):
20:   File "/devel/crocoddyl/unittest/bindings/test_states.py", line 33, in test_python_derived_diff
20:     "state.diff() function doesn't agree with Python bindings.")
20: AssertionError: state.diff() function doesn't agree with Python bindings.
20: 
20: ----------------------------------------------------------------------
20: Ran 10 tests in 0.003s
20: 
20: FAILED (failures=1)
20/24 Test #20: pybinds-states ...................   Passed    0.59 sec

and we can also run manually this test, i.e.:

cmastall@yarisan:bindings$ python test_states.py 
.......F..
======================================================================
FAIL: test_python_derived_diff (__main__.StateMultibodyTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_states.py", line 33, in test_python_derived_diff
    "state.diff() function doesn't agree with Python bindings.")
AssertionError: state.diff() function doesn't agree with Python bindings.

----------------------------------------------------------------------
Ran 10 tests in 0.002s

FAILED (failures=1)

This is very alarming problem, we can introduce bugs without detecting them in the CI. I suspect there is a problem in the cmake submodule. @gsaurel could you help me with this issue?