Skip to content

Added arm benchmark + improved running process of benchmarks

Carlos Mastalli requested to merge cmastall/crocoddyl:topic/arm-benchmark into devel

This PR mainly tackles #259 (closed). Additionally it included the following improvements in the benchmark

  • Added ShootingProblem.calc and ShootingProblem.calcDiff benchmarks for all the cases.
  • Added pure Python benchmark for the talos arm problem
  • Improved the running process, now we can do it like this:
make benchmark-unicycle INPUT=100 # 100 is the number of trials
make benchmark-lqr INPUT=100 # 100 is the number of trials
make benchmark-arm-manipulator INPUT=100 # 100 is the number of trials
make benchmark-quadrupedal-gaits INPUT='100 walk' # walk is the gait, we have other options (trot, pace, bound and jump)
make benchmark-bipedal-walk INPUT=100 # 100 is the number of trials

Additionally, I don't see any breaking of the determinism (neither c++ and Python bindings code) using multithreading as reported #256 (closed). Indeed, I notice an improvement in the computation time of calcDiff. These are the results

C++:
  DDP.solve [ms]: 1.59551 (1.55648-1.94576)
  ShootingProblem.calc [ms]: 0.46114 (0.43442-0.676178)
  ShootingProblem.calcDiff [ms]: 0.560603 (0.497396-0.984347)
Python bindings:
  DDP.solve [ms]: 1.69171571732 (1.63292884827, 2.27212905884)
  ShootingProblem.calc [ms]: 0.49816608429 (0.463008880615, 0.734090805054)
  ShootingProblem.calcDiff [ms]: 0.578708648682 (0.531911849976, 1.02019309998)

and without multithreading we compute calcDiff around 2.5ms.

@wxmerkt could you run the arm-manipulation benchmark in your PC?

@proyan it's working on the c++ benchmark for quadrupedal-gaits

Edited by Carlos Mastalli

Merge request reports