Robustness of optimal working points for non-adiabatic holonomic quantum computation