Thermodynamics of open quantum systems: from a critical study to the optimization of non-equilibrium heat engines