We study the photon transfer along a linear array of three coupled cavities where the central one contains an interacting two-level system in the strong- and ultrastrong-coupling regimes. We find that an inhomogeneously coupled array forbids a complete single-photon transfer between the external cavities when the central one performs a Jaynes-Cummings dynamics. This is not the case in the ultrastrong-coupling regime, where the system exhibits singularities in the photon transfer time as a function of the cavity-qubit coupling strength. Our model can be implemented within the state-of-the-art circuit quantum electrodynamics technology and it represents a building block for studying photon state transfer through scalable cavity arrays.