Early galaxy growth: mergers or gravitational instability?