Firstly, for the images which aren't displayed on-load, defer the load of these images (lazy load). This makes it so they don't interfere with the load-time of the rest of the page which will be actually used by the user. Secondly, I'm a bit of a speed freak, so I'd never stick any more than 3-4 images in a carousel.
Weighing up what value the images bring to the user, by the decrease in page speed, to me it's pretty clear page speed wins.