Are you doing exports from Aggregate? If so, I expect that's what causes problems and I would encourage you to stop doing them. You can instead pull with Briefcase making sure to only pull new submissions.
We have stressed tested Central with millions of submissions and confirmed that it remains performant. We have stress tested with media at smaller scale and confirmed performance but have not done tests with tens of thousands of images. I would expect that exports would take a while but would not affect the rest of Central's functions. At that scale, I would encourage using the OData feed to do analysis instead of relying on exports. Accessing images on PowerBI - #2 by LN has more on pulling images into PowerBI/Excel using the Central API.