I've noticed the slowness too
but .. they did warn us that expanding that big would cause a farm to 'act' slow like a multi layered - crowded treed , flowered farm .....
I've fully expanded my farms 1 , 2, 3 and made them my 2 'factory' farms + quest farm .... so slowness on those 3 doesn't affect my playing because I only harvest trees and flowers on those 3 ... but the slowness on my other farms HAS to be because they are expanded ..
I don't 'stack' crops , flowers, and trees or buildings etc on top of each other .. I do stack layer my crop areas but that's never affected the game before ..
I have 8 farms for growing crops , the other 6 have just buildings trees and flowers plus decorations ...

and farm 15 will just be made into some kind of town to put out more of my decorations ..

I just LOVE this game !!