That's correct. The main issue is number of messages per folder, performance wise. However, in my experience, overall mailbox size does negatively impact Outlook performance.
As far as the actual mailbox database itself though, it should handle it without problem. I have a user with over 250k messages in his mailbox.