The number of listeners should not be an issue. One Pega customer has 2000 and has seen no performance issues. The listeners run as threads and obtain locks on the mailbox. Therefore, it is probably better if they run on a single node - there is very limited benefit to multiple nodes. Node failure is one draw back to this, but as email listeners are more "polling" than "listening", node restart will perform catchup. Hope this helps.
I don't think the listener runs a thread. Each listener will have a corresponding requestor , so if a system 2000 listeners it means 2000 requestors. As the no. of requestors increase there will be performance issue until and unless we handle.