As far as I know there is no queue mechanism in Email Listener data instance.
Ideally Email Listener listens on a particular folder(Let's say Inbox), whenever incoming email reaches the inbox, email listener starts processing the email and perform the operations mentioned in Email Service Activity. Once email gets processed it should go to read process and Email Listener won't be able to access again.
pyCreateAndManageWorkFromEmail OOB activity creates a case upon receiving the new email. Use this activity and test your scenario.
If your EmailListener is configured to reply and that reply fails for some reason, the email will not be marked seen and thus will get processed again. Turn debug on EmailListener class and read through the log file to see if that is happening.