I'm not aware of any documentation about what agents will do in case of errors. I think it depends on the error and if the activity being run has been created in a way so that it can complete it's processing if that type of error is encountered.
We did our testing and unfortunately this doesn't work as mentioned above.
As per screenshot below, the agent has not run for over three hours. It is scheduled to run every 60 seconds, but it is not even trying to execute the activity again. After a JVM restart there is a recovery, but that is useless, easier just to start the agent in the developer portal or from the SMA.
We have 3 agents configured in our environment for a AgentTester Queue rule. Once the first agent has thrown an exception the other two are never executed. They too, are scheduled to run every 60 seconds, but once an agent from the agent queue fails the entire queue appears to stop processing.