When you look into the requestors list running on your system (System --> Operations --> Requestor Management), only the ones starting with an "H" will be real agents working, so I think you can test it like that but we may have other options as well.
Would the information that no agents are logged be available from the Chat API? As we are trying to hide the Chat button from the customer site it would be useful to find a way to get this info straight from that page.
After some investigation I've found out that the best way is to use the REST API : "/api/chat/waitTime/company/COMPANY_TOKEN/queue/QUEUE_NAME". If the method "waitTime" returns a negative value then there are no agents available. Having this parameter received on the customer web site allows us to customize it and hide the chat button when desired.