This can be achieved using JQuery. You can add the following js in the perform harness and replace 'Rich Text Editor' with label you have to the Rich Text Editor and Replace title with the tooltip text you want
$("label:contains('Rich Text Editor')").append('<i class="icons pi pi-help pi-link helper-icon" title="This is a text editor"></i>');
To keep this post updated. There is a OOTB approach as well
Use the Rich Text Editor without label and add a dynamic layout on top of it with Layout Format as Inline and add a label and icon (icon class as source with pi pi-help). To the icon add actions on hover display tooltip. This inline dynamic layout acts as a label to that RTE. This is OOTB approach.
If you want to use the label that comes with RTE itself, you can use the above approach but that is customization and not encouraged.