Hiya Sandy,
That is unusual, - does this happen even when you specify '_self' or '_parent' in the dialogue box? (See image attached).
When you look at the editor instance in 'Source' view (if you have that as part of your menu group) does it show your selection correctly - i.e. '<p><a href="#testanchor" target="_self">testlink</a></p>'?