Some services restrict to only allow keys from certain vendors. For example, Vanguard.com supports only Yubikey, all other security keys are not permitted. For Azure AD, a handful of vendors are approved however any vendor key can be used by disabling ‘Enforce Attestation’ more info on that here - Error adding Yubikey to Security Info - Microsoft Q&A
There may be a similar setting in the server you are using. Another thing to try would be to use a different browser such as Firefox to register the security key. Let me know if that works, if it does there is a possibly related issue here - Cannot register to Microsoft Account - #13 by t11