Is it emotional abuse?

Was it a joke? If not...that is sucky and probably needs to be addressed. However, no one can undermine your self-esteem/confidence unless you let them.