What happens if I drink red wine with fish and white wine with steak?

It’s common sense that red wine goes well with red meat and white wine goes well with fish.

But what happens if I eat fish with red wine or eat steak with white wine?
Is it bad for health?
Or is it just because such combinations aren’t as good as the recommended combinations in terms of taste?