Sorry I have a wife

Is it wrong to say I'm sorry I have a wife or should it just be a straight NO!!