27F accused of babytrapping my 28M fiance when my tubes are tied?

I had an argument with my fiance this morning. We've been dating for 2 years, engaged since September, and for the most part, everything has been going well. We've been planning a quiet backyard ceremony so that we can save up for a house instead, we've been communicative and managed to get through fights in the past, but this takes the cake.
He's been evasive for the past two weeks about the wedding or any future plans we've made, and I basically had to corner him this morning before leaving for work to ask him what's going on.
Turns out, while he was dogsitting for his uncle early in February, they had a chat that stuck with him. When they were talking about life and how things have been, his uncle admitted he resented his ex-wife for babytrapping him, and now he's divorced while his ex-wife is dating again and my fiance's cousin is an entitled asshat who terrorized him when they were both teenagers. Turns out it's been sitting in his mind.
He says that he thinks I'm about to spring a pregnancy announcement on him and trap him in the marriage. This is despite the fact that he knows that I don't want kids. I basically raised my siblings and lost out on my childhood. I told him about not wanting kids when we first started dating. We were both on the same page and I've asked him about getting a vasectomy in the past, which is why it surprising that he thinks that I'm trying to babytrap him.
Thing is, the first chance I got (which still took a long time,) I got my tubes tied. I literally can't get pregnant. I reminded him of this fact, and that made him go really quiet. He didn't even apologize or say anything, so I told him that if he's going to be like this over a made up issue in his head, I don't know how much I'd trust him in a real crisis.
Now I'm wondering if I was too harsh and what steps we can take to move forward. Or if I'm the right amount of angry and I should just end it. I have no idea what to do right now???