There's no right or wrong answer to your question. Doctors don't know what causes endometriosis, and they don't know for sure how estrogen affects endometriosis. It's an opinion question.
Some doctors recommend no hormone replacement therapy for women with endo, after a hysterectomy. Others recommend waiting for a set period of time before starting HRT. And other doctors start women with endo on HRT immediately after a hyst. Some doctors recommend keeping one or both ovaries. Others recommend removing the ovaries.
Some women who keep one or both ovaries do very well. Others experience ovarian pain, and choose to have their ovary/ovaries removed later, as a second surgery.
What's a woman to do?
Your doctor knows you best, and I'm not a doctor. If you question your doctor's advise, please consider getting a second medical opinion. It's always a good idea to get at least two medical opinions before having surgery, anyway.
I chose to have my ovaries removed. I was 47 when I had my hysterectomy. I've done very well on HRT, and I'm happy with my choice. I had endometriosis, and the estrogen I'm taking doesn't seem to have caused any problems. But that's just me.
It's a tough decision, and you're wise to get all of the information you can, before making your choice. Good luck!