Do I have to let him see my daughter?

My ex and I are obviously no longer together but he wont leave me alone he is constantly calling my house and cell phone I never answer either when he calls. He left a vm the other night saying that if I didnt call him back he was going to kill himself(making me even more concerned that he is not mentally stable enough to care for a new born child) not only that but the house he lives in with his father is not fit for an adult to live there much less a newborn child to visit(rats,mold...etc) His name will not be on the birth certificate therefore do I have to let him see her at all? I said I would never be the kind of mother to do this but the conditions are just to serious.