How do you fight paternity?

okay i have a 3yr. old that will be 4 in april. anyways me and the dad were off and on for 10 yrs. finally im 18 and pregnant, he's addicted to drugs and broke up with me for another girl when i was 6 mos. prego. i was devistated. i got over it and met my now husband when my son was 3 wks. old(the dad wasnt there 4 the birth)we started dating when my son was about a month and a half old. he still wasnt there. my husband tought this baby everything! my husband fell in love with my baby before in love with me!he calls my hubby daddy, i got scared at first cause i dont want my little boy being hurt by him too. well the "dad" went to prison for 3 yrs. in that time he actually contacted me and wanted to get in my son's life. my son met his real dad for the first time last august. my hubby was hurt. now the"dad" is out and was around barely,went back in. out now and is demanding if he doesnt see him that he's taking me to court!? i have sole custody.he wants joint.dont know what to do.help?