I do NOT like him being sick

Last night I came home from work and heard Dominick crying through the front door I walk in and Johns sister was watching him and she said he wasnt acting himself last night and he felt pretty warm. So I checked his temperature and he had a 103.9 fever! Well, I lost it! So I forgot where I keep the tylenol and was walking around repeating "what do i do" while all that is going on Dominick is crying "momma" and i still cant remember where the tylenol was. So she calmed me down and i found the tylenol and gave Dom his dose and text John to come home. He came home and Im crying, Dom is crying and Terri is trying to calm both of us down lol. I was ready to go to the hospital but John and Terri convinced me to just wait a few minutes to check his temperature to see if it was going down if it didnt than we would go for sure. Well within 15 minutes his fever went to 101 so i calmed down and just thought maybe its a bug and ill wait til the morning to call the doctor and see what his temp was then. This morning it was 102.6 which is still very high for the morning so i called the doctor and he told me just to keep and eye on his fever today and keep giving him tylenol but dont worry. So I have decided that im going to wait until later today to see how Dom starts acting and see how his fever goes and if it goes back up to what it was last night im going to the hospital for sure. If there is any suggestions im open to any im still very new with this and dont know when i should be panicking or just relax so i pretty much panic all of the time.