All my hamster does is jump on the glass and try to escape and is not interested in me. Why?

I got a baby hamster 3 days ago, I think it’s about 3 inches in length. I’ve been told everything about how to handle them, but I don’t understand this one’s behavior becuase I’ve never had a hamster. He is in a 10 gallon tank, with a wheel, cardboard toiletpaper rolles, a house, toys, and food and a thick layer of bedding. All he does is either run and run and run on the wheel or jump on the glass trying to escape. If I even slowly lower my hand into the tank he scurries the other direction or buries himself. Someone told me to hold my hand out with a treat and wait for him to walk onto it- I waited 30 minutes, and nothing happened and I had to stop because it was cutting off the circulation in my hand!! If I try to pet him myself, he rolls on his back, freezes, and bares his teeth. I did not handle him the first 24 hours, I have no idea what I am doing wrong. He is a male and by himself in the cage