How do I replace the glass in a wooden double hung window?

Hey guys and gals.. I have wooden double hung windows in my home. The lower window is one peice of glass, single pane, approximately 2.5 feet square. The windows “frame” that slides up and down is rail and stile construction (if that matters..). The glass pane is cracked, and I need to replace it. I can’t tell if glazing putty holds the window in from the outside, or if the window sits in a routed slot in the wooden frame. If it is putty it’s so hard it feels like wood, and has been painted a million times.. Big trouble is this.. It’s still wicked cold out, and exploratory surgery may result in the glass completely cracking, making my home somewhat cool… I need to determine a plan of attack… Is there a way to take apart a window, are all glass panes “glued” in place with putty? Help!