Working with Glass Block. Waited to long to clean up mortor. Tips for cleaning?

I put a small 16″X32″ glass block window in my bathroom. This is the first time I’ve worked with glass block and I fear that I waited to long for my final mortar clean up. It’s hard as a rock now and I have quite a bit on the glass face. The sponge is just being shredded by the mortar. Is there anything I can do to clean up the mortar and get it just between the grout lines? Or will I have to rip the glass out and start over and chalk this one up to learning?

Thanks DIYers!