That depends on what kind of glue you used. If you used white glue, it is water soluble and you just need to soak it in water for a few minutes. I've been taking up some of my N scale track and ballast and I soak the ballast, then come back with a small chisel to work the ballast loose. I also use a putty knife to scrape up the large chunks. One word though, it is messy and you wind up with ballast stuck all over the place so I also have a wire brush, several other stiff brushes and a hand-held vacuum cleaner.