Jigsaw Box app comes with a large collection of Music in different styles including Classical, Relaxing and Orchestral. You can switch between the styles using the Music Setting menu. You can also skip to the next track, by using the "next" and "previous" buttons.
To turn off the music, you can set the Music Choice to iPad Audio. This will stop the music from playing. If you have a radio app or like to play Music from your own apps like the iPod app, then this music will continue to play while you play the puzzles.