Don't envy you at all. But it would be a lot worse if scenery or ballast was done.
I'd leave the tracks in place, cut the rails where the plywood pieces join together, and splice back together later with short sections of track. Without ballast the tracks are removable, but why remove them...