If the manifold was glass beaded or sand blasted without removing and cleaning then you should do it as soon as possible, like tomorrow. If it wasn't glass beaded or sand blasted then don't worry about it.
That wasn't done by me. Not sure what the previous owner did. I stripped the paint off, wire wheeled and sanded. Used a VHT paint. I had the manifold upside down, inverted, angled several times and didn't see anything fall out from the pan. Maybe I got lucky for a change, Carmine.