They're simple vectors, the backgrounds are based on scans of the inlays. The background vectors from Mario Bros are by ChoccyHobNob. I can't help you with Photoshop or AI, I use Inkscape and am not an expert.
As for the overlay, that sounds like a lot of work. I'd just set the screen alpha to a lower value instead. And maybe add an overlay rect on top to add a green/blueish tint. If you only change the white and leave the other colors alone, the contrast will look bad(the red and cyan as they are now will 'clash' with a grey bg).