You'll probably need to use 75 ohm coax for the R/G/B signals. A VGA cable is a good choice, because it contains three 75 ohm mini-coax cables inside the outer shield.
The shield of each coax line needs to be connected to its corresponding ground connection at both ends. Red to red ground, etc. Don't connect the different colour grounds together. Pinout is here: http://old.pinouts.ru/Video/atari_falco ... nout.shtml
The other ground connections should be run on separate wires, not joined with the red/green/blue grounds. The coax lines need to go as close as possible to the plugs at both ends. For every cm of cable you have that isn't coax, the picture quality gets worse. Just wrapping with aluminium foil won't help at all, because it's the characteristic impedance that matters - not the shielding.
According to that site, there is an error in the Falcon's owner manual about the pinout. I would imagine that site has the accurate pinout.