Voltmeter
Your voltmeter is a very slow reading instrument; it takes a sample, commonly 1/4 to 1/2 a second long, averages the voltage values or takes a RMS (Root-Mean-Square) value of the signal, and then shows only the average or RMS value. It does not tell you the wave-shape or the frequency of the signal.
There's a slim possibility that what you're looking at is an AM radio station signal (I've dealt with that a number of times). As a greater probably, it's the AC data portion of the signal from the Signaling Line Circuit (SLC). Your AC voltmeter, however, won't tell you the difference.
You mention that when the cables were separated, the AC voltage dropped to 0.8 volts. If the voltage value keeps varying, it's probably from the SLC; if it's a steady 0.8 volts, there is a good possibility that utility power, 50 Hz or 60 Hz, is also being received by the circuits. Again, your AC voltmeter won't tell you the difference.
Coupling
Any wire is a transmitting antenna, and any wire is a receiving antenna. If an AC signal is carried by a wire, that wire will produce a magnetic field, and the AC will vary the strength of the magnetic field. All other wires will receive this varying magnetic field, and generate an AC signal.
The strength of the AC signal depends on how well the wires are coupled together.
Transformers
Transformers work using magnetism. There are two wires, next to each other. One wire (we'll call it the transmitting wire) sends out a varying magnetic field, and the magnetic field uses the other wire (we'll call it the receiving wire) to generate power.
To increase the strength of the magnetic field, the transmitting wire is coiled. To increase the sensitivity of the receiving wire, it, too, is coiled. In other words, because the wires are coiled, they are better coupled.
Increasing the coupling even more, a magnetic core surrounds both coils.
The shape of the magnetic core isn't as important as the fact it's there. As far as coupling wires together magnetically goes, if both wires are inside the same conduit, the conduit is a magnetic core that surrounds the wires. That is a long, thin, transformer.
In other words, to some extent, metal conduit will increase the coupling of an AC signal carried on one wire to the rest of the wires inside the same conduit.
Coupling the SLC to the NAC
The SLC wires are sending out a varying magnetic field, and the varying magnetic field is generating AC voltage into the NAC wires. What you did when you separated the cables, was to reduce the signal-coupling between the conductors.
Compared to a transformer, though, the SLC wires and the NAC wires are not very well coupled together, so the actual power transfer is low.
Voltmeter, End of Line Resistor, and Panel
Your voltmeter is a very high impedance device, the voltmeter won't really affect the measured AC voltage.
The end of line resistor that's across the NAC wires, however, is a much lower impedance device; it soaks up a lot of the power that is coupled from the SLC to the NAC. It pulls down the AC voltage that you're measuring, so the voltage across the NAC wires is lowered.
The NAC Booster Power Supply, if it's connected to the building's NAC circuit, is also going to reduce the voltage.
Does the Coupled Voltage Matter?
If what you have is horns and strobes, unless there is a great amount of signal, occupants of the building will never hear the problem, and the signal also won't harm the horns and strobes.
When the horns and strobes are not in alarm, a blocking diode inside the horns and strobes effectively disconnect them from the circuit. When the alarm sounds, the NAC Booster Power Supply's battery and power supply capacitors effectively pull down the coupled signal voltage on the NAC to zero.
Because any wire is a receiving antenna, there will always be some very low level of voltage on the wires from something. Especially if the circuit isn't connected to the panel. Only on extremely rare occasions will the signal picked up by the NAC circuit cause a problem with a NAC Booster Power Supply.
OK, I've seen this as a problem one time in 20 years. But like I said, this is rare, and something like this isn't harmful the power supply.
Big Question
Did the system work before? If the fire alarm system has been in use a long time, has been tested regularly, and besides having those ground faults, the system has been working, I wouldn't be concerned with that low of an AC voltage.
No matter what you do, you'll always be picking up a little signal on the wires from something because all wires are receiving antennas. The only time to be concerned is when the system is not working, or when the system has intermittent troubles. Then you have to find the source of the troubles and fix it.
Douglas Krantz