The quick answer is yes, the amplifier, and not the speakers, would be part of the battery calculations.
The problem with audio devices is that, unless the measurements are based on calibrated audio sources, and laboratory equipment is being used, audio voltages and current are impossible to measure accurately. Rather than looking at the speaker's electrical current use, the best that can be done is to base the amplifier's maximum standby current and alarm current on the actual use, under full load and full sound condition.
If the alarm condition current that you are supposed to use is for a 50-watt amplifier, for instance, the speaker load on the amplifier is considered to be added up to 50 watts, and the sound level is a constant "as loud as it gets".
Rarely, if ever, are these speaker power loads and continuous sound voltage levels achieved, but they're the battery current levels we have to calculate with.
Keep in mind that many fire alarm system amplifiers will fail-by-turning-off if the speaker load exceeds the 50 watts by even 1.0 watt. When designing fire alarm systems, commonly there is a 20% headroom given to the speaker's Notification Appliance Circuit (NAC) so the speaker load would only be 40 watts (or 41 2/3 watts, depending on how the 20% headroom is calculated).
Douglas Krantz