I have a slightly alternate take on AS, but with similar conclusions. My 2p worth :
It might be normal to hear very little difference with AS settings. It depends on how much stylus/groove friction is going on (in turn on alignment and quality/type of diamond polish and cut, and wear). It also depends on cartridge suspension stiffness and damping, as it is the suspension that must accomodate (the headshell that moves) in producing the observed tracing distortion.
In a silent groove with no anti-skate, tonearm drag is always on the inner groove wall. In fact, unless the groove gets very curvy (as in high levels esp at inner grooves and with high frequencies), tonearm drag always pulls on this wall. The line of force for the tonearm drag is the tracking offset angle, typically c 27 deg. Until the groove angle exceeds this, the pull (and associated friction) is always on the same wall.
Optimal AS arranges for the force on both groove walls to be equal in a silent groove.
As to what one can hear as AS is altered and becomes optimal, as I see it, is a) an improvement in peak level trackability, esp on pure right side b) a change in the type of ever present distortion from 2nd harmonic to 3rd harmonic c) a shift in the location of harmonic distortion from right to centre.
It's possible that none of these occurs at a significant level in normal play, or is masked by a larger effect. And that's why the effect can vary between setups, IMO.
Optimal AS varies between records, and to some extent across the disc, in principle. So I wouldn't get too hung about achieving perfection, it probably won't last long, and as a rule normal variation is very tolerable.