Oh, phono pre-amps do differs quite a lot, just like any amplifier - different internal designs.
I used to have a NAD PP1 with I though was good, but when I replaced it with the twice as expensive Thorens MM01 - oh boy, what a difference! Vinyl has never sounded so good before in my house. And note, this was the entry level budget offering from Thorens... They do also offers much more expensive phono pre-amps.
Pro-Ject also has nice ones, and they even offers tubes in some of their models, yes not transistors but old style tube technology. This of course comes at a price... Vincent is another maker, also has tube phono pre-amps. Rega is another brand famous for their phono pre-amps. Musical Fidelity has good but quite expensive ones. Cambride Audio has nice ones too and so on and so on... Plenty of options out there!
But the basic Behringer is good, it does the job. Not fantastic, but ok. It depends on what demands you have.
Regarding output volume, phono pre-amps do - in most cases - have lower output level than a typical CD player. Many LPs also have less compression than typical CD's, which means that they have a wider dynamic range - (dynamic range = the difference in volume between the quietest and the loudest passages in the music, the more dynamic range, the more difference in the music between the low volume and loud passages).
No noise? Oh, cartridges and the vinyl records can produce noise. It is seldom the phono pre-amp, not if you invest in one of the better ones. If it is an extended humming or brrrzzz in the system, then the ground wire is not properly attached, so it is also a matter of setting up. It is not really like plug'n and go as CD players are
One thing to decide - MM, MC or both? This depends on the type of cartridge you intend to use of course. If you have a MM cartridge and does not have intention in the near future to get a low output MC cartridge, then you don't need to pay for the MC function in a phono pre-amp.