If you're labelling things in medium volume, like items in data centre racks, then changing the paper in the office printer is a hassle, and making sure no-one else prints on your labels by mistake is a hassle, and getting the text to align with the labels can be a hassle too. None of those is impossible, but hand-writing a label will be a lot easier. If you want to have professional-looking printed labels, a label printer is the right tool for the job.
If you're running a higher throughput operation like ebay order fulfilment, printing labels one at a time can simplify your processes - a worker who only has one order and one label can't get things crossed over. At the same time, serious label printers can offer thermal printing (no toner or ink to replace), huge reels of labels, automatic removal of backing material, and so on.
And if you are a nano-volume user like me (I label external HDDS/SSDs mostly), you can do well with a low-tech solution i.e. a fully mechanical label printer [0]. I use it twice a month and for that frequency it's absolutely perfect.
[0] like https://www.amazon.com/DYMO-Organizer-Xpress-Label-12966/dp/...
It cost around €20, three meters of off-brand sticker tape is less than €5, and it takes AA batteries which then last forever.
Very useful to have somewhere in a drawer.
You can get A4 (or US letter I assume) or whatever sheets of labels for a 'regular' printer yes, but that's honestly not as convenient as a roll, especially a continuous roll where it prints to whatever length and then chops off. Especially if your use is occasional, don't want to print a whole sheet at once - and even if you do, I don't know how good the tooling is for laying out an A4 sheet of pre-cut labels with a different address in each one?
I have a Brother QL which others have remarked in this thread as well. It has WiFi and everything, trivial to just create a label from your phone or computer.