In terms of what kind of work you do for them, it's up to you to sell whatever you're comfortable with so long as every expectation is explicitly documented in the contract/SOW. Companies can hire technology services to do almost anything up and down the value chain. From strategy and user experience design, marketing, content production, system architecture, development to operations and maintenance. The bottom tier is something like staff augmentation where you just rent out warm bodies to do IT work. The agency model is more like landing a complete self-organizing, product development team who take business objectives and translate them into a working product.
Stuff like operations and maintenance is stuff I would typically avoid because it can be open-ended and difficult to make money doing. I usually stipulate that clients have to own infrastructure even if we do the setup and add some specific warranty language on how long we will answer bug reports, what qualifies as a bug and what kind of turnaround time we'd need to resolve them. Especially at small scale, clients will search for any excuse to drag out their contract to get more work out of you or find a reason to not pay so you need enforceable language.
Doing good work and building trust can lead to future work. You need to toe a line between aggressively managing scope to not get taken advantage of while still staying flexible enough to be seen as a good partner. It's really common for a client to have a big vision and no idea what it costs, so we sell an MVP that does maybe 1/4 of their dreams to get a product to market within their budget, but there's still a lot of meat on the bone for future work. Sometimes it leads to a long relationship and sometimes they just fold.
It's a risky business, but there's also a huge market that only seems to be expanding. If you make a name for yourself and build up a few nice portfolio pieces, clients will come to you after not too long.