If you tapped someone's fiber line, all the traffic between the ISP headend and customer premises (OLT and ONT if we're talking fiber) will be encrypted. In fact on a PON network using TDM it has to be, because if you stared down your own fiber you would be seeing all your neighbors' traffic as you're all time-sharing the same laser diode at the ISP headend (because it's a passive network, you will be seeing your neighbor's traffic when the diode is transmitting outside of your designated time cycle).
Anyway my only point was to inform OP that contrary to their belief, they effectively are in the situation of having an ISP-owned modem.
There's also an additional VLAN 301 for TR069 management traffic, which is used by the HG612 modems (and possibly others) that Openreach used to enjoy flinging at all VDSL subscribers. The modem itself claims an IP address in this VLAN.
Although usually hidden from the end-user, it's actually surprisingly easy to drop yourself onto VLAN 301 even with the HG612 and get an IP address on that management network. I imagine that this is the kind of way that modem exploits become dangerous if they are indeed routable on networks like this.