The agreement does not grant China full control to do whatever it likes over the territory at this time.
So legally, in response, Westerners in the form of the UK still have some kind of duty towards the people there.
A second reason, which I rank higher ethically, is that just by virtue of people existing, we still have a duty to demand better treatment for people who are being treated badly, especially when it's an imposed political event (and therefore can be changed) which is going to radically change their lives for the worse as they see it. Landmass connection or not.
The handover of Hong Kong from Britain to PRC in 1997 is based on the Sino-British Joint Declaration of 1984, which is being violated [1]. PRC is violating an international declaration it signed, which at least Britain has a say.
Putting aside the international nature of Hong Kong, other nations should take a part in some extreme cases, such as to push back on concentration camps and genocide, be it in Nazi Germany or modern day PRC.
[1]: https://www.reuters.com/article/us-hongkong-protests-britain...
P.S. As other comments said, Hong Kong (transferred from Qing to Britain in 1841, 1860, and 1898) has never been a former territory of PRC (formed in 1949). It is also arguable if Britain should give Hong Kong to ROC instead of PRC.
https://en.wikipedia.org/wiki/Territorial_integrity
But just imagine how crazy it would be if every place could declare itself independent. Would states count? What about cities?
There's also the tradition of cities being "free" as in they were not feudal land.
Of course self-determination doesn't mean complete total independence. Sometimes cities/regions/states lack the ability to control their currency supply (see the problem with in the Eurozone due to the differing interests of Germany and Greece), but inclusive institutions should maximize the ability of the member to work within the institutional framework to have its problem addressed. And that's why democracy is such an institution, and in theory other institutions built on democracy have this ability too. Of course it needs many things for it to work well. (For example empathy of the other members.)
They ruled China for 37 years. I'm sure they had diplomatic relations
Hong Kong should return to the Taiwanese government. The legitimate government of China
"Before the Nationalist government was ousted from the mainland, the Republic of China had diplomatic relations with 59 countries, such as Australia, Canada, Cuba, Czechoslovakia, Estonia, France, Germany, Guatemala, Honduras, Italy, Japan, Latvia, Lithuania, Norway, Panama, Siam, Soviet Union, Spain, the United Kingdom, the United States, and Vatican City. Most of these relations continued at least until the 1970s, and the Republic of China remained a member of the United Nations until 1971."
China struck HK 2 months ago with the new law at the weakest point in recent history with the pandemic.