The agreement between China and Britain to guarantee a “high degree of autonomy” for Hong Kong for 50 years following its handover to Chinese control a quarter-century ago has been “breached comprehensively” in recent years by Beijing, according to Chris Patten, Hong Kong’s last British governor.

The principle of “one country, two systems” in the Joint Declaration on the question of Hong Kong, which was filed at the United Nations in 1985, was supposed to guarantee a “high degree of autonomy” for the city, protect its way of life as well as its freedoms and rule of law, Patten recalled. The handover went into effect on July 1, 1997, and the guarantee was to last to 2047.

“It’s fair to say for 10 years after 1997, maybe a bit longer, things did not go too badly wrong, but they’ve gone downhill since, partly because [Chinese President] Xi Jinping and his colleagues are terrified of what Hong Kong actually stands for,” said Patten, who in a new book is quite critical of a controversial national security law enacted for the city two years ago.

“I think the things that Hong Kong stands for – the freedoms and the characteristics of the Hong Kong sense of citizenship – are important to all of us,” he added.

Patten was speaking at a news conference in London promoting “The Hong Kong Diaries”, which summarises journals he kept in the five years he served as Hong Kong’s governor before the handover to China in July 1997 and analyses developments in subsequent years.

Now chancellor of the University of Oxford, Patten said that one of the most difficult conversations he has had is with Hong Kong students studying abroad about returning to the city.

“I do believe that Hong Kong is a great city. I hope it will be a great city again. If you have to lock up a 90-year-old cardinal, or threaten to, because they raise money to help pay people’s legal costs, what hope is there in a system like that?” Patten said. “This is the system going to rule the world? I find the whole thing distressing.”

The migration of talent from Hong Kong to places like Britain in recent years is a “huge loss” for the city and comes at a challenging time for China’s economy, which is facing issues around rising debt and an ageing population, Patten said.

“I think [China’s] going to rue the day it thought it could depend on a sophisticated international trading hub in Hong Kong without the rule of law,” he said.

Victoria Harbour from the waterfront in Tsim Sha Tsui.

However, Patten noted, it would be hard for Britain to make arguments about Chinese ignoring the established rule of law in Hong Kong as Prime Minister Boris Johnson’s government prepares to rip up the Northern Ireland protocol established after Britain’s exit from the European Union.

“I don’t think they’re quite in the same category, but it is very, very difficult for the British government to take a stand on rule-of-law issues when it breaks international agreements at will itself,” he said.

With some bemusement, Patten also weighed into the latest controversy over new textbooks in the city stating that Hong Kong was not a British colony, as the Chinese government did not recognise what it said were unequal treaties that ceded the city to Britain.

“Rather than calling it a colony, the Chinese Communist Party is calling it an occupied territory. It’s a bit of an own goal. You have to ask yourself who occupied it,” Patten said.

“Overwhelmingly, Hong Kong has been occupied by refugees or the descendants of refugees from the brutalities of communism on the mainland.”

“The occupation was by refugees who found a safe haven in – wait for it – a British colony, which they turned into one of the most successful cities in the world, an open society which brought together economic and political freedom to an extraordinary successful degree,” he added.