It might take time for Hong Kong's present jobless rate to go down despite more people going out and an improvement in business at shops over the weekend after social-distancing measures were relaxed on Thursday, Financial Secretary Paul Chan Mo-po said.