If the government believes that he still has valuable intelligence potentially damaging to national security that may not have already been released, and if the government's primary concern with his actions is addressing the national security impact (which all public statements point to), the primary goal of the government should be preventing foreign powers, potentially hostile ones, from getting access to the secrets he possesses.
Publicly threatening to arrest and try him for serious and potentially capital offenses like espionage while he is in territory not subject to the US or any close ally that would be likely inclined to cooperate given the circumstances is, from that perspective, quite foolish.