Any company can install a reporting agent on their desktops and that will collect basic information like what .exe files have been executed.

Once a portable app shows up in the reports you are simply fired by blatant disregard for the rules and procedures you agreed to when signing your contract.

Finance is a heavily regulated environment and you can't get away with things that would be excusable in other places.