If you use Weechat for IRC, there is a Matrix bridge plugin that would allow you to connect to Matrix from the command line.