The "serial port" setting, not the "tcp port" setting.

The telnet protocol is not just raw data. The 0xFF byte is an escape code for commands (and a 0xFF needs to be repeated twice pass an actual 0xFF). There are a few commands including those to send a rs232 break signal or to negotiate options such as serial port baud rate (serial ports don't negotiate a speed, so you have to be able to configure the right speed on both ends of the communication)