View Full Version : SCO Xenix 286 in Virtualbox

April 13th, 2012, 01:30 AM
I'm playing around with SCO Xenix within Virtualbox at the moment and have hit a bit of a problem with serial ports.

Other than floppies, really the only way to get content in and out of the vm is via the emulated serial ports - so I'm using the well documented technique of used named pipes within Virtualbox as the virtual serial ports, so that serial/terminal emulation tools within the host can talk to the guest OS as if they were attached via serial/null-modem cable.

However, Xenix appears to be *really* flaky when sending or receiving data via this method.

I have managed to get a version of c-Kermit onto the Xenix vm, and have run "mkdev serial" to build the serial port devices (/dev/tty1a and /dev/tty2a).
Setting c-kermit to 9600 in both the Xenix guest and the host (Linux), but whenever I try to send or get from *either* side, the Xenix machine spits out:

"Garbage or loose cable on serial dev 0, port shut down"

I've also done a 'enable /dev/tty1a' to allow login terminals via the first serial port, and the login prompt shows when I access the serial port from Linux, but after entering username and password I get a hung connection after a short number of commands (eg a single 'ls' is enough to trigger it) and the 'Garbage or loose cable...' message on the Xenix console.

So, the emulated serial connection works - as evidenced by the login and password prompt appearing at the 'other end', ie in Linux, but sending or receiving more than a few bytes of data appears to cause Xenix to flake out.

I did find very old postings about a fix for that 'Garbage or loose cable....' error, but it was only applicable to versions of Xenix prior to 2.3.2 (and not 2.3.2 itself, which is what I am using), I even tried installing the patch anyway, but it errors out with a 'not for this version' message.

Anyone got a working Xenix 286 system in a virtual machine and can try the serial ports?

April 13th, 2012, 02:48 PM
I haven't used emulated serial ports in VirtualBox, so I can't comment on that part.
But how are you setting up your handshaking in Xenix? In general, for any os of that vintage, you need either software (xon/xoff) or hardware handshaking (cts/rts) to be working to get a reliable connection to another machine.
I don't even know if the emulated serial ports in VirtualBox have such capabilities.

April 13th, 2012, 04:51 PM
When using kermit I set the Linux host to match the defaults as used by the Xenix kermit client - 9600bps, xon/xoff, no parity. With the terminal over serial, there's nothing to configure; I just hook the 'screen' program into the pseudo tty offered by the pipe created by the emulated serial ports in Virtualbox and it displays the Xenix logon banner and I'm able to login, but anything more than one or two commands and it blows up. Both methods result in the main Xenix console inside Virtualbox displaying the 'Garbage or loose cable...' message.

There is a SLS patch (xnx128) that fits that error message, but it is only for Xenix up to 2.3.1 - I've extracted the contents and it's a replacement for /usr/sys/io/libio.a - but the file as included is dated April 1989, whereas my libio from 2.3.2/286 is dated July and is almost twice the size, so I didn't want to try forcefully replacing it (the install script seems to relink the kernel too - that's not something I want to try yet!).

April 13th, 2012, 05:08 PM
Ok, this is definitely something specific to SCO Xenix 2.3.2 286.

I've just tried the exact same thing with 2.3.4 386:

Base install of 2.3.4
Enable serial ports in Virtual box (named pipe: /tmp/comm1 & comm2)
Start up the Xenix vm
In Xenix
mkdev serial
In Xenix
enable /dev/tty1a
Reboot Xenix
On the Linux host run
socat UNIX-CONNECT:/tmp/comm1 PTY,link=/tmp/comm1tty
On the Linux host run
screen /tmp/comm1tty

The result is that the Xenix 386 login appears in the Linux terminal, you can login fine and use the Xenix system as normal, no 'Garbage or loose cable' messages are generated and the Xenix system continues to work as if there is a proper remote serial terminal directly connected to its serial port.

April 16th, 2012, 03:00 PM
Interesting. Well, now we know that emulated serial ports works (can work) in VirtualBox.
Thanks for the update.