mTCP updates coming soon: Send me your bug reports

[mbbrutman]

Try using "XFERMODE PORT" before your next file transfer.

Passive mode is preferred and it really shouldn't make a difference. But for some reason the mTCP FTP client is having trouble talking to ftp.oldskool.org when using passive mode. I tried it on my PCjr and it stumbled; the trace shows lost packets. But with port mode is it fine and fast, which leads me to believe some gateway or firewall in between is screwing things up.

In Passive mode, the FTP client makes the data connection to the server. In Port mode the server makes the data connection to the client. Once the connection is established there should be no difference.

[mbbrutman]

So I don't have the problem debugged yet, but I don't think it is a problem with the FTP client.

To recap:

• Getting a file from Trixter's site (ftp.oldskool.org) doesn't work; it starts and then stalls, never to complete.
• I recreated this on both my PCjr and on my 386-40.
• The default transfer mode is "passive", which is what I assume you were using.
• "Port" mode works for me ... which indicates some firewall/NAT device is screwing things up.

To verify the last point I disconnected my entire house from the network and put my 386-40 directly on my 100Mb/sec fiber connection. That should have caused a black hole, but amazingly the DHCP client spoke to the FIOS box just fine. And when I connected to ftp.oldskool.org again, everything worked including both PORT and PASSIVE modes.

My network is a bit screwy - I have "double NAT" for reasons I don't want to get into. (Thanks Frontier!) I suspect that you also have NAT somewhere in the mix, and that this is a firewall issue, not an mTCP FTP client problem.

HTGet works because there is only one connection involved - it sends the request to the server and the server replies on the same socket connection. FTP is much more terrible, insisting on a separate and new connection for every data transfer, including listings. In theory if you listed a directory with enough files that would also stall out and fail.

The program traces are not showing any packets being discarded on the PC itself. The packet loss is coming from where between the two machines. It's as though something in between lost state and decided the socket connection was invalid, and then stopped passing/rewriting packets.

[Eudimorphodon]

The thing that makes me think it's not a local NAT/firewall problem (or at least solely) is the client seems to work fine on ftp.zimmers.net in passive mode on this computer, and passive mode is working on this server from other machines on my network. I can try a few more servers, I guess...

So just now I'm trying again to hit oldskool.org, this time after issuing an "XFERMODE PORT", and it still seems to be stalling out to a dead stop on files larger than about 64k, so switching transfer modes doesn't work for me. ("Classic" doesn't seem to work at all, it just hangs on "Listening on port 1337 for incoming data" and eventually times out with a 425. I'm unfamiliar with the distinction between "classic" and "port" mode? I'm pretty rusty so I only remember "active" and "passive", and usually it's passive that's easier on firewalls.) The old Wattcom FTP doesn't seem to support changing xfer mode (I assume it only supports active), so I can't try fiddling that, but my Linux laptop here on the same network seems to have no problem downloading in either active or classic mode from its CLI client (and downloads via web browsers work, which are also passive), so it really doesn't seem like it's a problem with my firewall and passive mode per-se. It's not easy for me to put the Tandy outside the NAT to see if that clears it, unfortunately, but... it does seem like it's working for everything else trying to do the same thing?

When I had this running EPPPD and did some captures with wireshark what it seemed like was happening trying to access this site was an initial rush of the transfer would make it through, but around the time the buffer filled up there'd be a long pause in the Tandy sending ACKs, enough so that the server would try re-sending a chunk... which the machine might or might not ack, until it fell into this sort of death spiral where I could actually see the remote machine get bored enough to hang up the control connection while the ftp client was still acting as if it was receiving bytes. At the time I assumed it was some kind of processing delay in EPPPD to blame, but I'm thinking it might be a worthwhile exercise to configure the laptop as a bridge so I can tcpdump it again and see if the pattern is the same. (It looks like it is, because as I was typing this I'd left another download attempt sitting there, which got to 94k out of 350k in 1125.355 seconds, and the control connection is indeed dead after ctrl-C-ing out.) I'm kind of wondering if there might be some very aggressive TCP tuning going on at the far end, like it's sending too many data packets without waiting for acks for a slow client to keep up with reliably or... something? TCP slow-start should handle this sort of thing correctly, but if they've gone ape trying to optimize for a fat pipe...?

[mbbrutman]

"Classic" is obsolete - nothing uses it unless you find a very vintage FTP server. (The FTP server in NCSA Telnet requires it.) Also, forget the firewall thing, except where it might be changing packet timing - it might be a red herring.

The general problem is that mTCP is seeing dropped packets. Each TCP packet has a sequence number; a lost packet is detected when a new packet arrives that doesn't have the correct expected sequence number. mTCP is detecting these gaps and trying to prod the sending side into re-transmitting by sending an empty packet with the expected sequence number. After a few seconds the sending side should see that the newest packet is not being acknowledged and automatically re-transmit starting from the packet after the last acknowledged packet. This is not happening here.

Try this:

SET DEBUGGING=1
SET LOGFILE=TCP.LOG

Run FTP again; you'll get warning messages in the TCP.LOG file.

At the end of the transfer look at the TCP.LOG file. The warning messages should be along the lines of "W Sending empty ACK packet" .. that's mTCP trying to get the host to retrain. At the end you'll see some statistics; TCP will show the number of times it lost sequence while the Packet line will show the number of locally dropped packets. I'm assuming that you are not dropping packets because things are full on the machine, yet TCP will show the sequence errors. Those packets are not making it through and the other side is not re-syncing.

This part of debugging sucks.

Really, HTGET should have the same problem - it's just another data connection. But that could be a timing difference there too. And god only knows if some ISP/network is traffic shaping FTP vs. HTTP traffic.

[Moondog]

Strange coincidence, I installed an NE2000 isa card in my Compaq Portable II, and was trying to figure out how to get TCP/IP going. i just discovered mTCP this morning, but haven't unzipped it yet. I'm running currently running DR-DOS 7 with Novell driver support and IPX, so I have two possible paths to explore.

Just wanted to say thanks for working on such a cool project. DOS support has been drying up over the years, and manufacturer's software repositories don't carry all what they used to.

[Eudimorphodon]

This part of debugging sucks.

Totally. If I get a chance to set up the bridging so I can capture the whole conversation externally I'll see what I can see, but if it's behaving like my vague recollection of what was happening with EPPPD it definitely seemed like there was some systematic breakdown of the whole tcp sequence handling. Since I'm not seeing this on other sites (yet) I wonder if there's something at their end that's breaking this, perhaps only for sufficiently slow clients. The WATTCP ftp client that does work tells me "PASV" is unsupported when I try it but seems to be working perfectly fine with its active mode transfers, maybe it's a tuning issue with the ProFTP server.

[Trixter]

I routinely hit ftp.oldskool.org using mTCP's FTP client and I haven't noticed anything recently, but if someone finds a problem limited to ftp.oldskool.org, let me know and I'll do my best to fix it.

[Eudimorphodon]

I routinely hit ftp.oldskool.org using mTCP's FTP client and I haven't noticed anything recently, but if someone finds a problem limited to ftp.oldskool.org, let me know and I'll do my best to fix it.

I guess it's on my to-do list now to come up with a list of public FTP servers and hit them all to see if I can come up with a reasonable basis to say it's just this one site. ftp.zimmers.net was seemingly working great but that's a control sample of one.

[BloodyCactus]

Ive ran into these issues, I think I wrote about it on vogons. The bigger the file, the more likely to lock up. I reduced it by switching off full duplex. I want to say its a race condition but I dont know for sure. Sometimes I could pull a 50mb zip with no issues but it was like a 1/10 chance. othertimes a 200kb file would hang. once I switched off full duplex my hang rate was reduced to near zero. I dont know if it was a hardware issue on my card, tcp flow control or a buffer race condition.

[dieymir]

You can try to reduce the MTU size to a "safe" value (i.e. 576 bytes) and gradually increment it till you find the "sweet spot".