hpn-ssh

Note there is no intrinsic reason running multiple streams should be faster than one [EDIT: "at this scale"]. It almost always indicates some bottleneck in the application or TCP tuning. (Though, very fast links can overwhelm slow hardware, and ISPs might do some traffic shaping too, but this doesn't apply to local links). SSH was never really meant to be a high performance data transfer tool, and it shows. For example, it has a hardcoded maximum receive buffer of 2MiB (separate from the TCP one), which drastically limits transfer speed over high BDP links (even a fast local link, like the 10gbps one the author has). The encryption can also be a bottleneck. hpn-ssh [1] aims to solve this issue but I'm not so sure about running an ssh fork on important systems. 1. https://github.com/rapier1/hpn-ssh

Two things that can have a big impact on SSH throughput are cipher choice and the hardcoded receive buffer size. These are described in the fork https://github.com/rapier1/hpn-ssh Maybe that will be useful for thinking about workarounds or maybe you can just use hpn-ssh.

Yeah, the longstanding hpn-ssh fork started off by adjusting ssh’s window sizes for long fat pipes. https://github.com/rapier1/hpn-ssh

I asked my "why not upstream" question here and got a response: https://github.com/rapier1/hpn-ssh/issues/89#issuecomment-22...

In almost all cases, the reason is OpenSSH's silly limitation of buffer sizes [1]. It limits the amount of data that's "in the cable" (which needs to be more if the cable is long). > The default SSH window size was 64 - 128 KB, which worked well for interactive sessions, but was severely limiting for bulk transfer in high bandwidth-delay product situations. > OpenSSH later increased the default SSH window size to 2 MB in 2007. 2 MB is still incredibly little. It means that on a 100 ms connection, you can not exceed 160 Mbit/s, even if your machines have 10 Gbit/s. OpenSSH is one of the very few TCP programs that have garbage throughput on TCP. This is also what makes rsync slow. The people from [1] patched that. You can support that work here: https://github.com/rapier1/hpn-ssh In my opinion, this should really be fixed in OpenSSH upstrea