[Sis-SCPS-INTEREST] Connection being reset

[Nicolas Blais]

Hi,

I am using SCPS_RI_1_1_132 on FreeBSD 7 (both i386 and amd64 as a lib32 bin) and so far I am very impressed with the performance. Unfortunately, I am encountering problems when I pass more that 10 or so simultaneous connections on the lab. Multiple connections over a real satellite link were also an issue and I can't seem to pinpoint the actual problem.

Here's the lab setup:

client (test1) -> SCPS gateway -> KG-175 -> dummynet simulator <- KG-175 <- SCPS gateway -< client (test2)

The simulator simulates our 2Mbit sat-link with 520-560ms delay. The lab reproduces almost exactly our actual situation when the scps gateways are not installed (IE no TCP acceleration, therefore BW is limited to TCP window size)

The SCPS gateways were compiled with : --gateway=yes --gateway_larger=yes --tap=yes --low_idle=yes --low_cpu=yes --memory=512000000
and are physically very capable of handling the load.

As mentionned in the handbook, I modified thread.h to have:
MAX_SCPS_SOCKET 8192 and THREAD_SWAP_STACK_SIZE (131072*32)
while scps_defines.h has:
MAX_SCPS_SOCKET 8192
and ../include/scps.h
scps_fd_set[256]

Nothing special in the rfile except perhaps:
AIF_BUF 65536
AIF_TP_COMPRESS 1
AIF_ACK_BEHAVE 1
BIF_TP_COMPRESS 1

And I have the following sysctl options:
kern.ipc.maxsockbuf=1048576
kern.ipc.somaxconn=2048
net.inet.tcp.sendspace=65536
net.inet.tcp.recvspace=65536

When I use iperf to benchmark between test1 and test2 beyond 9 or 10 parallel TCP threads, I get broken pipes on the iperf initiating machine, and one of the gateways outputs multiple "gateway: Gateway: Connection being reset 20 0 15627"

On the real network, many multiple connections are also causing similar messages. Accelerations works fine (I get 1.6mbit out of our 2mbit link) but when a big transfer is done, it seems to close ongoing connections or not allow new ones to connect and drops them. Our actual network has around 30 Windows machines on a domain (I estimate about 10 connections per machine) running shares, sql queries, local irc connections to a main server, etc...

I tried to compile with --debug=yes to get more info, but I get the following error while compiling and didn't have a chance to look at it further:
tp_debugPort.c: In function 'fireman':
tp_debugPort.c:253: error: storage size of 'wstatus' isn't know

Any help is truly appreciated, as we would really like to have this operational on our network.

Thanks,
Nicolas.