[Sis-SCPS-INTEREST] temporary ouf of sockets

[Feighery, Patrick D.]

The error indicates you are out of buffers in the memory pool.  At a
minimum 700 active connections will require __at least__ 100 Mbytes of
memory in the buffer pool.  And maybe a lot more depending on your
resource file.

You have two options.
	1)  Increase the size of the memory pool  and/or
	2)  Decrease the buffer size for the WAN side of
	    the rfile.

Keep in mind let's assume you have a 45 Mbps WAN pipe, with 700 active
connections each connection (assuming everything is shared equally)
will consume about 64Kbps of bandwidth.  With 2000 active connections,
each on average will only consume about 22Kbps of bandwidth..  Setting
the buffer sizes much larger than that will only consume more resources
in the memory pool...


As for benchmarking I've mainly used a couple of home grown tools.
Other people on the list may be able to provide suggestions...

	Pat



>>-----Original Message-----
>>From: sis-scps-interest-bounces at mailman.ccsds.org 
>>[mailto:sis-scps-interest-bounces at mailman.ccsds.org] On 
>>Behalf Of suvashrestha at wlink.com.np
>>Sent: Thursday, August 17, 2006 2:49 AM
>>To: sis-scps-interest at mailman.ccsds.org
>>Subject: [Sis-SCPS-INTEREST] temporary ouf of sockets
>>
>>hi,
>>
>>i had already made changes to the following files (thread.h,
>>scps_defines.h, scsp.h) to allow 2000 connections.
>>
>>and also configured with gateway_larger = yes.
>>
>>i still get the following warning when i reach about 700 connections:
>>gateway: reseting s1 - not enough resources
>>gateway: failed in chain init 3
>>gateway: not allowing any more connections ss2
>>
>>Could this be due to the lack of RAM??? and what would u suggest i
use
>>for benchmarking?? right now i am using iperf to create the number of
>>simultaneous streams.
>>
>>thank u
>>
>>
>>
>>
>>On 8/10/2006, "Feighery, Patrick D." <feighery at mitre.org> wrote:
>>
>>>Well as you found, the default number of active connections the SCPS
>>>gateway will support is much less that 2000.
>>>
>>>There are definitely techniques to increase them, but before 
>>I go into
>>>much detail, you need to look at the bandwidth of the system, the
>>>detail through the system and the number of connections you 
>>really want
>>>to support.
>>>
>>>If the bandwidth of the system is let's say 5 Mbps then each 
>>connection
>>>(assuming all are active at the same time) would get about 
>>2500 bps...
>>>
>>>First lets talk about memory.   As a default the SCPS 
>>allocates a pool
>>>of 10 Mbps for buffering the data through the system.  Since 
>>buffer is
>>>allocated in 32K chunks, this results is about 300 buffers for the
>>>system.  Each E2E connection through the system requires both a
>>>transport later connection on the WAN and LAN side, and each 
>>connection
>>>has at a minimum of a send and receive buffer so at a minimum each
>>>connection needs 4 buffers = 300/4 approx 75 connections 
>>support.  2000
>>>connections at a minimum requires 2000*4*3276 == approx 1/4 gig of
>>>memory dedicated to the proxy.  Best case.  see Section 9 in 
>>the users
>>>manual for a little more detail.
>>>
>>>
>>>As for the number of sockets, section ten describes the details.
>>>However the following is that you need to do.
>>>
>>>1)  First configure the code with the --gateway_larger=yes
>>>
>>>2)  Make the changes to the following three source code files...
>>>
>>>In ./source/thread.h change
>>>#define MAX_SCPS_SOCKET	512
>>>          to
>>>#define MAX_SCPS_SOCKET	4096
>>>
>>>In ./source/scps_defines.h change
>>>#define MAX_SCPS_SOCKET	512
>>>          to
>>>#define MAX_SCPS_SOCKET	4096
>>>
>>>In ./include.scps.h change
>>>typedef long scps_fd_set[16];   /* Assumes long = 4 bytes = 
>>32 bits */
>>>To
>>>typedef long scps_fd_set[128];   /* Assumes long = 4 bytes = 
>>32 bits */
>>>
>>>Hope this helps.  Honestly I have not pushed the SCPS RI to 
>>this level
>>>yet.
>>>
>>>Best Regards
>>>
>>>	Pat
>>>
>>>
>>>
>>>
>>>
>>>
>>>>>-----Original Message-----
>>>>>From: sis-scps-interest-bounces at mailman.ccsds.org 
>>>>>[mailto:sis-scps-interest-bounces at mailman.ccsds.org] On 
>>>>>Behalf Of suvashrestha at wlink.com.np
>>>>>Sent: Monday, August 07, 2006 6:26 AM
>>>>>To: sis-scps-interest at mailman.ccsds.org
>>>>>Subject: [Sis-SCPS-INTEREST] temporary ouf of sockets
>>>>>
>>>>>Hello,
>>>>>
>>>>>I am running SCPS TP on a network as follows on freeBSD 6.0:
>>>>>
>>>>>client 1 -- SCPSgateway -- satellite emulator -- SCPSGateway --
>>>client
>>>>>2.....
>>>>>
>>>>>I want to generate around 2000 connections simultaneously. 
>>>>>When i reach
>>>>>upto around 500 connections, i get the following messages :
>>>>>We are temporary out of sockets.....reseting s1- not enough 
>>>>>resources....
>>>>>Gateway not allowing any more connections ss2... on the 2 
>>>>>SCPS gateways.
>>>>>
>>>>>How should i go about benchmarking??
>>>>>What size of RAM, nmbclusters, somaxconn, sendspace, recvspace
etc
>>>is
>>>>>advisable for the 2000 connections???
>>>>>
>>>>>Thanks in advance.
>>>>>
>>>>>_______________________________________________
>>>>>Sis-SCPS-INTEREST mailing list
>>>>>Sis-SCPS-INTEREST at mailman.ccsds.org
>>>>>http://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-scps-interes
t
>>>>>
>>>
>>>
>>
>>_______________________________________________
>>Sis-SCPS-INTEREST mailing list
>>Sis-SCPS-INTEREST at mailman.ccsds.org
>>http://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-scps-interest
>>