[Sis-SCPS-INTEREST] SCPS + WAN Compression
Feighery, Patrick D.
feighery at mitre.org
Thu Jun 15 13:43:37 EDT 2006
Well hmmm.
Eric provided an excellent description for the interaction of Vegas and
compression.
To answer your question directly current_credit is the amount of credit
(in bytes) that you have in your token bucket for emitting packets.
Keep in mind that the gateway like most implementations of TCP only
want to send full sized segments. Sending a lot of runts (non full
sized segments) simply because you have some window or credit it
actually a bad think to do. Given that, current_credit in checked in
the beginning of tp_NewSend() to see if you have enough tokens to even
start the processing of trying to emit the packet. It compares
current_credit with the size of the packet (which is uncompressed in
your case). If this check fails no other processing will continue,
until this check has been passed. Even if you use Vegas, rate control
(current_credit) always provides cap on the amount of data that can be
emitted.
So given your description it does appear that Vegas may be limiting the
throughput. As you are aware of Vegas it compares the results of the
current RTT with the best RTT it sees. My fear is that due to
compression, rtt_best may approach 0 (may be only a few msec) which
would definitely be not good. The other though is you may need to
change the values of Vegas' ALPHA, BETA, and GAMMA - more of an art
than science..
Hope this helps...
Pat
>>-----Original Message-----
>>From: Frank Lynam [mailto:frank at klasonline.com]
>>Sent: Wednesday, June 14, 2006 5:41 AM
>>To: Feighery, Patrick D.;
>>sis-scps-interest at mailman.ccsds.org; travis at globalprotocols.com
>>Subject: RE: [Sis-SCPS-INTEREST] SCPS + WAN Compression
>>
>>Hi Pat/Eric,
>>
>>When using VEGAS, I have begun to get some rate improvement
>>by adding the
>>compressed data count to rt_route->current_credit variable
>>(after the total
>>packet size has been subtracted from it first). I do this in
>>tp_iovCoalesce
>>for each packet. However I don't seem to be able to modify
>>current_credit by
>>the actual amount of data that was compressed. I can only add
>>up to 1/4 of
>>the current packet size. Anything above this seems to either
>>cause SCPS or
>>the network I'm running SCPS across a problem (the bandwidth
>>drops off and
>>becomes very stop start. VEGAS calculations slow down and it
>>looks like data
>>is being lost in the network). Interestingly I am able to modify
>>current_credit by the actual compressed amount when running
>>over a nistnet
>>network simulator setup that has no network queue size limitations.
My
>>feeling is that if current_credit is modified by too great an
>>amount, it
>>overloads the network at any one time. This overloading causes my
live
>>network to drop packets but not nistnet because of the
>>limitless queues.
>>
>>When I switch to CC=0 (PURE-RATE) the changing of
>>current_credit by the
>>actual compression ratio provides dynamic bandwidth growth
>>that is in line
>>with the compression ratios. These PURE-RATE results were
>>seen when using
>>the line rate as the BIF_RATE setting. So all seems to be fine with
>>PURE-RATE.
>>
>>What is the role that rt_route->current_credit plays in the
>>transmitting of
>>data when using VEGAS? Even when I don't modify it all, I
>>never see it reach
>>0. Is this to be expected? How does it interact with
>>snd_cwnd? My initial
>>feeling when looking at the rt_route structure for the first
>>time was that
>>it had more to do with controlling data flow when using PURE-RATE
than
>>VEGAS.
>>
>>Thanks for all the help.
>>
>>Best regards,
>>
>>Frank Lynam
>>Senior Engineer
>>Klas Ltd.
>>-------------------------------------------
>>frank at klasonline.com
>>Ext: 307
>>
>>Tel: USA toll free 1-866-263-5467
>>Tel: Canada toll free 1-866-296-5199
>>Tel: UK toll free 0800-056-4250
>>Tel: International +353-1-662-4270
>>
>>Fax: USA toll free 1-866-532-3091
>>Fax: Canada toll free 1-866-237-0879
>>Fax: UK toll free 0800-056-4366
>>Fax: International +353-1-662-4272
>>
>>www.klasonline.com - 'Critical IT'
>>
>>-----Original Message-----
>>From: Feighery, Patrick D. [mailto:feighery at mitre.org]
>>Sent: 09 June 2006 18:04
>>To: Frank Lynam; sis-scps-interest at mailman.ccsds.org
>>Cc: Cian Masterson; Frank Murray
>>Subject: RE: [Sis-SCPS-INTEREST] SCPS + WAN Compression
>>
>>It is hard to figure out what might be happening...
>>
>>Some thoughts.
>>
>>If all of the traffic is not TCP based, their maybe other instances
of
>>current_credit you may need to change.
>>
>>With your new changes, set BIF_RATE to the capacity of the
>>link and run with
>>pure rate control CC=0 - what do you see? hopefully what you
>>see will be
>>data limited by the buffer sizes or link capacity, not rate
>>settings.
>>
>>tcptrace/xplot or other tools - should verify this.
>>
>>Now change to Vegas CC=2 and repeat the process. My initial
>>guess is there
>>may be an interaction between compression, variations in link
>>delay, and
>>Vegas. Again, looking at tcptrace graphs may provide some insights.
>>
>>I may be completely off base, but... Give it a shot.
>>
>> Pat
>>
>>
>>
>>>>-----Original Message-----
>>>>From: Frank Lynam [mailto:frank at klasonline.com]
>>>>Sent: Friday, June 09, 2006 11:44 AM
>>>>To: Feighery, Patrick D.; sis-scps-interest at mailman.ccsds.org
>>>>Cc: 'Cian Masterson'; 'Frank Murray'
>>>>Subject: RE: [Sis-SCPS-INTEREST] SCPS + WAN Compression
>>>>
>>>>Hi Pat,
>>>>
>>>>Thanks for the quick response. I now have the following setup:
>>>>
>>>>[LOCAL RFILE]
>>>>
>>>>BIF_BUF 1000000
>>>>BIF_RATE 440000
>>>>BIF_CC 2
>>>>
>>>>[LOCAL CODE]
>>>>
>>>>sWAN->rt_route->current_credit -= (currentbuffer->size /
>>>>current_compression_ratio);
>>>>Note that I'm not editing sWAN->rt_route->min_current_credit
>>>>as I don't have
>>>>MIN_RATE_THRESH defined.
>>>>
>>>>[REMOTE RFILE]
>>>>
>>>>BIF_BUF 1000000
>>>>BIF_RATE 4000000
>>>>BIF_CC 2
>>>>
>>>>[REMOTE CODE]
>>>>
>>>>NO CHANGES
>>>>
>>>>[NETWORK]
>>>>
>>>>NISTNET NET SIMULATOR: RTT=900->2400msecs, BANDWIDTH=400000bps
>>>>
>>>>I am doing FTP uploads from the local LAN. I now get a slight
>>>>improvement (average 78KBps) overall. Looking at the data
>>throughput
>>>>from the PC, it looks like the entire BIF_BUF amount is accepted by
>>>>the LOCAL SCPS peer at the start of the transfer and then it slowly
>>>>builds to just above the network bandwidth (50KBps). This
>>improvement
>>>>doesn't equate to the compression ratio (around 10 to 1 in
>>places) of
>>>>the packets.
>>>>Looking at a
>>>>NISTNET real-time network bandwidth display shows that the
>>bandwidth
>>>>is not being fully utilized. Sometimes it's as low as 10KBps.
>>>>
>>>>What do you think?
>>>>
>>>>Best regards,
>>>>
>>>>Frank Lynam
>>>>
>>>>-----Original Message-----
>>>>From: sis-scps-interest-bounces at mailman.ccsds.org
>>>>[mailto:sis-scps-interest-bounces at mailman.ccsds.org] On Behalf Of
>>>>Feighery, Patrick D.
>>>>Sent: 09 June 2006 15:33
>>>>To: Frank Murray; sis-scps-interest at mailman.ccsds.org
>>>>Cc: Cian Masterson
>>>>Subject: RE: [Sis-SCPS-INTEREST] SCPS + WAN Compression
>>>>
>>>>You are on the right track with your question about the
>>>>rt->route structure.
>>>>Essentially this structure contains information that is
>>shared amount
>>>>different connection's control blocks - this includes the rate
>>control
>>>>mechanisms.
>>>>
>>>>There are two fields in the rt_route structure
>>>> current_credit
>>>> min_current_credit
>>>>
>>>>These values are decremented in tp_output.c when a packet have been
>>>>emitted, by the size of that packet. You need to adjust these two
>>>>variables accordingly.
>>>>
>>>>Now as for the buffer sizes -- SCPS needs to buffer data at
>>the sender
>>>>in case retransmission is necessary and buffer data at the
>>receiver in
>>>>case it was received out of order. Just because you
>>compress the data
>>>>over the RF, SCPS still needs to buffer it. The buffer
>>size control
>>>>the maximum amount of data that can be emitted by the
>>sender, until it
>>>>receives an acknowledgement by the receiver. Typically in
>>a gateway
>>>>environment, as a rule of thumb, I set the buffer sizes to twice
the
>>>>bandwidth delay product. So if the bandwidth is 1Mbps and
>>the round
>>>>trip delay is 600 msec or 150 KBytes.
>>>>
>>>>By compressing the data, you are essentially increasing the
>>bandwidth
>>>>of the pipe; thus the buffer sizes will need to be increased. You
>>>>many to need to experiment a bit for an 'optimal value'
>>based on both
>>>>single and multiple flows.
>>>>
>>>>Does this make sense..
>>>>
>>>> Pat
>>>>
>>>>
>>>>>>-----Original Message-----
>>>>>>From: sis-scps-interest-bounces at mailman.ccsds.org
>>>>>>[mailto:sis-scps-interest-bounces at mailman.ccsds.org] On Behalf Of
>>>>>>Frank Murray
>>>>>>Sent: Friday, June 09, 2006 9:23 AM
>>>>>>To: sis-scps-interest at mailman.ccsds.org
>>>>>>Cc: Cian Masterson
>>>>>>Subject: [Sis-SCPS-INTEREST] SCPS + WAN Compression
>>>>>>
>>>>>>
>>>>>>
>>>>>>Hi all,
>>>>>>
>>>>>>We are using SCPS in gateway mode to accelerate TCP
>>traffic running
>>
>>>>>>over a satellite link. Native IP is used as the network protocol.
>>>>>>For congestion
>>>>>>control we are using Vegas. I've got a question about
>>>>compressing data
>>>>>>on a per packet basis after it has left SCPS on the WAN side. The
>>>>>>setup looks something like this:
>>>>>>
>>>>>>LAN INTERFACE
>>>>>>|
>>>>>>SCPS
>>>>>>|
>>>>>>COMPRESS/DECOMPRESS DATA
>>>>>>|
>>>>>>WAN INTERFACE
>>>>>>|
>>>>>>SAT LINK
>>>>>>|
>>>>>>WAN INTERFACE
>>>>>>|
>>>>>>COMPRESS/DECOMPRESS DATA
>>>>>>|
>>>>>>SCPS
>>>>>>|
>>>>>>LAN INTERFACE
>>>>>>
>>>>>>At the moment the WAN bandwidth is limited by the
>>throughout of the
>>
>>>>>>uncompressed data. I am trying to find a way to inform the
>>>>SCPS system
>>>>>>that instead of logging the transmit of x bytes for a particular
>>>>>>segment, an amount of x/[compression ratio] should in fact
>>>>be logged.
>>>>>>The current compression ratio is available in real-time as
>>>>each packet
>>>>>>is transmitted.
>>>>>>Our theory then is that SCPS should be able to increase
>>the rate at
>>
>>>>>>which it transmits data over the WAN interface as it would
>>>>then have a
>>>>>>correct reading of the amount of unack'ed data in the network.
>>>>>>
>>>>>>Up to now I hold the following assumptions:
>>>>>>
>>>>>>sWAN->snd_cwnd: WAN Congestion Window. This is
>>>>>>decremented when new
>>>>>>unack'ed data is introduced into the network and
>>>>incremented when the
>>>>>>data is ack'ed.
>>>>>>
>>>>>>sWAN->sndwin: WAN Send Window. This seems to behave
>>as
>>>>>>sWAN->snd_cwnd. I presume I'm wrong here.
>>>>>>
>>>>>>sWAN->snd_awnd: WAN Receive Window Of Remote
>>>>>>Peer. Again this is
>>>>>>decremented when new unack'ed data is introduced into the
>>>>network and
>>>>>>incremented when the data is ack'ed.
>>>>>>
>>>>>>All these variables are decremented in tp_iovCoalesce. I
>>have tried
>>
>>>>>>updating this change proportionally using the current compression
>>>>>>ratio. However the transfer still seems to be limited by
>>>>the BIF_RATE
>>>>>>and BIF_BUF parameters set at SCPS startup. If I set BIF_RATE and
>>>>>>BIF_BUF to be much greater at startup, the WAN throughput
>>>>increases as
>>>>>>expected. This however isn't an option as it is not dynamic
>>>>and causes
>>>>>>problems if compression is turned off or it's affects are
>>>>negligible.
>>>>>>
>>>>>>I also assume that these variables are incremented as the data is
>>>>>>ack'ed and these too would have to be changed
>>>>proportionally based on
>>>>>>the compression ratio.
>>>>>>
>>>>>>Finally I'm not sure what part the sWAN->rt_route structure
>>>>plays in
>>>>>>the limiting of throughput on the WAN interface. I have
>>>>noticed though
>>>>>>that the current_credit variable is updated in a similar
>>way to the
>>
>>>>>>variables listed above. The range of values seen here
>>>>though seem to
>>>>>>be lower.
>>>>>>
>>>>>>Let me know if you need any further information. I'd greatly
>>>>>>appreciate any help forthcoming.
>>>>>>
>>>>>>Best regards,
>>>>>>
>>>>>>Frank Lynam
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>_______________________________________________
>>>>>>Sis-SCPS-INTEREST mailing list
>>>>>>Sis-SCPS-INTEREST at mailman.ccsds.org
>>>>>>http://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-scps
-interest
>>>>>>
>>>>
>>>>_______________________________________________
>>>>Sis-SCPS-INTEREST mailing list
>>>>Sis-SCPS-INTEREST at mailman.ccsds.org
>>>>http://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-scps-interest
>>>>
>>>>
>>
>>
More information about the Sis-SCPS-INTEREST
mailing list