[Sis-SCPS-INTEREST] SCPS Gateway - Memory usage
Feighery, Patrick D.
feighery at mitre.org
Wed Nov 30 11:30:29 EST 2005
Well in terms of the memory issues, there is a bug in the code that has
been recently reported to me and been corrected.
MITRE is working on a new technique for users to obtain copies of the
SCPS RI software distribution. We will be using a service called open
channel foundation (https://www.openchannelsoftware.org) to handle the
distribution of the software. This will replace the antiquated process
of sending an e-mail to Adrian Hooke who forwards it to MITRE. MITRE
then e-mails you a copy of the SCPS RI. Once you register at the
site, you can click on
(https://www.openchannelsoftware.org/projects/SCPS) to obtain copies of
the SCPS RI. The software has been and will remain free for the
downloading. In addition, this site has a nice feature called 'Stay Up
To Date' to notify you automatically when a new version of the SCPS RI
is available. I was told my open channel that it should be up the
week.
Once the site becomes live, I was going to post an e-mail to
sis-scps-interest at mailman.ccsds.org to provide a new release that fixes
this and other issues along with adding some new features. However you
beat me to the punch. Therefore, I've attached the text I was holding
until I got scps open channel site live.
If you want a copy of the newest code now, I will forward it to you
right away, but I would still recommend signing up on the openchannel
site, to make sure you stay up-to-date...
Best Regards
Pat
=========================================================
For those of you that don't have a copy of the SCPS Reference
Implementation, you can delete this now.
A problem has been identified and corrected that may cause some data
integrity issues in the SCPS Reference Implementation (version 1.1.124
and below). This has been corrected in version 1.1.126 of the
software.
Here is a quick synopsis
This problem occurs when data at the end transfer is lost and
subsequently retransmitted on the LAN side before it gets to the SCPS
Proxy - which should have a very low probability. This only occurs if
the segment closing the transport session (contains a FIN flag) also
contains data AND the preceding segment or two is lost. When the proxy
receives the last packet, it places it on the 'out of sequence' queue
until the missing packet gets retransmitted. It turns out if this
segment contains the end of transmission (TCP FIN) flag, the proxy does
not copy the data to the 'out of sequence' queue. Not good. This is
the source of the problem. The proxy then takes the data received from
the first TCP connection and transmits it via the SCPS connection. But
since the data was not copied to the out of sequence queue, the data
contained in those memory locations (all zeroes in our case) is copied
over and eventually is received by the peer applications.
When the proxy sends the last piece of data in a transfer it will NEVER
send the TCP FIN flag in the same segment as data. Therefore when the
proxy emits data either over the WAN (MCAP) or on the remote LAN, this
situation will never occur. This situation must occur on the source
LAN side. This is why this issue has only been recently raised to
MITRE.
In addition to the above problem, A new type of packet emission policy
has been developed based on trying to keep the bottle neck interface
queue non-empty. BIG CAVEAT: This technique has not been documented
in any CCSDS arena. It has been developed based on some MITRE related
work that has been incorporated back into the SCPS RI.
SCPS rate control works best when a known dedicated bandwidth is
provided to SCPS. There are occasion however when the bandwidth is
still dedicated, but can vary. For example.
O There may be data compression on the link which effectively
increases the overall bandwidth to SCPS.
O The link layer may perform some reliability techniques causing
retransmissions to occur at the link layer. This effectively
decreases the overall bandwidth to SCPS.
O The link may be half-duplex or need to be scheduled, thus
decreasing the overall bandwidth to SCPS.
The idea here is the best throughput that can be obtained when there is
always data in the bottle resource interface queue to be sent, but not
too much to adversely interact with the transport layer. From an
application's perspective, the best throughput than can be obtained is
when the pipe is constantly full and data does not get retransmitted.
The subnetwork knows how much data is currently in the transmit
interface queue and based on that periodically sends a message to the
proxy indicating how may bytes it is willing to receive. The SCPS RI
supplements the rate based emission strategy with a new token bucket
strategy, where the token consumed are queue space in the interface
queue. With this technique SCPS will only send data if both rate
control and interface queue tokens are available.
In addition, if the link does temporarily down, a link down indication
message can be sent to SCPS,. SCPS will persist at 1 second intervals
until the link is available. Further a message is also sent to SCPS,
when the link is available.
This technique has been tested at relatively low data rate << 100K
where there is a tight feedback loop between the bottleneck resource
interface and SCPS.
Under the utils direction, there is a ppp_current_tx_len directory that
explains this in a little more detail. This directory also provides a
utility to monitor the current queue length of a ppp interface.
----------------------------
Besides the change listed above, following are some even shorter
synopses of some changes from the 1.1.119 release - which we have been
sending out for the last couple of months.
----------------------------
Thanks to Marcin Jessa at yazzy at yazzy.com, there is an initial port of
the SCPS gateway for the FreeBSD/NetBSD operation system via the TAP
interface. This requires the latest and greatest copy of FreeBSD or
NetBSD to work. See README.GATEWAY.FREEBSD.TAP in the distribution for
more details.
----------------------------
Corrected a memory leak in the SCPS RI code. - we had a extra malloc
that should not have been there.
----------------------------
Added a couple of ifdef compilation blocks around a syslog call in
scps_ri_console to write to the syslog if we explicitly want.
----------------------------
Corrected some more compilation issues with newer versions of gcc.
----------------------------
MITRE is trying a new technique for users to obtain copies of the SCPS
RI software distribution. We are using a service called open channel
foundation (https://www.openchannelsoftware.org) to handle the
distribution of the software. The software has been and will remain
free for the downloading. Once you register at the site, you can click
on (https://www.openchannelsoftware.org/projects/SCPS) to obtain copies
of the SCPS RI. In addition this site has a nice feature called 'Stay
Up To Date' to notify you automatically when a new version of the SCPS
RI is available.
Best Regards
Pat
=========================================================
>>-----Original Message-----
>>From: sis-scps-interest-bounces at mailman.ccsds.org
>>[mailto:sis-scps-interest-bounces at mailman.ccsds.org] On
>>Behalf Of Walid Assafiri
>>Sent: Monday, November 28, 2005 6:38 PM
>>To: sis-scps-interest at mailman.ccsds.org
>>Subject: [Sis-SCPS-INTEREST] SCPS Gateway - Memory usage
>>
>>Hello guys,
>>
>>My team has currently got SCPS running using the Tun devices.
>>At the moment we have a limited amount of memory on our linux
>>box for the SCPS to run on.
>>We have used the "top" command to see the actual memory usage
>>of the SCPS gateway and it turns out that it uses around 2.84Mbytes.
>>Can any of you SCPS gurus shed some light as to why it might
>>be using this much memory, just for running the Gateway with
>>no connections being established?
>>Furthermore, the CPU usage is also very high using up 99.6%
>>of the CPU.
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 3548 root 25 0 4136 2840 616 S 99.6 1.2 3:47.13 gateway
>>
>>Would the CPU problem be due to some polling, running the
>>gateway in user space mode or any other ideas?
>>We're thinking that the memory usage and/or the CPU may be
>>specific to your implementation and not to the SCPS algorithms.
>>Any ideas/thoughts would be greatly appreciated.
>>
>>Regards,
>>Walid.
>>
>>
>>_______________________________________________
>>Sis-SCPS-INTEREST mailing list
>>Sis-SCPS-INTEREST at mailman.ccsds.org
>>http://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-scps-interest
>>
More information about the Sis-SCPS-INTEREST
mailing list