How To Boost File Transfer Speeds With Aftp
How To Boost File Transfer Speeds With Aftp
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 1
Introduction
It's hard to imagine that in an age where mere household broadband
connections are already in the Megabits (or even hundred Megabits)
per second, some large organizations still have to transfer files at
unbelievably slow speeds.
Slow file transfers may not be a big issue if you're just sharing small,
personal files. But if you're working in an organization that has to process
gigabytes of mission-critical data under a tight schedule, a transfer speed
of just a few hundred kbps may not be acceptable, unless of course,
If you want to nothing can be done about it.
understand where Well, for a long time, that has been the case for file transfers between
speed limitations are offices of multinational corporations, universities, governments, research
coming from and facilities, and other organizations separated by large distances. The
how to overcome reason is that networks spanning long distances are subject to high
them, this paper latency and other factors that result in poor network conditions.
is for you.
These conditions can lead to failed file transfers, significant delays in
processing of digital data, underutilization of network resources, and
even lost business opportunities.
If you want to understand where these speed limitations are coming from
and what you can now do to overcome them, then this paper is for you.
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 2
When you send information during a file transfer session, it is first
chopped into smaller pieces and then encapsulated into what are known
as packets.
Aside from the pieces of information you send, which is known as the
payload, each packet comes with a bunch of additional information
comprising what is known as the TCP Header.
It is these packets, made up of the payload and TCP Header that are sent
over the network to the intended receiver or destination. It is also these
packets that are directly affected by the network conditions we are about
to discuss.
Latency
When packets are transmitted over a local area network (LAN), they
appear to arrive instantaneously. In reality, however, they do not.
Again, the greater the value of latency, the longer the delay. So, if you
have a high-latency network, you can expect significant delays.
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 3
Packet Loss
Not all packets you send arrive at their destination. Some of them are
dropped/lost along the way. This can happen due to a variety of reasons
like signal degradation, faulty networking devices or drivers, and
congestion. Naturally, if a packet is lost, the information that will be
received on the other end would not be complete.
In some cases, that's not a big issue. For instance, a few lost packets
won't have a critical impact when you're having a VoIP session, playing a
Even a single lost real-time online multiplayer game, or viewing streaming media. Audio
packet can be a big and video quality may suffer a little but that's all.
problem because
some text and other However, if you're sending a document, even just a single lost packet
portions may end can be a big problem because some text and other portions may end
up missing. up missing.
Bandwidth
If you have ever taken part in a buying decision involving network
devices, Internet connections, and other stuff related to networking,
chances are bandwidth was one of your major considerations. That's
because sales people often put emphasis on bandwidth when they sell
you networking products.
All things being equal, when you compare two networks or devices with
different bandwidths, the one that has greater bandwidth is normally
understood to be faster.
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 4
Technologies Bandwidths
T1 / DS1
1.5 Mbps
Wireless 802.11g
11 Mbps
T3 / DS3
45 Mbps
10 Gigabit Ethernet
10 Gbps
Again, those are just theoretical values. In reality, due to factors like
latency and packet loss, it's almost impossible to achieve them. In fact, in
wide area networks, the actual rate of data transfer, which is known as
throughput, is usually just a small fraction of advertised bandwidth.
Congestion
What happens when the total volume of data simultaneously directed to a
T1 connection exceeds its bandwidth of 1.5 Megabits per second? The
same thing that happens on most main roads during peak hours. You get
traffic congestion, and in the case of networks, network congestion.
Just like roads, a Just like roads, a congested network experiences reduced throughput. All
congested network types of networks have bandwidth limitations. The moment the volume of
experiences reduced incoming data exceeds a network's maximum capacity, congestion
throughput. occurs.
But bandwidth is not the only network resource that has limitations.
Network devices such as routers and switches have portions in their
memory known as buffers in which received data is temporarily stored
before they are processed. Just like any kind of memory, these buffers
cannot accommodate an infinite amount of data. If the buffers start filling
up because incoming data is more than the buffer can handle, this too
can cause congestion.
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 5
Network congestion is a very undesirable condition because it can also
increase latency and packet loss, which can aggravate the situation. This
is how it happens.
When the rate of output on a device gets significantly lower than the rate
of input, as what happens during congestion, packets in queue start
backing up and processing gets delayed. Thus, latency eventually
increases.
At the same time, when the receiving device's buffer starts filling up, the
device starts dropping packets. But that's not the end of the story. The
sending device, upon realizing that packets have been lost, starts
resending packets. These new packets add to the congestion and the
situation worsens.
1. Reliable transmission
TCP is designed to provide reliable delivery of information. That means,
if you send a document using TCP, your recipient can expect to get it in
exactly the same form as the original.
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 6
end, the receiver responds with an acknowledgement to inform the
sender that the data arrived successfully.
2. Flow Control
TCP is also equipped with a flow control mechanism aimed at preventing
the sending device from transmitting an amount of data greater than
what the receiving device or the network can handle. This is known as
the TCP sliding window flow control algorithm. How does it work?
Let's say the TCP window is 4. In this case, the sending device can
send out a maximum of 4 packets before stopping to wait for an
acknowledgement.
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 7
This also means that the number of packets that the receiver is allowed
to send before it has to stop to wait for an acknowledgement slowly
increases. When you have a larger window, the number of packets on
their way to the receiver at a given time will be greater.
The increase of the TCP window slows down again once a certain
threshold limit is reached. This threshold is initially set by the receiver
at the start of the connection and is the maximum amount of data that
can be accommodated by the receiver's buffer.
Once the threshold is reached, the TCP session enters what is known as
the congestion avoidance state. At this point, the system would have
reached what it perceives as the highest throughput possible.
Remember, if you have a small TCP window, the sender can only send a
few packets at a given time. So really, it would be in your best interests
if the TCP window would grow quickly.
There wouldn't be an issue if both the sender and receiver are located in
the same building or compound. The acknowledgement would arrive
instantly and the sender's “waiting time” would be negligible.
But if you have a high latency network wherein the distance between
sender and receiver is significantly large, e.g. if the sender is in LA and
the receiver is in Tokyo, then the sending machine would have to wait
much longer.
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 8
Things can get worse when packets or acknowledgements get lost. As
soon as the sender detects packet loss, like when an acknowledgement
fails to arrive, it reduces the TCP window considerably and the session
goes back to slow start state.
Not only that. The system would also change its perception as with
regards to the highest throughput possible. As a result, it will also
significantly reduce the threshold limit we talked about earlier. This now
has serious consequences.
Even if the TCP window goes back up after slow start, its rise will be
limited by this greatly reduced threshold limit. Thus, even a small degree
of packet loss can be catastrophic to your file transfer.
Here's how throughput fared at zero packet loss but increasing latency:
35,000
30,000
25,000
22,136
20,000
15,000
9,416
10,000
4,936
5,000 2,488
0 1,000
0 10 25 50 100 250
Latency (ms)
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 9
with zero packet loss. As you can see, while a latency of 200ms may
seem negligible, the resulting throughput will tell you it isn't.
35,000 34,128
30,000
25,000
20,000
15,000
10,000
4,544
5,000
0 832
0 0.1 0.5 1 5 10
Packet Loss (%)
Notice how steeply throughput drops beyond 1% packet loss. That's the
packet loss you'd normally experience when transmitting from LA to
Tokyo.
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 10
Figure 3: Effects of Latency & Packet Loss (0.5%) – FTP
40,000
37,323
35,000
Throughput (Kbps)
30,000
25,000
20,000
15,000
10,000 8,616
3,560
5,000 1,776 896
350
0
0 10 25 50 100 250
Latency (ms)
When you know that your throughput's only going to be like this, i.e., no
more than 1% of bandwidth, it's not going to be easy to approve your
staff's request for larger bandwidth.
Not all transmissions are carried out using TCP. The relatively steady flow
of data in real-time online multiplayer games (or at least the real-time
aspects of these games), VoIP, and video streaming certainly could not
be achieved if you rely purely on TCP.
Obviously, a much faster protocol is being brought into play here. That
protocol is the User Datagram Protocol or UDP.
Thus, if your sending device transmits packets using simply UDP, the
packets that arrive at the receiving end may not be in order and may
suffer duplication. Furthermore, if a packet gets lost, your sender and
receiver may not know about it.
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 11
But if you recall our discussion on packet loss, that's not always a
problem. A couple of lost packets certainly won't have a critical effect on,
say, a VoIP session.
There may be a slight degradation in voice quality, but you'll still be able
to understand what the other person is saying. In fact, the number of lost
packets may be so small that you may not be able to notice anything
wrong with the voice quality at all.
In a typical AFTP file transfer, the bulk of the data transfers are done on a
UDP channel while other tasks such as user authentication, file
Fast and reliable file management, and the coordination of the file transfers are done on a TCP
transfers can actually channel.
be achieved if you
combine the Because UDP does not have an acknowledgement process like TCP, it is
strengths of TCP not as affected by latency and packet loss. Hence, a file transfer using
and UDP. AFTP performs much better than one relying purely on FTP.
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 12
Figure 4: Effects of Latency – FTP vs. AFTP
45,000
40,000
35,000
Throughput (Kbps)
30,000
25,000
20,000
15,000
10,000
5,000
0
0 10 25 50 100 250
Latency (ms)
FTP AFTP
0 10 25 50 100 250
30,000
25,000
20,000
15,000
10,000
5,000
0
0 0.1 0.5 1 5 10
Packet Loss (%)
FTP AFTP
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 13
0 0.1 0.5 1.0 5 10
Notice how AFTP throughput remains almost unchanged. Here's one more
graph showing what happens if you introduce both latency and packet
loss:
Figure 6: Effects of Latency & Packet Loss (0.5%) – FTP vs. AFTP
45,000
40,000
35,000
Throughput (Kbps)
30,000
25,000
20,000
15,000
10,000
5,000
0
0 10 25 50 100 250
Packet Loss (%)
FTP AFTP
0 10 25 50 100 250
Clearly, AFTP outperforms pure FTP by a wide margin and the advantage
gets even more pronounced as network performance deteriorates, i.e.,
when there is greater latency and packet loss.
So what particular product can boost your file transfers at least a 100x?
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 14
Choosing a file transfer solution
You might be surprised, but there are already a few file transfer
programs that support some form of accelerated file transfer. And in most
cases, these products don't differ much in terms of improved throughput.
But then that doesn't mean you can just go on and pick a product
randomly. Throughput shouldn't be the only factor you need to consider
when choosing a file transfer solution.
Is it platform independent?
It's not unusual for a large organization to maintain computers having
different operating systems. Some of your employees may be using
Windows laptops and desktops. Others may be using Macs. Some of your
servers may be running on Linux, while others on Solaris, or AIX.
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 15
Does it automate business processes?
With so many tasks vying for attention everyday, having the ability to
automate some business processes can help you get more things
Having the ability accomplished much faster.
to automate some
It would therefore be a big boost to your team's productivity if your file
business processes
transfer solution were already equipped with features that automate
can help you get
certain processes. Some tasks that should be automated whenever
more things
possible include the following:
accomplished
much faster. p File transfers to trading partners especially during non-office hours
when bandwidth utilization is low;
p Virus scans on every file upload;
p PGP encryptions on every file upload;
p Compression/decompression of transferred files;
p Email notifications regarding certain events such as those mentioned in
this list;
For example, if a data breach hits you, you can conduct a more thorough
investigation if you can identify who accessed what during a particular
period of time. A file transfer solution with logging capabilities can be a
big help in these situations.
These days, threats against confidential data can come from both outside
and within your organization. It is therefore important for your file
transfer solution to be equipped with basic and advanced security
features like DLP (data loss prevention), access controls, encryption, two-
factor authentication, and others.
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 16
This will enable you with enough flexibility to implement appropriate
security measures when addressing identified risks.
Example Uses
p Provide a highly secure method for exchanging data both internally and
between trading partners that meets PCI DSS, SOX, HIPAA and GLBA
compliance requirements
p Simplify file transfers by unifying all services and processes into a
single platform independent solution
p Gain control of users and system resources
p Automate file transfers between trading partners and respond to server
side events
p Provide a secure web-based file transfer gateway to clients without
installing any software
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 17
Feature Summary
p Platform independent solution with installers for Windows, Linux,
Solaris, UNIX, AIX and Mac OS X operating systems
p Support for all popular file transfer protocols including FTP/S, SFTP,
SCP and HTTP/S
p Accelerated file transfer over high speed / high latency networks moves
files up to 100 times faster than FTP
p Brandable and easy to use web based client interface for transferring
and managing files
p Support for multiple domains each with their own unique file transfer
services and configurations
p Multi-lingual web user interface with built in language packs for
English, Spanish, French, German and Russian
p Authentication modules for integrating with existing LDAP, Active
Directory, NTLM, PAM and database user repositories
p Triggers module for use in automating business processes such as file
transfers with trading partners and responding to server events
p Data Loss Prevention module for use in preventing leakage of sensitive
data such as credit card, SSN and bank account numbers.
p Jailed user accounts and virtual file system keep your users data separate
p Ad-hoc file transfer module for granting non-account holders temporary
access to data
p Several security features designed to meet PCI DSS, HIPAA, SOX and
GLBA compliance requirements
p Support for high availability and load balancing environments
p Integrated key management tools for OpenPGP, SSH and SSL
standards
p Directory monitors with ability to detect when directories have been
modified
p Java based API for managing the server, users and configuration
settings
p Integrity checksum and resume file transfer capabilities
p ... and much more
Download
Start accelerating your file transfers today by downloading a free fully
functional evaluation of JSCAPE MFT Server.
http://www.jscape.com/products/file-transfer-servers/jscape-mft-server/
How to Boost File Transfer Speeds 100x Without Increasing Your Bandwidth 18