Comments on: Using netcat and tar for network file transfer http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/?utm_source=rss&utm_medium=rss&utm_campaign=using-netcat-and-tar-for-network-file-transfer Wed, 01 Feb 2012 19:40:34 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: Kryztoval http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/comment-page-1/#comment-67010 Kryztoval Thu, 08 Dec 2011 16:59:09 +0000 http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/#comment-67010 I have been using net cat for just a few days now. My preferred way of transferring a file or a directory over the network is Receiver: nc -l 3452 | bunzip2 -vvv | tar -x Sender: tar -c folder/ | bzip2 -vv9 | nc 127.123.0.1 3452 The benefit is that I do not need to decompress the file on the receiving file, it decompresses by itself, I do not have to set a name or a folder name, tar will create those for me. I can just as easily omit the unzip and untag and save it. I know tar -j does compression too, but I prefer to see the terminal "doing something" instead of staring at a blank screen. I have seen savings of 40% of transfer time using this method and I like it a lot, even more so when moving VMs from one server to another. Nice post. I have been using net cat for just a few days now.

My preferred way of transferring a file or a directory over the network is

Receiver: nc -l 3452 | bunzip2 -vvv | tar -x
Sender: tar -c folder/ | bzip2 -vv9 | nc 127.123.0.1 3452

The benefit is that I do not need to decompress the file on the receiving file, it decompresses by itself, I do not have to set a name or a folder name, tar will create those for me. I can just as easily omit the unzip and untag and save it.

I know tar -j does compression too, but I prefer to see the terminal “doing something” instead of staring at a blank screen.

I have seen savings of 40% of transfer time using this method and I like it a lot, even more so when moving VMs from one server to another.

Nice post.

]]> By: Litch http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/comment-page-1/#comment-42618 Litch Tue, 12 Jul 2011 08:22:57 +0000 http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/#comment-42618 Had 1000's of files varying from a few bytes to several 10's of GB. Found that the following command provided the fastest transfer: tar cz SRC_DIR | ssh root@DEST_SVR 'cd DEST_DIR && tar xz' Over a crossover cable between servers it hovered at around 930 mbps. Had 1000′s of files varying from a few bytes to several 10′s of GB. Found that the following command provided the fastest transfer:

tar cz SRC_DIR | ssh root@DEST_SVR ‘cd DEST_DIR && tar xz’

Over a crossover cable between servers it hovered at around 930 mbps.

]]> By: Marcel Duran http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/comment-page-1/#comment-40775 Marcel Duran Thu, 30 Jun 2011 00:25:17 +0000 http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/#comment-40775 On Mac, netcat has no -q option nor -p seems to work as expected. The following solution (as "the sender") seems to work fine: tar c directory | nc -l localhost_or_ip 7878 found on: http://superuser.com/questions/115553/netcat-on-mac-os-x On Mac, netcat has no -q option nor -p seems to work as expected. The following solution (as “the sender”) seems to work fine:

tar c directory | nc -l localhost_or_ip 7878

found on: http://superuser.com/questions/115553/netcat-on-mac-os-x

]]> By: Simple File Transfers | eightbits.org http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/comment-page-1/#comment-30770 Simple File Transfers | eightbits.org Thu, 10 Feb 2011 05:51:05 +0000 http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/#comment-30770 [...] of performing file transfers with the netcat command. After reading up about it in this article and this article, I decided to throw together a couple of Bash scripts to make it even easier. The transmit script [...] [...] of performing file transfers with the netcat command. After reading up about it in this article and this article, I decided to throw together a couple of Bash scripts to make it even easier. The transmit script [...]

]]> By: DucQuoc.wordpress.com http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/comment-page-1/#comment-25891 DucQuoc.wordpress.com Sat, 13 Nov 2010 08:09:11 +0000 http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/#comment-25891 IMHO, scp with proper options are fast enough: scp -c arcfour256 -C -4 user@remotehost:Downloads/bigfile.iso . These options can be made implicit by using alias or ssh config file, too :-) IMHO, scp with proper options are fast enough:

scp -c arcfour256 -C -4 user@remotehost:Downloads/bigfile.iso .

These options can be made implicit by using alias or ssh config file, too :-)

]]> By: nc (netcat) « boompty boomp http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/comment-page-1/#comment-8926 nc (netcat) « boompty boomp Thu, 23 Apr 2009 02:42:54 +0000 http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/#comment-8926 [...] here for more [...] [...] here for more [...]

]]> By: Anal http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/comment-page-1/#comment-936 Anal Tue, 27 May 2008 14:49:11 +0000 http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/#comment-936 thanks for the tip. another scenario where this is a lot useful is transferring file over a VPN connection. this way you avoid the overhead - both in bandwidth usage and in CPU time - of double encypher/decypher and you can go as fast as you can. This can make a significant difference for slow connections and/or high load of data to transfer. decompressing the tarfile on the fly on the destination machine is even better. thanks for the tip.

another scenario where this is a lot useful is transferring file over a VPN connection.

this way you avoid the overhead – both in bandwidth usage and in CPU time – of double encypher/decypher and you can go as fast as you can. This can make a significant difference for slow connections and/or high load of data to transfer.

decompressing the tarfile on the fly on the destination machine is even better.

]]> By: TJB http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/comment-page-1/#comment-738 TJB Mon, 28 Apr 2008 14:37:39 +0000 http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/#comment-738 Sorry, in the last comment itshould be: tar c directory_to_copy | nc destination_host 7000 Sorry, in the last comment itshould be:

tar c directory_to_copy | nc destination_host 7000

]]> By: TJB http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/comment-page-1/#comment-737 TJB Mon, 28 Apr 2008 14:36:35 +0000 http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/#comment-737 Another cool trick, to copy over directory, minus the ssh. On the "destination" host where you want the files to end up: nc -l 7000 | tar -xvf - On the "source" host, where the files are initially located: tac c | nc 7000 Another cool trick, to copy over directory, minus the ssh.

On the “destination” host where you want the files to end up:

nc -l 7000 | tar -xvf -

On the “source” host, where the files are initially located:

tac c | nc 7000

]]> By: ccm http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/comment-page-1/#comment-37 ccm Sun, 30 Dec 2007 12:27:19 +0000 http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/#comment-37 @bog: That's totally right. I prefer tar-ssh over scp in a lot of cases, too. In this setting here I just assumed you don't have and don't want to setup a ssh server. @WebKid: Think you should the timeout flag as otherwise you have to ctrl-c after it's complete. Correct me if I am wrong but with tar I had to do this. @bog: That’s totally right. I prefer tar-ssh over scp in a lot of cases, too. In this setting here I just assumed you don’t have and don’t want to setup a ssh server.

@WebKid: Think you should the timeout flag as otherwise you have to ctrl-c after it’s complete. Correct me if I am wrong but with tar I had to do this.

]]>