Using netcat and tar for network file transfer

Imagine you are on lan party or on the road and quickly want to transfer a file or directory to another computer. Both computer owners are just to lazy to setup something like ftp, smb, nfs. A very simple and even cross platform solution is using netcat and in case of a directory in combination with tar like the following steps. I will just show you how to use it without compression for a directory. Fell free to play around. You can test it locally of course.

1. The sender

The sender has to call netcat in server mode and pipe content into it. The next line tells tar to build a tarball and write it to standard output which is redirected via a pipe to netcat. Netcat is told to start in server mode (-l), listen on port 7878 (-p 7878) and shutdown itself after waiting 10 seconds after having seen an end of file in standard input (-q 10):

$ tar c directory | nc -q 10 -l -p 7878

2. The receiver

The receiver has to call netcat and tell him to connect to the remote machine and the correct port and redirects the standard output to a file. For convenience he also sets a timeout parameter (-w 10):

$ nc -w 10 remotehost 7878 > nameoftar.tar

That’s all. You just setup a very fast file transfer. For testing purposes use localhost. Please note that on the receiver side you are completely free to choose a file name for the .tar file. If you use something like gzip or bzip2 compression you should choose something like .tar.gz or .tar.bz2 of course.

Thanks to mnemonik pointing this out.

Feel free to add your file transfer quick hack here.

16 Gedanken zu “Using netcat and tar for network file transfer

  1. Pingback: The Linux Index » Caspar Clemens Mierau: Using netcat and tar for network file transfer

  2. tar stream is also much faster to copy files than scp „ping-pong“, especially when you have lots of files. to copy files from server to your machine, do this:

    ssh some-server ‚cd /some/dir && tar cz dir‘ | tar xz

    and the other direction:

    tar cz dir | ssh some-server ‚cd /some/dir && tar xz‘

  3. You can also clone a complete partition via netcat and dd:
    sender:
    dd if=/dev/sda | nc 192.168.0.100 1234
    receiver:
    nc -l -p 1234 | dd of=/dev/sda

  4. @bog: That’s totally right. I prefer tar-ssh over scp in a lot of cases, too. In this setting here I just assumed you don’t have and don’t want to setup a ssh server.

    @WebKid: Think you should the timeout flag as otherwise you have to ctrl-c after it’s complete. Correct me if I am wrong but with tar I had to do this.

  5. Another cool trick, to copy over directory, minus the ssh.

    On the „destination“ host where you want the files to end up:

    nc -l 7000 | tar -xvf –

    On the „source“ host, where the files are initially located:

    tac c | nc 7000

  6. Sorry, in the last comment itshould be:

    tar c directory_to_copy | nc destination_host 7000

  7. thanks for the tip.

    another scenario where this is a lot useful is transferring file over a VPN connection.

    this way you avoid the overhead – both in bandwidth usage and in CPU time – of double encypher/decypher and you can go as fast as you can. This can make a significant difference for slow connections and/or high load of data to transfer.

    decompressing the tarfile on the fly on the destination machine is even better.

  8. Pingback: nc (netcat) « boompty boomp

  9. IMHO, scp with proper options are fast enough:

    scp -c arcfour256 -C -4 user@remotehost:Downloads/bigfile.iso .

    These options can be made implicit by using alias or ssh config file, too :-)

  10. Pingback: Simple File Transfers | eightbits.org

  11. Had 1000’s of files varying from a few bytes to several 10’s of GB. Found that the following command provided the fastest transfer:

    tar cz SRC_DIR | ssh root@DEST_SVR ‚cd DEST_DIR && tar xz‘

    Over a crossover cable between servers it hovered at around 930 mbps.

  12. I have been using net cat for just a few days now.

    My preferred way of transferring a file or a directory over the network is

    Receiver: nc -l 3452 | bunzip2 -vvv | tar -x
    Sender: tar -c folder/ | bzip2 -vv9 | nc 127.123.0.1 3452

    The benefit is that I do not need to decompress the file on the receiving file, it decompresses by itself, I do not have to set a name or a folder name, tar will create those for me. I can just as easily omit the unzip and untag and save it.

    I know tar -j does compression too, but I prefer to see the terminal „doing something“ instead of staring at a blank screen.

    I have seen savings of 40% of transfer time using this method and I like it a lot, even more so when moving VMs from one server to another.

    Nice post.

  13. [root@localhost user]# tar c index.html | nc -q10 -l -7878
    nc: invalid option — ‚q‘
    usage: nc [-46CDdhklnrStUuvz] [-I length] [-i interval] [-O length]
    [-P proxy_username] [-p source_port] [-s source] [-T ToS]
    [-V rtable] [-w timeout] [-X proxy_protocol]
    [-x proxy_address[:port]] [destination] [port]

  14. Pingback: OpenWrt Sysupgrade | Gotoif.com

  15. Pingback: Using netcat and tar for network file transfer | pdasite

Schreib einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind markiert *