SCP sucks at small files. It also doesn’t give you much control over what is preserved or sent from one side to the other. So for large transfers of small files it’s really not the right choice.
One might say “Oh, you should use rsync” but it has a lot of overhead as well and, while it’s faster, it’s not nearly as fast as…
$ cd destination/root/; ssh server.local tar cf /some/path | tar xk
The -k is for “keep” and won’t overwrite files you already have. tar excels at small files and preserving attributes and I see speeds of 20-60MB/s on GigE for large maildirs with this method (rsync got around 6-10MB/s and scp somewhere around 100KB/s. Yes, Kilo.).
Downside: While -k will prevent local overwrites, this is not a sync. The remote tar will send you everything on that side. This is for an initial copy, not for incremental sends (for which rsync is perfect).
PS: If you’re just backing up the remote system and don’t need the files locally, you can make it even faster by just writing that pipe out to a local file instead (ssh … > backup.tar). That removes the slowdown of your local system creating tons of small files, but it also helps preserve attributes and such as well.
“How is it that the poets have said so many fine things about our first love, so few about our later love? Are their first poems their best? or are not those the best which come from their fuller thought, their larger experience, their deeper-rooted affections? The boy’s flute-like voice has its own spring charm; but the man should yield a richer, deeper music.”
― George Eliot, Adam Bede
Jonl sez, “Streets near the University of Texas at Austin today are safer, thanks to quick and effective action by the local police, who caught and arrested a jogger, observed wearing a pony tail, black shorts, and a headset in the area. She was apparently menacing traffic by crossing the street in a manner construed as ‘jaywalking,’ however her apparent arrest was due to her failure to provide identification: i.e., she was guilty not just of jaywalking, but of anonymous jaywalking.”
Are you fucking kidding me, Austin Police Department? This is how you treat the citizens you’re charged with protecting? This was the most important thing you could be doing at this moment?
Gosh, I wonder why people don’t respect police authority.
The Marvell 88SE6141 SATA chip doesn’t work in Linux 3.11. At all.
The Internet is full of people trying to get it working and failing, ever since kernel 2.6. The kernel/ide mailing lists are full of people trying to submit patches to make it work and being told by maintainers that it’s already working as an AHCI device (no, it isn’t). The guys maintaining sata_mv, para_marvell, and ahci are all pointing fingers. No one bothered to make it work. That’s kind of shitty, seeing how long this has been out there.
(Yes, I tried the ahci.marvell_enable trick, and all the others.)