Joe's Amazing Technicolor Weblog » httrack http://slagwerks.com/blog Fri, 23 Jul 2010 22:31:13 +0000 en hourly 1 http://wordpress.org/?v=3.0 HTTrack: new go-to program for web mirroring / archiving http://slagwerks.com/blog/index.php/2009/04/02/httrack-new-go-to-program-for-web-mirroring-archiving/ http://slagwerks.com/blog/index.php/2009/04/02/httrack-new-go-to-program-for-web-mirroring-archiving/#comments Thu, 02 Apr 2009 19:10:44 +0000 joe http://slagwerks.com/blog/?p=265 Faced with a big site full of URLs like http://mysite.com/Internal1.asp?id=357 to mirror & archive, I recently tried out a new (to me) tool, HTTrack. I’ve fiddled with wget for this sort of job in the past, but it always takes me ages of man-page reading to get my options right, and even then not everything seems to work out.

This time around, for example, I’d convinced myself that wget -r -N -l inf --no-remove-listing -E -k -p http://mysite.com would do the trick. It mostly did, except for seemingly random pages that didn’t get all of their links converted.

HTTrack, on the other hand, did The Right Thing without any switches or arguments whatsoever. It was a bit more of a pain to get running; even though it’s in macports, right now the port is lagging behind the available versions, so I had to actually type ./configure and ./make myself. Well worth it for a usable mirror.

]]> http://slagwerks.com/blog/index.php/2009/04/02/httrack-new-go-to-program-for-web-mirroring-archiving/feed/ 0