HTTrack Website Copier
Free software offline browser - FORUM
Subject: Workaround for www.foo.com <-> foo.com and //
Author: Xavier Roche
Date: 03/23/2003 16:24
 
Two problems are often encountered with bogus sites:

1- loop problems with multiple // in urls; such as:
www.foo.com/bar/foobar
www.foo.com/bar//foobar
www.foo.com/bar///foobar
..

2- duplicate site copy; such as
www.foo.com/bar/foobar
foo.com/bar/foobar
www-45.foo.com/bar/foobar

3- multidomain duplicate site copy; such as
www.foo.com/bar/foobar
www.foo.net/bar/foobar
www.foo.fr/bar/foobar

I have implemented a workaround for 1. and 2. (maybe 3. in 
the future when I have some time) in the current 3.30-alpha-
05 ; which detects such cases, and merge "similar" links 
together:

These links will be merged with the "urlhack" option:
www.foo.com/bar.html
foo.com/bar.html
www-42.foo.com/bar.html

And these links too:
www.foo.com/bar.html
www.foo.com//bar.html
www.foo.com///bar.html

If you have problems similar to 1. and 2., please test the 
current alpha-05 and give me some feedback, aspecially:
- is the workaround working?- is the workaround causing "bad side effects" on
other 
links/sites?
 
Reply


All articles

Subject Author Date
Workaround for www.foo.com <-> foo.com and //

03/23/2003 16:24
dnecnce rrxctrp

09/25/2013 20:11
pplptoz xmffcbg

09/26/2013 06:30
ewotaoh fnuluba

09/26/2013 18:04
dyizhen hxywube

09/27/2013 03:25
wkhtlft mgazenc

09/27/2013 13:30
rhszzcz iqzdtqt

09/28/2013 01:23
crqhuww jqumbmh

10/02/2013 14:48
oqqwjfn yedpiby

10/03/2013 00:38
qqicphc gllrcqq

10/03/2013 10:19
xtleqcv kyolbni

10/03/2013 20:31
tuxyzgq rnzhthv

10/04/2013 07:01
vbqryne tjjqksf

10/04/2013 07:01
yilehjl oxtngjn

10/04/2013 16:44
rzqjpuo hfreyqy

10/05/2013 03:03
bqpzcig ryjiksp

10/05/2013 03:03
mssdiyn ilqkqqf

10/05/2013 12:30
lwqcoug uthvugk

10/05/2013 12:30
taadoft lmupnkf

10/06/2013 08:21
xjfcors xyesgxg

10/07/2013 18:51
cmuiosz nlurexh

10/08/2013 15:43
awopaao ohecfox

10/09/2013 13:52
vlhblnr fpsfeyn

10/10/2013 11:14
yytzlve jziliyg

10/11/2013 08:52
tvaqety xfbgpkq

10/12/2013 06:51
amgzzfd udfjjbd

10/13/2013 04:21
zgtznba eouhtjs

10/14/2013 01:05
rzaylnl radnzpl

10/14/2013 21:53
dyagqsp ubacewv

10/15/2013 03:16
batascq kgvmaka

10/15/2013 18:49
lswthkn vmryomm

10/21/2013 18:00
rymfshd bmaupcw

10/21/2013 18:00
hlhtlsa ffvcpeo

10/22/2013 03:45
llfnyhc jjvpzio

10/22/2013 03:45
tgyganr gfgigfi

10/22/2013 14:36
tdzwfei zttrfix

10/23/2013 00:45
reuixqv dvafpto

10/23/2013 10:41
gnvtkcd tcjbdrw

10/23/2013 10:41
xaqhuts wdwgucu

10/24/2013 06:35
xeogbpl ilorysz

10/24/2013 06:35
iggidud zahdjbd

10/24/2013 15:59
aaylvks dmqttba

10/24/2013 15:59
kcybccp fncroal

10/25/2013 02:11
bwqevre jymhxai

10/25/2013 02:11
orryuqi tmclxuw

10/25/2013 11:18
nynxhiu rdlhebf

10/25/2013 11:19
lriovyz yimexjl

10/25/2013 21:41
vjheimr ibdijld

10/25/2013 21:41
exgiord gtcwwuw

10/26/2013 07:52
ryqcoxn eqtwocy

10/26/2013 16:57




4

Created with FORUM 2.0.11