HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Writing to ARC format
Author: Lars Clausen
Date: 11/18/2003 14:40
 
> > > - 'receive-header'
> > That one isn't included by default, but only if 
> HTS_ANALYSTE
> > is != 0 -- but setting that kills main().
> 
> You do NOT have to recompile httrack to plug callbacks!!
> The only thing to compile is the very small .c file you 
> will make using the example provided!

Well, it won't accept the receive-header callback unless the
below patch is applied.  In the default source, the
'receive-headers' callback isn't registered unless
HTS_ANALYSTE is 1.

--- httrack.c~  2003-10-11 09:03:51.000000000 +0200
+++ httrack.c   2003-11-17 14:22:13.000000000 +0100
@@ -190,6 +190,8 @@
   htswrap_add("link-detected",htsshow_linkdetected);
   htswrap_add("transfer-status",htsshow_xfrstatus);
   htswrap_add("save-name",htsshow_savename);
+  //  htswrap_add("send-header", htsshow_sendheader);
+  htswrap_add("receive-header", htsshow_receiveheader);
 
   ret = hts_main(argc,argv);
   if (ret) {
@@ -579,6 +581,10 @@
   return 1;
 }
 
+int __cdecl htsshow_receiveheader(char *buff, char*
adr_complete,char* fil_complete,char* referer_adr,char*
referer_fil, htsblk* incoming) {
+  return 1;
+}
+
 
 /* *** Various functions *** */
 


> > How can I get the IP corresponding to the address?> 
> This is not possible yet - but I will add two members to 
> the htsblk structure:
>   SOCaddr address;       // IP address
>   int     address_size;  // IP address structure length
> 
> SOCaddr can be either sockaddr_in, sockaddr_in6, or 
> sockaddr depending on the size

Sounds good!  In the meanwhile I'm using hts_gethostbyname,
but it seems heavyweight.

> > Where's the actual content passed to transfer-status?> 
> - If the 'adr' member of the htsblk structure is != NULL, 
> adr is the address of the buffer and 'size' its size (the 
> htsblk structure is the r member of the lien_back 
> structure - that is, back->r.adr and back->r.size)
> - Else, the 'url_sav' member of the lien_back structure is 
> the destination file of the stored URL (you'll have to do a 
> fopen(r->url_sav, 'rb') to fetch the file)

Good, that seems to work.  Haven't had to deal with a file
yet, that'd be a bit nastier, as I have to do MD5 on
headers+body before writing them down.

> > Is the size field in lien_back the one sent by the server?> 
> No - this one is 'totalsize'. 'size' is the size 'seen' 
> (downloaded) by httrack.

Ok.  Not that the two numbers differ frequently, but they do
on occasion.

I will post the code soon, so other interested parties can
take a look.

-Lars
 
Reply Create subthread


All articles

Subject Author Date
Re: Writing to ARC format

11/17/2003 23:01
Re: Writing to ARC format

11/18/2003 14:40
Re: Writing to ARC format

11/18/2003 17:21
Re: Writing to ARC format

11/18/2003 20:27
Re: Writing to ARC format

11/18/2003 20:39




4

Created with FORUM 2.0.11