Page 1 of 1

Can't index very large zip file.

PostPosted: Tue Jan 12, 2010 1:19 pm
by zhangjinsong
Hi.
I got Britannica Encyclopedia 2009 of dsl format from ru.board.
I compressed all the avi, jpg and wav files into one zip file named a.dsl.files.zip whose size is greater than 3.5GB, while the dsl file named a.dsl.
However, GD can't display any images, and can't play audio files either. The size of the index file in ~/.goldendict is only 2.5MB, much smaller than expected.
But when only compressing fewer files, everything is OK.
I traced the code and found the function File::exists () using the return value of stat () to determine whether a file exists or not, but when the file size is too large, stat () always returns 1.

Sample code:
Code: Select all
int main(int argc, char *argv[])
{
    struct stat buf;
      int exist = 0;
      if (stat( "/opt/goldendict/dictionary/BE/a.dsl.files.zip", &buf ) == 0) {
          exist = 1;
      } else {
          perror ("STAT: ");
      }
      printf("exist = %d", exist);
      return 0;
}


The output is :
Code: Select all
STAT: : Value too large for defined data type
exist = 0


Ubuntu 8.04 + ext3 + kernel 2.6.24 + libc 2.7-10ubuntu5

Thanks.

Re: Can't index very large zip file.

PostPosted: Tue Jan 12, 2010 1:39 pm
by ikm
Thanks -- I didn't know that about stat(). I've committed the fix in GIT.

Re: Can't index very large zip file.

PostPosted: Thu Jan 14, 2010 3:08 pm
by reels
Hi,

I have same problems. For me seems to be better when GD would accept a folder named "dictionary.dsl.files" and all zips placed in that folder (img.zip, sound1.zip, sound2.zip, movie.zip......). For example in Longman dictionary you have more then 200,000 media files and it is a big problem (size, difficulties with zip creation etc.).

Otherwise thank you for the program - excelent!

Milan

Re: Can't index very large zip file.

PostPosted: Fri Jan 15, 2010 2:09 am
by zhangjinsong
reels wrote:Hi,

I have same problems. For me seems to be better when GD would accept a folder named "dictionary.dsl.files" and all zips placed in that folder (img.zip, sound1.zip, sound2.zip, movie.zip......). For example in Longman dictionary you have more then 200,000 media files and it is a big problem (size, difficulties with zip creation etc.).

Otherwise thank you for the program - excelent!

Milan


The problem has been fixed in the latest git version.

A simple shell command to create zip file:
Code: Select all
find . \( -name "*.wav" -o -name "*.jpg" \) -print0 | xargs -0 zip -q filebasename.dsl.files.zip -@