New user registration is currently disabled due to spam abuse / Регистрация новых пользователей в настоящее время приостановлена из-за злоупотреблений спаммерами

Can't index very large zip file.

Report bugs here

Can't index very large zip file.

Postby zhangjinsong » Tue Jan 12, 2010 1:19 pm

Hi.
I got Britannica Encyclopedia 2009 of dsl format from ru.board.
I compressed all the avi, jpg and wav files into one zip file named a.dsl.files.zip whose size is greater than 3.5GB, while the dsl file named a.dsl.
However, GD can't display any images, and can't play audio files either. The size of the index file in ~/.goldendict is only 2.5MB, much smaller than expected.
But when only compressing fewer files, everything is OK.
I traced the code and found the function File::exists () using the return value of stat () to determine whether a file exists or not, but when the file size is too large, stat () always returns 1.

Sample code:
Code: Select all
int main(int argc, char *argv[])
{
    struct stat buf;
      int exist = 0;
      if (stat( "/opt/goldendict/dictionary/BE/a.dsl.files.zip", &buf ) == 0) {
          exist = 1;
      } else {
          perror ("STAT: ");
      }
      printf("exist = %d", exist);
      return 0;
}


The output is :
Code: Select all
STAT: : Value too large for defined data type
exist = 0


Ubuntu 8.04 + ext3 + kernel 2.6.24 + libc 2.7-10ubuntu5

Thanks.
zhangjinsong
 
Posts: 28
Joined: Sat Oct 31, 2009 7:05 am

Re: Can't index very large zip file.

Postby ikm » Tue Jan 12, 2010 1:39 pm

Thanks -- I didn't know that about stat(). I've committed the fix in GIT.
ikm
Автор GoldenDict
 
Posts: 1595
Joined: Wed Feb 04, 2009 10:40 am

Re: Can't index very large zip file.

Postby reels » Thu Jan 14, 2010 3:08 pm

Hi,

I have same problems. For me seems to be better when GD would accept a folder named "dictionary.dsl.files" and all zips placed in that folder (img.zip, sound1.zip, sound2.zip, movie.zip......). For example in Longman dictionary you have more then 200,000 media files and it is a big problem (size, difficulties with zip creation etc.).

Otherwise thank you for the program - excelent!

Milan
reels
 
Posts: 1
Joined: Thu Jan 14, 2010 12:16 pm

Re: Can't index very large zip file.

Postby zhangjinsong » Fri Jan 15, 2010 2:09 am

reels wrote:Hi,

I have same problems. For me seems to be better when GD would accept a folder named "dictionary.dsl.files" and all zips placed in that folder (img.zip, sound1.zip, sound2.zip, movie.zip......). For example in Longman dictionary you have more then 200,000 media files and it is a big problem (size, difficulties with zip creation etc.).

Otherwise thank you for the program - excelent!

Milan


The problem has been fixed in the latest git version.

A simple shell command to create zip file:
Code: Select all
find . \( -name "*.wav" -o -name "*.jpg" \) -print0 | xargs -0 zip -q filebasename.dsl.files.zip -@
zhangjinsong
 
Posts: 28
Joined: Sat Oct 31, 2009 7:05 am


Return to Bugs

Who is online

Users browsing this forum: No registered users and 12 guests