www.ice-graphics.com Forum Index www.ice-graphics.com
The main forum for the ICE-Graphics software
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

ICE ICC : Check the whole set if not enough blocks

 
Post new topic   Reply to topic    www.ice-graphics.com Forum Index -> Bugs and Problems
View previous topic :: View next topic  
Author Message
bobby89



Joined: 20 May 2005
Posts: 9

PostPosted: Sat May 21, 2005 4:30 pm    Post subject: ICE ICC : Check the whole set if not enough blocks Reply with quote

Hi,


I've been testing your program and when you check the file with an ECC file, if something is corrupted and there is not enough ECC files to recover, your program is checking the corrupted file and it last as much time as the first check. i.e:

49 RAR, 15Mb each (700Mb the whole set). To check all files it takes 2min 40sec. Say RAR 15 is corrupted, at the end at the check of RAR 49 and if there is not enough blocks to recover, ICE ICC check RAR 15 again and it last around 2min 40sec!!!

There is no need to check this file again, it has already been and ICE ICC discovered how much blocks were corrupted. This is a bug.

Another abnormal behaviour is when there is not enough blocks to recover, if you add ECC files, ICE ICC is going to check the whole set again!! This is a serious error, because the program should remember what file is good and what file is not. QuickPar remember that. So again you have to spend 2min 40sec to recheck all files and then the recover process may begin.

Of course I take an example that lasts 2min 40sec on my AMD64 - 4000+, people using 2000+ or Intel 2Ghz and check not a 700Mb CD but a entire DVD set of 4,2Gb or even dual-layer, a full recheck is not good.

Those 2 bugs are very serious and to me I still stay with QuickPar until fixed.

Your program is musch faster than QuickPar to recover file, you make a real boost!! Cheers on that point. Smile

However the checking process is longer than QuickPar. Around twice much time to check 700Mb with ICE ICC than with QuickPar, I'm amazed!! Also why the creation process takes as much time as QuickPar? To recover files is incredibly fast and to create the blocks (ECC files) takes as much time.

Your program may be widely used (and I would bethe first) ONLY if those 2 bugs are fixed and the checking & creation speed enhanced. Otherwise, everybody ask the question: why to use ICE ICC instead of QuickPar? Ok ICE ICC is much faster to recover but if we make counts, QuickPar is the winner because it is faster to check files, we know how to use it, it is widely used. Why learn another tool that works as quick?

You can surpass QuickPar, I hope you'll do it. Smile

Cheers for your work and Best Regards.


Bob
Back to top
View user's profile Send private message
ICE Graphics
Site Admin


Joined: 31 Mar 2003
Posts: 430

PostPosted: Thu May 26, 2005 7:53 am    Post subject: Re: ICE ICC : Check the whole set if not enough blocks Reply with quote

bobby89 wrote:
I've been testing your program and when you check the file with an ECC file, if something is corrupted and there is not enough ECC files to recover, your program is checking the corrupted file and it last as much time as the first check. i.e:

49 RAR, 15Mb each (700Mb the whole set). To check all files it takes 2min 40sec. Say RAR 15 is corrupted, at the end at the check of RAR 49 and if there is not enough blocks to recover, ICE ICC check RAR 15 again and it last around 2min 40sec!!!

There is no need to check this file again, it has already been and ICE ICC discovered how much blocks were corrupted. This is a bug.

If not enought ECC block for recover, ICE ECC try to find shifted blocks. In version v1.0 search of shifted blocks was slow. Try v1.1. It do search of shifted blocks in 100 times faster.


bobby89 wrote:
Another abnormal behaviour is when there is not enough blocks to recover, if you add ECC files, ICE ICC is going to check the whole set again!! This is a serious error, because the program should remember what file is good and what file is not. QuickPar remember that. So again you have to spend 2min 40sec to recheck all files and then the recover process may begin.

ICE ECC also is remember it. But there is another reason. ICE ECC do not suggest, what after good block 125 have to be block 126. It check whole file. And ai am not sure what it's bad solution.


bobby89 wrote:
Of course I take an example that lasts 2min 40sec on my AMD64 - 4000+, people using 2000+ or Intel 2Ghz and check not a 700Mb CD but a entire DVD set of 4,2Gb or even dual-layer, a full recheck is not good.

Those 2 bugs are very serious and to me I still stay with QuickPar until fixed.

I hope v1.1 is free from this is bugs


bobby89 wrote:
Your program is musch faster than QuickPar to recover file, you make a real boost!! Cheers on that point. Smile

However the checking process is longer than QuickPar. Around twice much time to check 700Mb with ICE ICC than with QuickPar, I'm amazed!! Also why the creation process takes as much time as QuickPar? To recover files is incredibly fast and to create the blocks (ECC files) takes as much time.

ICE ECC v1.0 use very slow CRC algorithm. Fixed. Now ICE ECC v1.1 faster than QuickPar in any modes.


bobby89 wrote:
Your program may be widely used (and I would bethe first) ONLY if those 2 bugs are fixed and the checking & creation speed enhanced. Otherwise, everybody ask the question: why to use ICE ICC instead of QuickPar? Ok ICE ICC is much faster to recover but if we make counts, QuickPar is the winner because it is faster to check files, we know how to use it, it is widely used. Why learn another tool that works as quick?

You can surpass QuickPar, I hope you'll do it. Smile

Just check again Smile
Back to top
View user's profile Send private message Visit poster's website
bobby89



Joined: 20 May 2005
Posts: 9

PostPosted: Thu May 26, 2005 7:29 pm    Post subject: Reply with quote

Hi,


Thanks for your answer.

I've been trying ICE ECC 1.1 and this version is as fast as QuickPar on creation and verification process, even faster but just a bit. I've just made a check comparaison, I got 1min51 with QuickPAR to check and ICE ECC 1min 45.

Also when not enough blocks are available, the check for shifted blocks are really fast now. Good point!!!

The creation process is still faster than QuickPar.

And you have the subdirectories function. Very good!!!

What about usenet:

- why use your program and give up QuickPar? At that point some people ask for Linux, Mac, Unix compatibility. When I post PAR2 on usenet, everybody right now can use it, if I post ECC, only windows users could enjoy it. I would like to use your ECC files on usenet but I hesitate. Your program is faster but maybe not enough to force people on usenet to switch to another program.

- why didn't you make a PAR2 compatibility? I mean why to use another extension and not create a competitor program with still the same PAR2 extension? It would certainly help to spread your program.

- do you plan in the near future to use PAR3 algorythm? I read firsts tests showing that PAR3 will be hundred times faster. PAR3 is planned to be released in the next QuickPar.

We need competitors like you Smile Thanks you so much for everything!! Smile
Back to top
View user's profile Send private message
ICE Graphics
Site Admin


Joined: 31 Mar 2003
Posts: 430

PostPosted: Fri May 27, 2005 8:25 am    Post subject: Reply with quote

bobby89 wrote:
What about usenet:

- why use your program and give up QuickPar?

There are two cases where PAR2 fail:
1. Protecting large set of files. More than 1000. Like collection of photos. Where number of blocks less than quantity of files.
2. Protecting files with subdirectories.


bobby89 wrote:
At that point some people ask for Linux, Mac, Unix compatibility. When I post PAR2 on usenet, everybody right now can use it, if I post ECC, only windows users could enjoy it. I would like to use your ECC files on usenet but I hesitate. Your program is faster but maybe not enough to force people on usenet to switch to another program.

Almost all systems allow to emulate Windows application: Virtual PC for MAC, Wine, VMWare for Linux, etc...

We do not plan to port ICE ECC on other system. It's hard task. Because ICE ECC use a lot of internal Windows features.


bobby89 wrote:
- why didn't you make a PAR2 compatibility? I mean why to use another extension and not create a competitor program with still the same PAR2 extension? It would certainly help to spread your program.

This is impossible. PAR2 structure has serious errors witch is impossible to fix. I think is possible to add subdirectories support in PAR2. But is impossible to add virtual blocks in PAR2. I mean case when number of files large than block count. Anybody who will create a new format with subdirectory and virtual blocks support will make its incompatible with existing PAR2.


bobby89 wrote:
- do you plan in the near future to use PAR3 algorythm?

I know nothing about new PAR3 algoritm. Can you give me some links, where i can find some usefull info ?

bobby89 wrote:
I read firsts tests showing that PAR3 will be hundred times faster. PAR3 is planned to be released in the next QuickPar.

1. To make faster Reed-Solomon implementation than ICE ECC is almost impossible task.

2. There are different ways for building Read-Solomon codes. All known algorithms do not allow to make faster implementation than exist now. The main problem: any bytes in ECC have to be interfere with any bytes in data file. If you have 1024 blocks + 128 blocks ECC, it's mean, what for calculation 1 byte in ECC you have to interfere it with byte from every block: 1024 times. Total operations count for ECC calculation are: 1024x128. I do not know faster solutions.

3. There is a class of linear ECC codes. It's faster Read-Solomon codes in hundred times. But there are some problems with them:

A. Read-Solomon code guarantee, what if there are 5 ECC blocks, you can recover 5 broken blocks anywhere. Linear codes can not do what. Linear codes tell what they can work if average error ratios less than some value. However, linear codes can not guarantee it !!! It's normal situation for linear codes, when 1 MB of ECC data can not recover 512KB of damaged info. Even if damaged info lost as single block !!!

B. Linear codes work well only if error ratio is low. If you try to make linear codes for large data size (gigabytes) with redundance more 25%, it will make linear codes slow almost like Reed-Solomon codes.

C. For calculation linear codes for large data set need to create large graph. It's not simple and slow process. Required a lot of RAM. I am not sure, what it's possible to calculate it for reasonable time for data size 4.7GB.

D. Linear codes absolutely incompatible with Reed-Solomon codes. It's mean whan new PAR3 format using linear codes will be absolutely incompatible with PAR2.
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    www.ice-graphics.com Forum Index -> Bugs and Problems All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group