www.ice-graphics.com Forum Index www.ice-graphics.com
The main forum for the ICE-Graphics software
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

ICEECC: Meaning of source and recovery block counts?

 
Post new topic   Reply to topic    www.ice-graphics.com Forum Index -> General
View previous topic :: View next topic  
Author Message
Samurai



Joined: 10 Jun 2010
Posts: 11

PostPosted: Fri Apr 29, 2011 12:13 pm    Post subject: ICEECC: Meaning of source and recovery block counts? Reply with quote

I have read the ICE ECC help file as well as the excellent tooltips in the user interface. I can also understand what adding more source blocks or more recovery blocks affect on the recovery file size and compute time, as well as what the redundancy % means in terms of those block counts. What I still do not understand clearly, is what is the real-life meaning of the block counts in terms of recover ability. Let me explain by spelling out a few questions showing my ignorance.

I tried to find an answer to these questions by using Google, but was not able to find an answer. It seems to me that there are only very few persons who really understand what the source block count and recovery block count mean in terms of recovery.

Let's suppose I have one data file with a size of 1GB (and lets say for simplicity that 1 GB = 1000 MB and 1 MB = 1000 kB). If I select 1000 source blocks and 100 recovery blocks, I will have a redundancy of 10% and size of recovery file is 100 MB.

- Does the 100 recovery blocks mean here that there can be some kind of corruption at most in 100 of the 1000 source blocks in the file if I wish to be able to recover it (i.e. at least 900 of the 1000 consecutive 1 MB source blocks in the file must be intact)?

- Does it also mean that the corruption degree in any of the corrupted source blocks must not exceed 10% (i.e. at least 900 kB of each source block bits - or bytes - must be correct)?

- Can the max. 100 corrupted source blocks be anywhere in the file (e.g. all the first 100 blocks, or every third block starting from block #500 etc)?

If my reasoning above is incorrect, could you please tell how they sould be corrected?

Again, many thanks for developing ICE ECC. It is a great product!
Back to top
View user's profile Send private message
Cynic



Joined: 01 May 2011
Posts: 3

PostPosted: Sun May 01, 2011 9:24 pm    Post subject: Re: ICEECC: Meaning of source and recovery block counts? Reply with quote

I am not an expert, but I'll answer according to my current understanding and if I'm wrong hopefully the author will correct me.


Samurai wrote:
- Does the 100 recovery blocks mean here that there can be some kind of corruption at most in 100 of the 1000 source blocks in the file if I wish to be able to recover it (i.e. at least 900 of the 1000 consecutive 1 MB source blocks in the file must be intact)?

Yes.


Samurai wrote:
- Does it also mean that the corruption degree in any of the corrupted source blocks must not exceed 10% (i.e. at least 900 kB of each source block bits - or bytes - must be correct)?

No, each individual block can be 100% corrupted. It's "yes or no", a single wrong byte is enough to make a block useless, that's why it's a good idea to select a block size as small as possible (within reasonable computation times).

I think blocks can even be missing entirely (i.e., the total filesize of the corrupted file(s) may be less than the original uncorrupted set) but I'm not sure about that.


Samurai wrote:
- Can the max. 100 corrupted source blocks be anywhere in the file (e.g. all the first 100 blocks, or every third block starting from block #500 etc)?

Yes.


I reccommend putting a bunch of different files in a test folder, creating ECC files, making a copy of the test folder (so you don't have to recompute the ECC files for every experiment), trying various combinations of deleting/renaming/modifying/corrupting the original and/or the ECC files and then attempting recovery to get an idea of what the program can/can't do.
Back to top
View user's profile Send private message
Cynic



Joined: 01 May 2011
Posts: 3

PostPosted: Sun May 01, 2011 9:49 pm    Post subject: Reply with quote

To expand a bit on your second question: starting from your example, let's suppose that you picked a redundancy of 10% but with a block size of 50MB instead (20 total blocks and only 2 recovery blocks) and that your 1GB file gets corrupted in just 3 different places.
If you're lucky, all 3 errors fall inside the same or at most 2 different blocks, and you can recover your file with no problems; if you're unlucky, each of the errors falls on a discrete block and you can't recover the file because you only have 2 recovery blocks. The errors could be a single byte each and yet you still couldn't recover anything, even though you have 100MB of ECC files.

You can also be extremely unlucky and get only 2 errors, but one of them falls in a really bad place and corrupts the end of one block and the beginning of the next, and the other falls in a completely different block from the former contiguous two. Once again, no recovery is possible.
Back to top
View user's profile Send private message
ICE Graphics
Site Admin


Joined: 31 Mar 2003
Posts: 430

PostPosted: Mon May 02, 2011 7:21 am    Post subject: Reply with quote

Cynic wrote:
The errors could be a single byte each and yet you still couldn't recover anything, even though you have 100MB of ECC files.

Yes

Cynic wrote:
You can also be extremely unlucky and get only 2 errors, but one of them falls in a really bad place and corrupts the end of one block and the beginning of the next, and the other falls in a completely different block from the former contiguous two. Once again, no recovery is possible.

There is not speccial bad place witch can corrupts the end of one block and the beginning of the next. Blocks are not overlapped.
Back to top
View user's profile Send private message Visit poster's website
Samurai



Joined: 10 Jun 2010
Posts: 11

PostPosted: Mon May 02, 2011 7:28 am    Post subject: Reply with quote

Thank you for your explanations, Cynic & Site Admin! This made the meaning of the source and recovery blocks in terms of recoverability clear.

BTW my experience has shown that if the file size changes in the corruption, then a recovery is not possible.
Back to top
View user's profile Send private message
Cynic



Joined: 01 May 2011
Posts: 3

PostPosted: Mon May 02, 2011 4:07 pm    Post subject: Reply with quote

ICE Graphics wrote:
There is not speccial bad place witch can corrupts the end of one block and the beginning of the next. Blocks are not overlapped.


I did not explain myself clearly.

What I meant was: suppose there is a big multi-byte error (for example 2048 B, an unreadable DVD sector) of corrupted data, and that the block size is 300000 B.

If that single error of 2048 B consecutive bytes falls 200 bytes at the end of one block, and 1848 bytes at the beginning of the next, it would effectively count as 2 errors. Or is there some reason why this isn't possible at all?
Back to top
View user's profile Send private message
ICE Graphics
Site Admin


Joined: 31 Mar 2003
Posts: 430

PostPosted: Tue May 03, 2011 5:59 am    Post subject: Reply with quote

Samurai wrote:
BTW my experience has shown that if the file size changes in the corruption, then a recovery is not possible.

It's possible.
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    www.ice-graphics.com Forum Index -> General All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group