Erasure Coding

  • 20 March 2020
  • 0 replies
  • 9377 views

Userlevel 3
Badge +3

What is Erasure Coding?

Erasure coding increases the usable capacity on a cluster. Instead of replicating data, erasure coding uses a parity information to rebuild data in the event of a disk failure. The capacity savings of erasure coding is in addition to deduplication and compression savings.

 

If you have configured redundancy factor 2, two data copies are maintained. For example, consider a 6-node cluster with 4 data blocks (a b c d). In this example, we start with 4 data blocks (a b c d) configured with redundancy factor 2. In the following image, the white text represents the data blocks and the green text represents the copies.

 

Data copies before Erasure Coding

 

Computing Parity  
Data copies after Computation of Parity

 

Erasure Coding Best Practices and Requirements:

  • A cluster must have at least four nodes populated with each storage tier (SSD/HDD) represented to enable erasure coding.
  • Avoid strips greater than (4, 1) because capacity savings provide diminishing returns and the larger strip size increases the cost of rebuild.
  • Erasure coding effectiveness (data reduction savings) might be reduced on workloads that have many overwrites outside of the erasure coding window, which by default is 7 days.
  • Read performance can degrade during failure scenarios.

Click here for the full list of best practices and requirements.

Find the full documentation of Erasure Coding here (Architecture, examples for RF2 & RF3 and more).


This topic has been closed for comments