I can't be 100% certain without more details about what the system was doing at this time, but I can take a good guess. Any drive failure is an extremely high priority event and Curator process will be started to replicate the data. This is an high priority event and will take away CPU cycles. This will impact IOPs load. However, performance delta due to this HDD failure with no data to replicate should be very short lived.
Performance impact should be only on writes. Concern on why it goes down is related with burst vs. sustained write performance due to write cache thresholds.
This is not specific to Nutanix. Any storage system will have limited write cache space and will have higher burst write performance but a lot lower sustained write performance. In real world, write workloads are bursty but for short durations. Benchmark tests are more aggressive than real world workloads and tend to hit one of the cache thresholds.