I have had a QHY268M since June of 2021. I initially used Mode 1 (High Gain) exclusively, with Gain 56 and Offset 10, in order to get the lowest possible read noise. It is a good choice with narrowband imaging, like H-alpha shots of nebulae.
I have been playing around with Mode 3 (Extended Fullwell 2CMS) lately, shooting LRGB images of small, faint galaxies with a 100 mm refractor, a tough test for any camera. Small galaxy images often suffer from overexposed stars because of shooting longer luminance subs to capture the faint targets. Mode 3 with Gain 14 provides a full well depth that is approximately four times greater than Mode 1, Gain 56, meaning that fewer stars become overexposed. The cost is higher read noise.
I have been pleased with the performance of Mode 3 for luminance imaging. Although the read noise is higher, I have generally found that it is swamped by sky noise for 5-minute luminance exposures, so it has very little effect. The number of "burned out" star cores is definitely reduced in a direct comparison of Mode 3 vs. Mode 1.
For a more detailed discussion of the modes and some quantitative analysis of the camera's performance in different modes and various Gain settings, please visit this extensive discussion on Cloudy Nights.
Cloudy Nights: Received my QHY268M