Sony ARW "compressed RAW", binning, and noise reduction - Forums

Topic starter

Nov 7, 2021

I've been using a Sony a7iii for a few months now. 24MP (4Kx6K), full frame, so pixels are 6umx6um, if I've calculated right. This seems to be a good match for my Redcat 51, giving 4.94 arcsec/pixel. The field is flat and bright except for vignetting in the corners that is easily handled by flat frames, so I think the optics and sensor are an excellent match. (And it's a huge improvement over a several-year-old Canon DSLR I was using before.)

So, the question…I just realized that RAW files are 24MB, pretty much exactly 1 byte/pixel. But the ADC is 14 bit. So where did the missing bits go? I checked into the menus, and there is an option under "file format" for "compressed RAW", which is checked. What the heck is compressed RAW?!? Is this DCT JPEG, or RLE, or something else? Should I be unchecking this? (I did some googling before asking, of course, but had trouble finding an exact description of what's going on.)

Noise is a big problem because I'm using short subs due to tracking problems (separate post about that).

I have noise reduction turned *OFF* in the camera, preferring to leave that to the stacking process. (I use AstroPixel Processor.) I assume that's right?

The camera seems to have no 2x2 binning option. I have never used it in APP, but am considering it. At full frame that would still give me 2kx3k, should be nice photos if I don't want to crop/expand too much. Any comments?

Helpful Insightful Respectful Engaging

Rod Van Meter

Topic starter

Nov 8, 2021

Partially answering my own question:

Reply from user "Monochrome" here seems to have useful info. It's lossy, and loses stuff at the low end, which of course is where I want it <headdesk>. It never even OCCURRED to me that RAW ever included lossy compression.
https://t.co/sIUeIBy1vt

andrea tasselli

Frequent contributor

Nov 8, 2021

Lossy compression file format and noise reduction in-camera should be ALL off. No DSLR supports binning, afaik . You'd have to do that during post-processing.

Rod Van Meter

Topic starter

Nov 9, 2021

Yeah, I'm thinking about binning in post-processing, not in the DSLR/MILC. But I'm finding suggestions on APP forums that propose drizzling and interpolation, but I don't yet understand how to use those.

Yes, definitely turning off compression going forward. I couldn't *believe* they would *really* have lossy compression in the RAW files, but apparently it's true.

In-camera noise reduction turned off is the right setting?

Die Launische Diva

Nov 9, 2021

Any kind of data manipulation in-camera is undesirable because it potentially invalidates any attempt to manually calibrate the data. Suppose you want to subtract a dark from a light frame. If both are compressed and denoised, they aren't compatible any more. For example, subtracting a dark frame from a light frame to remove the so-called dark current noise (DCN) and fixed-pattern (FPN) noise won't work as desired. For the astronomer, this dark current is *signal* in a dark frame, while it is *noise* in a light frame. But we don't know how the denoising algorithm will treat those images. Who knows, maybe the algorithm may remove some of the DCN and FPN in a light frame, and remove all of them in a dark frame. But then, the resulting images will be incompatible.

Well written Helpful Insightful Concise

Rod Van Meter

Topic starter

Nov 9, 2021

Thanks, that confirms what I thought.

I wonder how much of an effect this is having on the residual noise in the images I'm trying to process now. I suspect a moderate amount.

Well written

Jurgen Doreleijers

May 4, 2022

Be happy your camera can turn the lossy compression off. Mine, the a7S (first model from 2014) can't…

David Moore

May 4, 2022

I have this camera and like it a lot though I migrated to astro cameras. Yes compressed RAW should be off. Noise reduction >1 sec imaging will double exposure time as it happens for every shot. I haven't analysed how good it is. I set mine up to blank the rear screen before a shot and to enable high ISO to check on focus.

Perfect post

May 4, 2022

I have used the A7III For AP for over 2 years now, and yes, turn the compressed raw off. it can damage detail and create artifacts in the images. The decompressed RAW is much better, however it is MUCH larger. As for binning, at 4.94"/PX you should not bin, you'd just be losing detail in exchange for sensitivity, which should be no issue as the dynamic range of the 7III Stays nearly constant well up to ISO 1600.

Helpful Concise

James Peirce

May 10, 2022

Adding to the above, yeah, make sure "Uncompressed RAW" is chosen for that format. Sony's RAW compression does a lot of damage to photographs which need to be stressed heavily for signal, which is at the heart of astrophotography (that's likely true of all lossy compression as a rule). It also prevents calibration frames from working properly.

Another thing to check is to make sure "Lens Shading Comp," or whatever it is in your camera, is disabled. All lens corrections should be disabled. That particular feature, on Sony cameras, edits the RAW files in a manner which tries to diminish vignetting, and it creates awful ringing artifacts (and also prevents calibration frames from working properly). And turn off the other noise reduction options. Sony, unfortunately, doesn't do an especially good job of respecting RAW file integrity.

Well written Helpful Insightful Concise

Andy Wray

May 10, 2022

I used an old Sony NEX 5N in ARW mode (no compression, no noise reduction and no lense compensation) a few years back. The best I could achieve of M31 as an example was:

M31 taken with an old Sony NEX 5N camera

Related discussions

Help Sony a7III Interval shooting mode

Hi everyone ! Last week i've been imaging NGC6960 and had a weird problem with the internal interval shooting mode of my a7III. Every now and then (sometimes after less than 5 pictures, sometimes never without changing any settings) the camera st...

Sony a7iii camera settings and technical specifications relevant to interval shooting issues.

Apr 26, 2020