Dual narrowband imaging issue

What exposure time are you using with the filter? For a 3nm filter, try 10s. Looks like the EAF is focusing on green channel and not getting it right - maybe HFD calculation not accurate due to low exposure times. What do the individual Blue/Red channels look like?

If you are using a relatively fast ed refractor you're out of luck and you got to compromise with manual focusing. Shooting at f/7 I never had issues and so with lenses. This said I find it puzzling it gives that amount of defocus. It shouldn't. Unless the refractor is really poorly corrected.

Alexey Korobeynikov:
I use small refractor f/4.5 - Sharpstar 61. 
If you're right, then it's only suitable for monochrome imaging...

*Yes, but it really shouldn't. The issue is, or would be, with the refractor. I'm using this sort of filter on a f/2.8 lens of similar FL and there isn't any issue, nor there should be.

Even green looks slightly out of focus to me. Blue looks in best focus. I have use 4nm Altair filters with my Askar SQA70 at 4.8 and and WO GT81 IV at 4.72 with reducer and never seen anything like this.

Edit:

CFA0- G
CFA1- R
CFA2- B
CFA3- G

I recently bought a  Player One Ares C-Pro (IMX 533 sensor) and a Player one filter housing.  The camera has a back focus of 55mm but I was surprised to see that Player One recommended 56mm if a filter was used and they supplied an appropriate spacer.  A quick check on=line showed that there is a lot of information to confirm this so I set my optics up with a 56mm back focus and it woks perfectly with an Optolong L-Pro or an L-eNhance filter.  Maybe you could give this a go.

Autofocus step size 50 and it works fine for me. 
Also I think that if the problem was in the back focus, it would manifest itself in all channels and to a greater extent in the corners of the frame.
That's why Andrea's idea seemed interesting to me. 
And here's another reason:

I have two lenses for my telescope because after the first tests on a monochrome camera with ZWO filters after buying the scope , I got halos around bright stars in the blue channel. 
Knowing that this is typical for early versions of edph II, I wrote to the Sharpstar support and they sent me a new lens (not for free).
After installing the lens and testing it, I realized that the problem remained. After that, I switched to Antlia filters and everything fell into place.
I left the "new" lens on the scope and continued to use it in combination with the monochrome camera.
But yesterday's discussions prompted me to check the filter offset settings in NINA (for monochrome camera), and it turned out that the red channel was shifted much more than the others.
Typically I use my telescope with a monochrome camera with IMX533 sensor. And I just didn't pay attention to it before.

It was a clear night yesterday and I replaced the old lens and and I ran tests on a monochrome camera. I set up the offset filter again and the result surprised me.
There's a good chance that the "new" lens wasn't suitable for the telescope, and I'm looking forward to testing old lens with a color camera.

"new" lens with issues:


old lens:

Hi Alexey.

I've had similar issue toy yours and also suspected that autofocus struggled because of the dual band filter. It turned out, that the problem was in a focuser sag. After each exposure stars got softer and softer. I also did not notice this before stacking. I recommend checking whether this is the case.

For me only a simple fix was needed. I've set  backlash compensation in NINA to 0/150 from 150/150  (IN/OUT). It helped to "lock" the focuser with EAF motor and it works perfectly ever since.

I use overshoot compensation algorithm with 0/250.

My focuser - ZWO EAF.

Usually the left side of the curve is more gentle than the right.

Alexey Korobeynikov:
Autofocus step size 50 and it works fine for me. 
Also I think that if the problem was in the back focus, it would manifest itself in all channels and to a greater extent in the corners of the frame.
That's why Andrea's idea seemed interesting to me. 
And here's another reason:

I have two lenses for my telescope because after the first tests on a monochrome camera with ZWO filters after buying the scope , I got halos around bright stars in the blue channel. 
Knowing that this is typical for early versions of edph II, I wrote to the Sharpstar support and they sent me a new lens (not for free).
After installing the lens and testing it, I realized that the problem remained. After that, I switched to Antlia filters and everything fell into place.
I left the "new" lens on the scope and continued to use it in combination with the monochrome camera.
But yesterday's discussions prompted me to check the filter offset settings in NINA (for monochrome camera), and it turned out that the red channel was shifted much more than the others.
Typically I use my telescope with a monochrome camera with IMX533 sensor. And I just didn't pay attention to it before.

It was a clear night yesterday and I replaced the old lens and and I ran tests on a monochrome camera. I set up the offset filter again and the result surprised me.
There's a good chance that the "new" lens wasn't suitable for the telescope, and I'm looking forward to testing old lens with a color camera.

"new" lens with issues:


old lens:

How flat is the focus curve? Depending on the focuser model and how it is connected to the scope, a step size of 50 is pretty small.

Alexey Korobeynikov:
edph II,

If your EAF is connected to the fine focus knob, this divides the EAF's 2.812 μm step size by the fine focuser reduction amount. I couldn't find the exact reduction amount, but it appears to be 2.812/5, 2.812/7, or 2.812/10. This will result in a small step size. Try some focus runs with the step size set at 200.  If the V is too steep, reduce the step size; if the V is too flat, raise the number. 

The angle of the V-curve should be about the same on each side after a few focus runs. The fact that your curve is usually more gentle on one side or the V-curve is not similar between focus runs, leading me to suspect that the fine focus is slipping. 

Once I stopped attaching my autofocus to the fine focus, my focus runs became uniform, and focus offsets worked correctly. 

If the fine focus is slipping, filter offsets will not work correctly because the focuser will not consistently move to the correct offset position. 

It is pretty easy to tighten the fine focus if it is slipping, but even better to attach the EAF to the fast focus. But when you attach the EAF to the fast focus, make sure you set the end points correctly.

Thanks for the recommendations, 
My EAF is connected to the focuser on the side of the fast focus knob.
I'll try to play with the focus step values when I get the chance, maybe next weekend.

Alexey Korobeynikov:

But when you attach the EAF to the fast focus, make sure you set the end points correctly.

I didn't quite understand about the endpoints
is this referring to the allowable EAF movement? (65,000)

Yes, the allowable movement. You want to ensure that the permissible movement is set so that the EAF stops before causing any damage.

I use a dual band filter and a OSC camera with a redcat 91. So the setup is similar'ish. Using an Oasis Focuser Rose auto focuser (which incidentally I think is superior to the ZWO EAF). 
Technically it is always a tradeoff. You can only ever be perfectly focussed at one wavelength but these wavelengths you (we) use do not differ enough to be a problem. NINA uses the overall star profile to determine if you are in focus or not. If it is in focus at Ha it should be in focus at OIII (with both focus points being ever so slightly off from perfect for both wavelengths.) I'd be suspicious of the filter itself. Also, I am using 150 focus steps with 150 backlash overshoot in one direction. Have you tried a different filter?