Does anyone have a way to exceed the 2048 open file limit in Siril? Thanks in advance!
Siril 2048 file limit
Well Written
Is that really a Siril limitation (sorry, I don't use it), or are you actually referring to the underlying OS? I would guess the latter, and any unix-based OS allow you to change that - the simplest way being "ulimit".
Well Written Insightful Respectful Concise Engaging
There is no such a hard-wired limit, other than the underlying process that may impose limits. And the available memory limit. I fail to understand how you can have 2048 files open at one time though…
andrea tasselli:
There is no such a hard-wired limit, other than the underlying process that may impose limits. And the available memory limit. I fail to understand how you can have 2048 files open at one time though...
*Its something that Siril does when stacking. Im no techno wizard but guessing the OS sees all subs being stacked as files open at the same time (and 2048 is an OS level limit i believe).
@Tony Gondola
You can work around this limit by converting your files into a .Fits cube. Instead of having 3 thousand .Fits files with one layer (or 3 layers if OSC) you have one fits file with 3000 layers and stacking can now proceed.
Helpful Concise
Oskari Nikkinen:
*Its something that Siril does when stacking. Im no techno wizard but guessing the OS sees all subs being stacked as files open at the same time (and 2048 is an OS level limit i believe).
Siril is based on Iris and I did stack more the 2 thousands files in one go back then. Maybe the OP should ask in the Discord channel for Siril but if I were to program something of that kind I would NOT open 2048 file handles at one time (and maybe the limit is in the coding language that's being used for compiling Siril rather than the OS but that is my guess)
andrea tasselli:Oskari Nikkinen:
*Its something that Siril does when stacking. Im no techno wizard but guessing the OS sees all subs being stacked as files open at the same time (and 2048 is an OS level limit i believe).
Siril is based on Iris and I did stack more the 2 thousands files in one go back then. Maybe the OP should ask in the Discord channel for Siril but if I were to program something of that kind I would NOT open 2048 file handles at one time (and maybe the limit is in the coding language that's being used for compiling Siril rather than the OS but that is my guess)
*Were you on Windows at the time? Just curious as i ran into this same issue a couple years back when trying to stack ~3000 files, on Windows 10. At the time i could not find any helpful info on how to remove the limit so used the fits cube method.
Windows XP, as it happens (or happened).
Is that really a Siril limitation (sorry, I don't use it), or are you actually referring to the underlying OS? I would guess the latter, and any unix-based OS allow you to change that - the simplest way being "ulimit=14pxYOu're right, it's a Windows limitaion
Is that really a Siril limitation (sorry, I don't use it), or are you actually referring to the underlying OS? I would guess the latter, and any unix-based OS allow you to change that - the simplest way being "ulimit".
You are right, it's a Windows limitation.
Oskari Nikkinen:andrea tasselli:
There is no such a hard-wired limit, other than the underlying process that may impose limits. And the available memory limit. I fail to understand how you can have 2048 files open at one time though...
*Its something that Siril does when stacking. Im no techno wizard but guessing the OS sees all subs being stacked as files open at the same time (and 2048 is an OS level limit i believe).
@Tony Gondola
You can work around this limit by converting your files into a .Fits cube. Instead of having 3 thousand .Fits files with one layer (or 3 layers if OSC) you have one fits file with 3000 layers and stacking can now proceed.
How do I create that type of file?
Tony Gondola:Oskari Nikkinen:andrea tasselli:
There is no such a hard-wired limit, other than the underlying process that may impose limits. And the available memory limit. I fail to understand how you can have 2048 files open at one time though...
*Its something that Siril does when stacking. Im no techno wizard but guessing the OS sees all subs being stacked as files open at the same time (and 2048 is an OS level limit i believe).
@Tony Gondola
You can work around this limit by converting your files into a .Fits cube. Instead of having 3 thousand .Fits files with one layer (or 3 layers if OSC) you have one fits file with 3000 layers and stacking can now proceed.
How do I create that type of file?
You can convert a sequence of subs to another format in the Sequence tab. From the Sequence export section click the dropdown menu to the right and select FITS cube. Give the new sequence a name and click Export sequence. Next watch as your hard drive begs for mercy, because you are writing all of those images into a single file so it can take a while.
There is one problem though, at least there was the last time i had to resort to this a couple years back. Not sure if more recent versions of Siril still have the same issue, but anyway. The problem is that registration will write new files, and the last time i did this it wrote single fits files, undoing all the work to convert the sequence into a FITS cube in the first place. If this happens, do registration first to the sequence before converting to a cube. That way you can stack straight away when the cube finishes writing on disk because the data is already registered. Downside is you lose the weights that were calculated during registration so its a compromise. But better than not being able to stack at all. Do try to stay under 2048 images in the future as its a pain in the ass to deal with even when Siril works really fast.
Another possible way to stack a large number of files is to convert them to a .SER video file, also from the sequence export menu. But these are limited to 16-bit of precision so not optimal either, probably easier though.
Helpful Engaging
Tony Gondola:Is that really a Siril limitation (sorry, I don't use it), or are you actually referring to the underlying OS? I would guess the latter, and any unix-based OS allow you to change that - the simplest way being "ulimit".
You are right, it's a Windows limitation.
Ahhh, Windows... different beast, different issues. There is indeed a 2048 open file descriptors limit in Windows, but that doesn't come from the OS... it is instead hardcoded in the C runtime library (msvcrt), which is used virtually by any Windows application in any language... I don't think there is anything you can do about it.
Well Written Helpful Insightful Concise
Oskari Nikkinen:Tony Gondola:Oskari Nikkinen:andrea tasselli:
There is no such a hard-wired limit, other than the underlying process that may impose limits. And the available memory limit. I fail to understand how you can have 2048 files open at one time though...
*Its something that Siril does when stacking. Im no techno wizard but guessing the OS sees all subs being stacked as files open at the same time (and 2048 is an OS level limit i believe).
@Tony Gondola
You can work around this limit by converting your files into a .Fits cube. Instead of having 3 thousand .Fits files with one layer (or 3 layers if OSC) you have one fits file with 3000 layers and stacking can now proceed.
How do I create that type of file?
You can convert a sequence of subs to another format in the Sequence tab. From the Sequence export section click the dropdown menu to the right and select FITS cube. Give the new sequence a name and click Export sequence. Next watch as your hard drive begs for mercy, because you are writing all of those images into a single file so it can take a while.
There is one problem though, at least there was the last time i had to resort to this a couple years back. Not sure if more recent versions of Siril still have the same issue, but anyway. The problem is that registration will write new files, and the last time i did this it wrote single fits files, undoing all the work to convert the sequence into a FITS cube in the first place. If this happens, do registration first to the sequence before converting to a cube. That way you can stack straight away when the cube finishes writing on disk because the data is already registered. Downside is you lose the weights that were calculated during registration so its a compromise. But better than not being able to stack at all. Do try to stay under 2048 images in the future as its a pain in the ass to deal with even when Siril works really fast.
Another possible way to stack a large number of files is to convert them to a .SER video file, also from the sequence export menu. But these are limited to 16-bit of precision so not optimal either, probably easier though.
It looks like converting the files to a FITS sequence takes care of the problem. Once I do that I can calibrate, align and stack as normal. I just can't do it a script but that's ok. One of these days I'll get around to writing one.