Latest darktable git crashes randomly in lighttable

[piratenpanda]

Is there an existing issue for this?

• I checked and did not find my issue in the already reported ones

Describe the bug

I just browsed through some older images and dt crashes seemingly randomly for me. I'm always in the lighttable when it happens

Steps to reproduce

Use dt,
edit images,
enter lighttable,
edit again,
enter lighttable, scroll, crash

Expected behavior

not crash

Logfile | Screenshot | Screencast

The produced bt:
bt_21.1.txt

the end of -d opencl -d pipe
end_of_pipe.txt

Commit

No response

Where did you obtain darktable from?

darktable.org / GitHub release

darktable version

5.5.0+110~g1793d3a030

It's dirty because I patch the CR3 highlight PR

What OS are you using?

Linux

What is the version of your OS?

Arch, latest

Describe your system

AMD graphics, 64GB RAM

Are you using OpenCL GPU in darktable?

Yes

If yes, what is the GPU card and driver?

No response

Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip

No response

[dterrahe]

@jenshannoschwalm small idea; maybe we could make it easier to catch problems in the thumbnail generation by writing the filename being processed to a file and removing that file when completed. When starting dt, if the file exists, pop up a dialog with the filename saying "it looks like dt crashed last time while processing this file. It is possible the crash was caused by something else"

[jenshannoschwalm]

@dterrahe i guess you know that we already provide that in the logs and you just wanna make sure we see it in the UI at startup ?

[dterrahe]

Yep. I know that once the user reports a problem to us, we can work it out with them. But probably a lot of frustration happens behind the scenes before that point. This could help. Especially less savvy users.

We could even offer to load the potentially bad image into the darkroom with the start of the history selected, so they can figure out which step it is that crashes...

[jenshannoschwalm]

It touches some other points too but how to keep persisting data?

1. if dt crashes while processing, could we keep track out this and disable it also from being processed? Some sort of encapsulating in a disable-process-enable situation?
2. Some would be for opencl and lua maybe?

[dterrahe]

It is hard to appoint blame. So you might be disabling the wrong thing. And then it might not be obvious (to the user) what was disabled and why and how to reenable it. So I thought just saying "well, there was this thing going on in the background that you may not have been aware of at all because there (intentionally) are no indications" might set them on the right path. Crashes at start (lua?) are "easier" in that respect and usually faster to replicate. Crashes in opencl often happen when you start to play with a particular module or are otherwise too complicated for "simple" solutions.

[jenshannoschwalm]

@piratenpanda could you provide a (very large) log including -d imageio -d lighttable -d expose ?

[piratenpanda]

log_23.1.txt

[jenshannoschwalm]

Thanks, this might be related to no testing before dereferencing. Would you be able to check again with #20203

[jenshannoschwalm]

@dterrahe instead of write&delete a file which might be a bit slow sometimes, are you aware of and cross-platform way to alloc some system memory being available across dt sessions?

[dterrahe]

If something like that existed, however it was implemented, it would involve some writing to disk, no?

Remember, my suggestion would be to only do this when generating thumbnails, which would involve reading a raw file, reading a history from database or xmp, processing a pipe, compressing a jpg and writing it to disk. I doubt writing/deleting a file with one filename or imageid would have a measurable impact.

Now if you want to do this for every step in the pipe or something like that, maybe (hard maybe) it would be worth giving this some short thoughts.

Edit: also remember, this is one file, in the config directory. Not one file next to each image that might be on a slow network drive. If you have your config/library file on a slow drive, you're doing it wrong anyway.

[piratenpanda]

Thanks, this might be related to no testing before dereferencing. Would you be able to check again with #20203

Can't seem to reproduce any longer so far

[jenshannoschwalm]

@dterrahe I absolutely agree on you short&simple proposal about just one file and where/how to hook. But, before implementing i just wanted to a mind-wonder around what we could do. Something like let's assume we have 1MB block that is allocated whenever we start dt the first time in a OS session and can be found&used later again. Don't even know if something like this exists at all.

[jenshannoschwalm]

@piratenpanda @victoryforce this might in fact be the same as #20199 and #20193

[jenshannoschwalm]

Should be fixed