Large designs unable to calculate volume

[jakbruce2012]

Large volume designs that worked last universe cause the volume calculation to lock up.
This issue seems to appear randomly between 28 million and 30 million cubic meters, and all designs above this will not work.
Confirmed on the laptop with 16GB of ram as well as the desktop with 64GB of ram.
The desktop can occasionally make it to 31 million without locking up.  Ram usage settles out around 10-11GB on each before CPU usage drops to 0%.

Also tested this issue using a single hull cube, this also causes the issue when the cube is above 28-20 million cubic meters.

Screenshot here of situation, been sitting in this state for about an hour,  ram usage gradually decreasing with time from peak usage.
No signs of progress in the studio.


https://www.dropbox.com/s/63d3dzvj1zweq4...o.PNG?dl=0

[Rockinsince87]

Perhaps this is the effect of the recent update? 

Maximum Ship Model Size Reduced

The size reduction was in megabytes, not in physical dimensions. The size limit of spacecraft designs is now 28MB; the uncompressed size limit is now 14MB. Based on the existing designs in the database, this new size would only exclude a few of them.

Src: https://www.hazeron.com/mybb/showthread.php?tid=2637

[jakbruce2012]

Perhaps this is the effect of the recent update? 

Maximum Ship Model Size Reduced

The size reduction was in megabytes, not in physical dimensions. The size limit of spacecraft designs is now 28MB; the uncompressed size limit is now 14MB. Based on the existing designs in the database, this new size would only exclude a few of them.

Src: https://www.hazeron.com/mybb/showthread.php?tid=2637

Negative.  This issue existed in 2021, just before the shutdown.  Appeared with the last update of starship.
This particular ship was compiled and built ingame just prior to shutdown.
I will be testing online at some point as well, just waiting for a day that Haxus is online in case it eats the cluster.

[jakbruce2012]

Upon further experimentation, across multiple machines there seems to be a trend. Cache size appears to be a factor for maximum calculable volume.
My laptop on an i7-7300HQ has 6MB cache, compiles up to approximately 21M cubic meters without crashing.
My desktop with i7-6700k has 8MB cache, and compiles up to approximately 28M cubic meters, sometimes 29M if I try a few times.
Aa empire member's machine I was working with on this issue has an i7-10700F with 16MB of cache, and was able to get about 42M cubic meters, however they were also runnning an instance of the game at the same time.

Question. Is the process for calculating the volume of a craft/building using a recursive algorithm, and what kind of error reporting back to the program do you get if the CPU runs out of stack and or cache to spend on the process? On windows it seems to just lock up, or kick to primeval world if near the limit. From what I heard on discord linux will auto kill the process when it runs out of memory.

Does anyone else have CPUs with larger Caches to test with?
Been using cubes in the design studio for this so far, centered on the origin. Spheres seemed to perform worse.

Has anyone tried refreshing obstructions on a large ship on AMD machine yet? Maybe intel specific issue?

[Giovanni]

I don't know if this data point will help, but with an intel i7-9750H with a 12MB L3 cache, my limit seems to be about 34M cubic meters. I draw a 100 x 1000 x 350 block, comes out to a bit less than 35M cubic meters. That works about 60% of the time. If I increase by 10% and try 100 x 1100 x 350, that has failed every time I tried so far.

[QuakeIV]

Originally Jack theorized there is a recursion depth issue, however I connected to AuSceneClient while it was running with GDB, and it looks more like a thread deadlock issue, all the threads get stuck waiting on mutexes (well, futex underneath calls into QMutex). Its possible that what is really happening is that staying in the cache reduces the odds of the deadlock, but as soon as you get into main memory the delays are such that the deadlock is more likely to happen.

This would also explain why the failure to finalize is unpredictable, if some kind of race condition is underlying it.

[jakbruce2012]

Oh man... you're going to need to show me how to do that some time. That's neat.

[QuakeIV]

Here is the state after it appears to deadlock for reference:

Code:

(gdb) info threads
  Id   Target Id                                           Frame
* 1    Thread 0x7f5ba56b9800 (LWP 49616) "AuSceneClient"   futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x7f5b6c90b070) at ../sysdeps/nptl/futex-internal.h:183
  2    Thread 0x7f5ba49b2700 (LWP 49617) "QXcbEventQueue"  0x00007f5ba8a87aff in __GI___poll (fds=0x7f5ba49b1ca8, nfds=1, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
  3    Thread 0x7f5b9ffff700 (LWP 49618) "DtTrash"         futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x7f5b6c910010) at ../sysdeps/nptl/futex-internal.h:183
  4    Thread 0x7f5b9f77d700 (LWP 49621) "threaded-ml"     0x00007f5ba8a87aff in __GI___poll (fds=0x7f5b90009c50, nfds=3, timeout=506) at ../sysdeps/unix/sysv/linux/poll.c:29
  5    Thread 0x7f5b9ef7c700 (LWP 49622) "AuSceneClient"   futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55e9c892c7f8) at ../sysdeps/nptl/futex-internal.h:183
  6    Thread 0x7f5b9e554700 (LWP 49623) "AuSceneClient"   futex_abstimed_wait_cancelable (private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x55e9c88d4110) at ../sysdeps/nptl/futex-internal.h:320
  7    Thread 0x7f5b9de0e700 (LWP 49624) "QDBusConnection" 0x00007f5ba8a87aff in __GI___poll (fds=0x7f5b7c0025e0, nfds=3, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
  8    Thread 0x7f5b760f2700 (LWP 49629) "DtTick"          futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x7f5b6c90b070) at ../sysdeps/nptl/futex-internal.h:183
  9    Thread 0x7f5b758f1700 (LWP 49630) "DtTock"          futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x7f5b6c90b070) at ../sysdeps/nptl/futex-internal.h:183
  10   Thread 0x7f5b74a20700 (LWP 49634) "AuModel"         0x00007f5ba8a523bf in __GI___clock_nanosleep (clock_id=clock_id@entry=0, flags=flags@entry=0, req=0x7f5b74a1fbd0, rem=0x7f5b74a1fbd0)
    at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:78
  11   Thread 0x7f5b67fff700 (LWP 49635) "DtTarget"        0x00007f5ba8a523bf in __GI___clock_nanosleep (clock_id=clock_id@entry=0, flags=flags@entry=0, req=0x7f5b67ffed80, rem=0x7f5b67ffed80)
    at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:78

Additionally, attached the ship I am fiddling with.

[Athena]

It's worse than it was before. I can't calculate designs I made and finalized three weeks ago lmao.

[Vhwatgoes]

I'd like to add on to the OP that even under normal circumstances, calculating volume on large designs can take a prohibitive about of time and require a fair amount of RAM.

Also currently, calculating large designs, or performing anything intensive that may hang the app for a few minutes such as a complex jig cut, ejects me into a preview world where I cannot return to the designer, effectively soft-crashing me.