Crash on load with entrypoint injection on Linux

[sersorrel]

On Linux, using the newly-supported entrypoint injection method, Penumbra crashes on load:

2024-03-20 20:32:54.538 +00:00 [INF] [PLUGINM] Loading plugin Penumbra
2024-03-20 20:32:54.542 +00:00 [INF] [PLUGINM] Penumbra defaultState: true wantedByAnyProfile: true loadPlugin: true
2024-03-20 20:32:54.544 +00:00 [INF] [LOCALPLUGIN] Loading Penumbra.dll
2024-03-20 20:32:54.577 +00:00 [DBG] [SVC] Service<FontAtlasFactory>: Construction complete
2024-03-20 20:32:54.577 +00:00 [DBG] [SVC] Service<DalamudInterface>: Begin construction
2024-03-20 20:32:54.577 +00:00 [DBG] [SVC] Service<NotificationManager>: Begin construction
2024-03-20 20:32:54.581 +00:00 [DBG] [SVC] Service<NotificationManager>: Construction complete
2024-03-20 20:32:54.638 +00:00 [DBG] [SVC] Service<DalamudInterface>: Construction complete
2024-03-20 20:32:54.766 +00:00 [ERR] [Penumbra] Error constructing Penumbra, Disposing again:
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at Penumbra.Services.ValidityChecker.GetGameVersion() in D:\a\Penumbra\Penumbra\Penumbra\Services\ValidityChecker.cs:line 39
   at Penumbra.Services.ValidityChecker..ctor(DalamudPluginInterface pi) in D:\a\Penumbra\Penumbra\Penumbra\Services\ValidityChecker.cs:line 35
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
   --- End of inner exception stack trace ---
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
   at System.Reflection.MethodBaseInvoker.InvokeWithOneArg(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
   at OtterGui.Services.ServiceManager.<>c__DisplayClass18_0.<AddSingleton>g__Func|0(IServiceProvider p) in D:\a\Penumbra\Penumbra\OtterGui\Services\ServiceManager.cs:line 129
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at OtterGui.Services.ServiceManager.GetService[T]() in D:\a\Penumbra\Penumbra\OtterGui\Services\ServiceManager.cs:line 38
   at Penumbra.Penumbra..ctor(DalamudPluginInterface pluginInterface) in D:\a\Penumbra\Penumbra\Penumbra\Penumbra.cs:line 56

I guess Framework.Instance() is null very early on startup?

Penumbra/Penumbra/Services/ValidityChecker.cs

Lines 38 to 39 in fda77b4

	private static unsafe string GetGameVersion()
	=> Framework.Instance()->GameVersion[0];

There is a null check for it elsewhere:

Penumbra/Penumbra/Interop/Services/FontReloader.cs

Lines 32 to 34 in fda77b4

	var framework = Framework.Instance();
	if (framework == null)
	return;

(entrypoint injection may also hang on Linux if Penumbra is enabled, but til this is fixed it's hard to test that)

[sersorrel]

1.0.2.1 fixes the crash :) unfortunately we're now back to hanging on startup (before the game window appears) instead

[Ottermandias]

Do you have a log for that too?

[sersorrel]

Sure: dalamud.log

Last couple of lines:

2024-03-21 19:20:15.270 +00:00 [VRB] [Penumbra] [BNpcNameDict] Created v3 for English in 427 ms.
2024-03-21 19:20:15.452 +00:00 [VRB] [Penumbra] [Emotes] Created v8 for English in 608 ms.

[randomei]

For me with version 1.0.2.1 the game loads almost every time. With 1.0.2.5 game successfully loads like 1 of 3 times.. Processes not responding, I must kill them to try again.

[randomei]

Version 1.0.2.6 still hanging on startup

[Ottermandias]

Try disabling the crash handler in the config json (if you can't get ingame to do it via settings) and tell me if it hangs due to that.

[randomei]

Crash handler disabled, still the same.
dalamud.old.log
Penumbra.json

When I'm returning version 1.0.2.1, the game starts normally:
dalamud.log

P.S. If I disable "Waitng for plugins", the game starts with version 1.0.2.6
dalamud.log

[randomei]

I've tested on another computer, the game starting with version 1.0.2.6. So it seems that the issue is not with penumbra...

[Ottermandias]

2024-03-30 17:45:14.815 +03:00 [INF] [Penumbra] Set new mod base directory from to .
this happening sounds kinda weird possibly, in your logs? Can you try on your main PC after setting a root directory maybe?

[randomei]

That was some problem with xlcore configuration on main PC, so I've cleaned configuration. Now game starts every time w/o Penumbra and with entrypoint enabled. With Penumbra still hangs about 2 of 3 times even w/o mods.
dalamud.log

About 1 of 3 times game starting normally.
dalamud.log

Same problem with version 1.0.2.6
dalamud.log

It seems, that with the latest version it hangs on start even more often.

[bayta-88]

Hello I'm on NixOS and I am also experiencing this issue:
Using "DLL injection" method with or without Penumbra: Game always loads correctly
Using "Entrypoint" method without Penumbra: Game always loads correctly
Using "Entrypoint" method with Penumbra: Game loads correctly like 10-20% of the time. Most of the time the XIVLauncher window disappears, the process continues running in the background but the game window never loads and one of the last lines in dalamud.log is "[LOCALPLUGIN] Loading Penumbra.dll".

[Ottermandias]

Well, I can't test anything outside of windows, so I'm not sure how to help you. If any of you is able to self-compile or attach debuggers, you can do the latter to try to figure out where it deadlocks.
It might also change to remove the TaskScheduler.Default I recently added to most ContinueWith calls when Dalamud fucked up the task context in some cases.

[bayta-88]

I have done some minor modifications of other plugins using Visual Studio on Windows before so I wouldn't mind trying to help with debugging if that helps. Issue is that from what I'm reading in the Dalamud guide plugin development/debugging has to be done with Visual Studio which doesn't seem to be supported on Linux... do you know if it works decently with Wine? Or is there any other way of debugging plugins running on a Linux system?

[Ottermandias]

Sorry, but I have no clue.

[bayta-88]

Just in case it helps, after some fiddling around I've found that disabling the "Wait for plugins on startup" option under "Advanced" makes Penumbra be loaded correctly every time even when the "Entrypoint" method is chosen but from what I've been told this basically negates the benefits of choosing "Entrypoint" in the first place.
For me everything seems to be loading correctly tho.

[bayta-88]

From what I've been told it appears that the issue is that the Entrypoint method (with the "Wait for plugins on startup" option enabled) stalls the game process while every plugin is being loaded. Apparently stalling the process on Windows is not an issue but when running the game on Wine this causes DXVK to timeout and the game to just not-start. With most plugins the amount of stalling required is not enough to cause problems but with Penumbra if people have enough mods (I have 65GB) this stalling can be long enough to make DXVK timeout causing the problem... if this assessment is correct I think unless some major redesign is done to make Penumbra capable of loading everything much faster (which is likely not possible) or the loading of some mods is delayed the issue is not really solvable... maybe a warning could be added somewhere to let Linux users know that if they have too many mods they may need to disable the "Wait for plugins on startup" option and/or pick the "DLLInject" method to make the game start.

Edit: Typo.

[randomei]

@bayta-88 You may be right. I've tested it on a newer and more performant hardware and the game starts every time with stable version. Not tested this with unstable tho.