💥 Crash Investigations 01: Generic Serialized Fields and #ifdef

Debugging crashes in Unity is always interesting since they are so rare. This is the story of how I discovered an undocumented engine serialization “not-a-bug” while fixing a runtime crash we had recently. The crash happened unconditionally whenever we loaded a scene, and the only lead to go on that I had was a single message in the logs: “sharedassets0.asset is corrupt”.

Disclaimer: The issue explored in this post was observed in Unity version 2021.3.

Where do we begin?

Crashes in Unity games are quite rare, since the majority of the code is executed in the scripting runtime. Because of this, crashes are most often a result of an issue with native code, corrupted files, or an engine bug. This makes them especially tricky to track down.

In this case inspecting the logs indicate a problem with the sharedassets0.asset file. This is a file where Unity stores various assets that may be referenced by multiple scenes. This file could include thousands of different assets. Albeit vague, a lead like this is better than nothing. Remembering a similar issue in the past, my first instinct was that the build had been corrupted somehow, and so the first thing I tried was clearing out all cached build artifacts properly and making a new clean build of the game. This unfortunately did not solve the issue this time. Oh well. Let’s keep looking for clues.

As I was experimenting with the build, having no idea where to even start, at some point I got a tiny clue as to what the issue might be related to. I spotted, for a split second, an error about serialization data layout not matching. Strange, and this has never caused a crash before, in my experience. Now may be a good time to mention that the previous working build was made a week or two ago, so a lot of changes had come in since then. Still, I had a pretty good idea of all the major changes to the project, which I was continuously enumerating in my head, but nothing felt like it could cause such an issue.

With my only lead being that the problem might be serialization related, eventually I realized that the only way forward would be to start stripping away assets from the project.

Divide and conquer

Without any particular clues as to which assets could be problematic, the first order of business was to get rid of all the scenes. I started removing scenes in batches, in a binary-search fashion. Luckily the build time was getting shorter and shorter as I removed things, because I had a feeling that this was going to take a few tries.

Eventually I had no game scenes left, but was still crashing. Very interesting indeed. That means it must be something referenced in our startup sequence, or perhaps pulled in from a Resources folder. So I started attacking those in the same fashion as with the scenes. At this point I had gotten down to a pretty good iteration time of 1 minute per build.

Doing this for quite a while I eventually reached a point where it stopped crashing. I had to take a step back to try to figure out exactly what file was causing the issue, but it was a bit tricky as it seemed it was not one file In particular that caused the issue, but in fact the presence of any of the gameplay mission prefab files. I had to remove all of them for the crashing to stop. I also found that removing the debug-menu prefab from resources resolved the issue. What gives?

Finding the bug

I started considering the clue about serialization again. How could any of these files cause serialization mismatch? One way would be #ifdef-ed out fields, but Unity is supposed to handle those properly. Since we don’t have anything else to go on, let’s follow that lead anyway.

The mission prefabs contain a lot of data fields, but no obvious #ifdef directives are present in the immediate data layout. A couple of custom types are used, for example our SceneReference type which as its name implies, wraps a reference to a scene. This uses #ifdef to remove editor fields from the build. But it has never caused any issues before.

[Serializable]
public struct SceneReference {
	// Only used in builds
	[SerializeField] private int m_buildIndex;
#if UNITY_EDITOR
	// Only used in the editor
	[SerializeField] private string m_sceneGuid;
#endif
	// ---snip---
}

The mission prefabs are actually indirectly referenced by the debug-menu prefab, so this could explain the relation between them. I remembered that we recently restructured how missions are loaded, and this also coincided with us starting to use a new ResourceReference wrapper type, very similar to the SceneReference, but instead referring to an asset in a Resources folder. This is what is used to reference the mission prefabs, and it also uses #ifdef in the same way that the SceneReference does. So I started poking around in it. While I was confident that it was correct, I started removing and testing things that clearly were correct and should not have any effect. But, eventually I noticed something strange; the length of the names of the mission prefabs that the ResourceReferences point to affected whether we crashed or not…? What?

[Serializable]
public struct ResourceReference<T> {
#if UNITY_EDITOR
	// Only used in editor
	[SerializeField] private string m_assetGuid;
#endif
	// Only used in builds
	[SerializeField] private string m_resourcePath; // <-- the string in question
	// ---snip---
}

After some investigation I figured out that if the length of the path string encoded in the ResourceReference, relative to the resource folder, was longer than 16 characters, the game would crash. Interesting, this seems to suggest a serialization/size mismatch issue indeed, and 16 is a curiously even number, perhaps related to some internal buffer size. I also noted that if the string is much longer, then the game would just freeze instead of crashing. Scratching my head, I just tried simply removing the #ifdef, as it now felt likely to be part of the problem, and surprisingly enough, that actually solved the crash. Success!

Finding out why

When the #ifdef was present in the ResourceReference type, Unity would store the editor layout of the type, but try to deserialize it as the non-editor type, which does not have all the same fields present. This causes a size mismatch in the binary serialization, which leads to a crash - most likely a buffer overrun of some sort. The debug-menu prefab was corrupt in the built game because it used the ResourceReference type. The reason removing all the missions also resolved the crash was because that caused all the ResourceReference fields to become invalid and be set to an empty string, which somehow caused the data layout to be handled properly.

I was of course not happy with this as a solution. I needed to find out why the engine doesn’t properly handle serialization around #ifdef in this particular case. So I looked at the SceneReference and the ResourceReference types thinking about what could be different between the two, since only one of them was causing problems. I noted that the ResourceReference was a generic type, but the SceneReference was not. Turns out that removing the generics also fixes the issue! So #ifdef-ing out fields to only be there in the editor does not seem to be handled properly by the engine within this type if it is generic. Unity would store the editor layout of the type, but try to deserialize it as the non-editor type. The serialization system has had a lot of limitations with generic types in the past, but this was entirely unexpected. I could not find any reference to this behavior in the documentation. If anything, the documentation actually suggests that it should be perfectly fine!

Unity Documentation - Script Serialization

After some research, my colleague found this on the Unity issue tracker: Unity Issue Tracker - Serialization layout error thrown in Build when using Generic Classes with fields that only exist in the editor

That’s it, that’s the behavior we are seeing! And it is marked “Won’t Fix”, noting: “We will not add support for this use and it therefore won’t be fixed.”. So in the end the issue turned out to be an undocumented engine serialization “not-a-bug”, that caused the whole game to crash! Fun times. In the end it took around 100 builds in order to fully uncover this issue.

Special thanks to my colleague for helping me brainstorm during the second half of the investigation.

Conclusion

Do not use #ifdef directives that cause the data layout to differ between the editor and build, within generic serialized types. (Non-generic types are still OK!) This is not supported by Unity and will lead to undefined behavior.