Loading Images from Disk in Unreal Engine 4
So over the past week I’ve been doing a lot of work on getting the pieces ready for my VR Comic Book Reader.
I found a great resource by UE4 forums user Ehamloptiran for loading an image from disk into a texture at game-time. A user on Reddit requested I put together a simple plugin example, which you can find here: TextureFromDisk Plugin for Unreal Engine 4. Also available as part of user Rama’s Blueprint node library.
Unfortunately, that code does not handle anything but DDS images – so I had some more work and investigation to do. After a lot of searching the engine code, and in particular the bits that are responsible for importing textures, I found the ImageWrapper class.
ImageWrapper, as the name implies, wraps up the functionality of specific image formats for easy loading. As such, you simply pass in what format your image is in, and call SetCompressed, passing in your image data. Then you can call GetRaw to get raw, uncompressed color data out.
After that, it’s simply a matter of copying the raw data into the UTexture2D’s mip map data. For my purposes, I only ever use a single mip.
Reading Zip Files
That done, I moved on to the hard part of this whole endeavor (or so I thought at the time), namely reading zip files and extracting out the files contained therein.
For those not familiar with the CBZ file format, or comic readers in general, CBZ (and other file formats like CBR) are just regular archive formats, with the last letter specifying what type of archive is used. CBZ is Comic Book Zip, CBR is Comic Book RAR.
These zip/rar files have a sequential (numbered with leading zeros 001 instead of 1 or 01) set of images which make up the comic book.
I had a heck of a time finding a liberally licensed, simple zip reading library. After a couple of false starts with miniz – C Zip Reading Library, and I managed to find JUnzip, a much simpler (though less fully featured) implementation.
Both libraries use ZLIB to inflate the compressed data, which I found acceptable mostly because Unreal 4 already compiles against ZLIB.
Miniz was almost right for my purposes, but I quickly found out that UE4 does not like it when you try and include standard code, such as a standard memcpy, inside the engine. Because miniz supports a decent amount of the functionality of zip archives, it was a bit too complex to quickly port to using the UE4 API functions for reading files etc.
I switched over to JUnzip, which has much easier to follow code, and is also on a nice public domain license (well, lack of license).
I was able to quickly go through JUnzip, rip out the standard file operations (fopen, fseek, ftell etc.) and replace them with UE4’s versions of same.
Problems were had. It turns out, __attribute__ (_packed_) was actually quite important in JUnzip in order to make sure that the structures that define the zip header etc. are the correct size. I had initially removed the __attribute__ (_packed_) declaration on the structures because they are not compatible with Visual Studio. This caused no end of headaches.
Then I realized that if __attribute__ is a compiler feature, VS probably has its own version I could use. Thus I found #pragma pack() and all was well (supposedly, gcc supports the #pragma as of some version or other, so I may not even need to change it later in order to support non-VS compilers).
That was not to be the end of my odyssey, however.
It Hurts When I Pee…er… It Hangs When I Load
Zip file and jpg texture loading works, at this point. Unfortunately, it also blocks the main game thread and hangs the game until done. While that might work okay for files that can be loaded one time at start up, it obviously won’t work out well for comic books, since you presumably want to be able to switch what you’re reading on the fly.
That meant I needed to thread away the zip file reads. Luckily UE4 comes with a nice handy asynchronous IO system built in: FIOSystem.
To fire off an async read, you simply get the singleton FIOSystem with FIOSystem::Get() and then call LoadData on it, like “FIOSystem::Get().LoadData()”. You pass in a file name, offset into the file, a size of bytes to read and a destination buffer.
I couldn’t figure out a clean way to check the status of these requests, so I simply wrote a couple of special bytes to the end of my destination buffer, and then check them in the timer I use to check all the requests. If the bytes are not equal to my special values, the buffer is read and can be operated on.
It still wasn’t enough – the decompression was still causing the game thread to block and hang.
Thanks to Rama’s Multi-Threading Tutorial and some digging into FulfillCompressedRead (which is called when you perform an FIOSystem::LoadCompressedData – which unfortunately only works on .PAK files, more on that later), I was able to set up a threaded decompression task.
After a (compressed with zip) chunk of file data is loaded from the above described process, a thread is fired off which will decompress the data into an output buffer (passed as a pointer in the thread init). Unfortunately, mip map generation for the texture as still hanging the main game thread.
I moved that into the decompression thread and changed to code to generate a texture directly from mip data (memcpy’d in to the mip buffer, rather than copied byte by byte) and it works! Finally, zero-hitching texture loading from a zip file on disk, during game time (i.e., not restricted to editor-only).
There remained a bug in that the blueprint library which exposes these calls for getting a texture was not cleaning up the cache properly if a game was stopped but the editor wasn’t closed down. That was an easy fix, and I can happily say that it all works.
Now starts the process of extending out my current BP based comic book handling functionality (i.e., get textures, set them, next/prev etc.).
Native Engine Support?
As I alluded to above, it would actually be only a minor extension for the engine to support compressed reads from zip files, rather than only from pak files.
In particular, two main things need to happen:
1) FulfillCompressedRead needs to check the file extension, and skip the PAK file header check if the file does not end in .pak. Or there needs to be another parameter for LoadCompressedData in FIOSystem to specify that directly.
2) The async thread class that’s used would need to be extended so it could use the inflate() functions of ZLIB, rather than just simply uncompress().
That’s pretty much it. I may actually take a stab at doing this myself, but for now what I have works okay, so engine level improvements are on the back burner. It would also help if someone from Epic could chime in about how they feel on this sort of extension and whether it actually would fit and make sense or not.
I haven’t had time to get a video, and to be honest it’s not that terribly impressive since it just looks like the texture suddenly is applied to the flat plane there, so I’m just going to show a screenshot I took when it was still hanging the game thread when loading the textures.
The cover page of Prophet (2012) loaded into UE4 at runtime.