Lumina Studio v0.1 pt. 2
Major Problems during v0.1
There are numerous problems that I've faced during the development of this engine. One of them was creating an RHI.
It was easy... well... to say, "I want RHI". Render Hardware Interface. Damn. Now before you say, "But you're already using Vulkan....", I know. I just wanted to make it complicated and making an RHI will at least help practice my abstraction. Until it didn't.
The forever changing architecture
The render engine was supposed to be simple (ok simple like straightfowrard. not simple simple).
Obviously this is the most simple way to explain it.
However, with me demading an RHI, it added a new layer
DX12 and Metal still doesn't exist here, but this was the simple idea. We're all familiar with interfaces, and having an RHI would allow me to do this.
However, if you think about it, RHI isn't really that helpful, or in a business perspective, doesn't really add value to the product.
Sure, you support multiple platforms, but Vulkan already does that. You might argue that PS5 doesnt support Vulkan... yeah sure. If I did want to create games for PS5, I am not going to make an engine. I'd be using either Unity or Unreal.
It's often rare for users to change APIs as well, they never do.
But hey, at least its fun right?
To add more complication to it, the architecture changed yet again.
Multithreading happened and I had to change how rendering is done. Of course, this isn't fully the representation of the actual classes, but the idea is there.
Of course, this isn't isolated to the render engine, but in other systems too:
This was the initial asset management system where all of my assets are retrievable from the ResourceManager. Now:
It now has a full asset system behind it.
Its easy to get lost in the architecture of a game engine and to be honest, there isn't really a right solution to it. The systems blows up, the question now is how to manage those explosions.
MVCS Architecture
It was around 10 years ago when I started using this architecture in all of my projects. I initially learned it when I was using StrangeIoC in Unity https://strangeioc.github.io/strangeioc/.
MVCS was and still is a great way to split up a software project, and it makes sense. Data, Controllers, and Views, all have different responsibilities. StrangeIoC added the services as part of the MVC framework. Services could be authentication services, multiplayer services, or even concerete features.
In the game engine, I have the Data/Models let's say transform components (ECS), a service (S in MVCS) would be the TransformSystem (ECS), and the runtime update would be the controller (MVC), and of course, entities (ECS) will be the views (MVC). StrangeIoC has its own diagram but it's more inclined to Unity than in general software engineering (which can be translated into software engineering if you strip out the unity stuff).
But yeah, with these kind of framework, I was able to abuse. Yes, abuse, the services part because this is where I have most of the freedom than in the MVC part of the game engine. I can treat them as actual services, or even concrete features of the engine.
Samples of which are these which you might realize that... these will be similar to a micro service architecture in web!
Okay, i am not too sure to be honest. What I know about micro services is that two small backends talking to each other and doing different things. Can be used in parallel, async, et.. wait.
That's exactly the point of the services in MVCS!
The MVCS Caveat
The way I use MVCS in my game engine is as you guessed it (and I mentioned it) is subjected to service abuse. In which I guess will be the same regardless if I use MVCS or not, but it is much better to have a framework, than none.
Oh lord imagine not having a framework.
But anyway,
at least having a framework makes every system clearer and isolated. Of course this wasn't always the case. There will be always one system in a game engine that will always, always break you. And I have subtley detailed it earlier.
The overarching assets problem
Ohhh boy. This was the system that always blows out of proportion. Well, I guess it me who never really had any experiences in game engines.
I thought of assets as, well, assets. images, 3D assets, sounds, etc.
Now that I have made this far in the engine, I classify them into three things now:
Raw Assets
Obviously, this is what I meant. You understand it. I understand it.
CPU Ready Assets
Now this is a bit tricky, but let's think of this asset as processed or imported assets. Now, their placement could differ as well and there are two places where they can be retrieved at least in my game engine: RAM, or Disk.
Assets that are imported in Lumina Studio gets processed into CPU/GPU ready assets, or in their binary form, unpacked PNG/JPG, or a list of vertices/indices for meshes. These processed data are going to be cached in a folder in their unpacked, lossless form. This is okay because unpacking assets are slower than reading them in their lossless, binary format.
Now the Asset System checks for the cache first, and brings the processed asset or cache, with out the need to unpack them:
GPU Ready Assets
Not all assets can be GPU ready. Obviously the onese that needs to be GPU ready are the ones needed to be rendered. This just means that the assets are sent to the gpu, and gpu had provided a handle for it.
This is helpful because knowing that assets are in the GPU, means we can get rid of their CPU counterparts. Imagine a 4K image taking space both on your RAM and GPU. Insane. That memory in RAM can be reclaimed for other uses.
Classifying these are easy, but managing these is different. Of course some assets needs to be released in the GPU and RAM, that's a different problem altogether.
Caches needs to be updated as well if they get changed. That's a different problem as well. (I used hashing)
and not to mention, packing and dependency mapping. I'm too tired and hungry to explain further, I basically crawl the scene for dependencies and make a dependency map, and then pack the CPU ready assets in a binary blob. I use the dependency map to create a dictoinary to locate those assets in the binary blob, and etc. Its all convoluted, and probably a topic on its own.
As you may have guessed, all of these are lossless assets. It doesn't involve any compression at all because I have yet to decide on that later on.
I guess I was right, I still have a lot to yap about the v0.1. I think I'm going to talk about the scripting and C# integration next, or maybe I'd focus more on the editor. Probably the first one.
I have been testing and fixing things in the engine for the meantime. I am also on a feature freeze, so that I can focus more on the stability, and cohesiveness of the game engine.