Hi everyone! Today we’ll be starting on one of the most exciting topics in the tutorial: Hot reloading! With hot reloading, you will be able to allow users of your engine to modify, recompile and reload their C# scripts independently of the engine’s source code. This is very important as it rapidly speeds up iteration times in your engine.
This is the seventh in a series of posts on building a .NET 6.0-based C# scripting engine for a game engine. If you’ve not yet read the previous posts, you can start here.
Decoupling the Script Assembly
The first step to enable hot reloading is to decouple our script assembly away from the engine’s solution. This would enable us to build scripts separately which would let us easily, unload, compile and reload the script assembly in that order.
To do this, we’ll open up the Solution Explorer and remove the assembly.
For the purpose of this tutorial, we will keep things simple by leaving the “ManagedScripts” project folder in the same place. However, in a real application, you would want to move it to it’s own folder depending on the way you have structured your project’s folder hierarchy.
Now that the project is outside of our Visual Studio solution, we need to modify it so that it knows where to look for the ScriptAPI.dll that contains the code it requires to access classes like Script and TransformComponent. We’ll remove the XML blocks for ProjectReference and replace them with a Reference block that specifies ScriptAPI and the relative path to the DLL from the ManagedScript.csproj‘s directory. So we’ll get something like this:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Library</OutputType>
<TargetFramework>net6.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<Platforms>x64</Platforms>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<OutputPath>$(SolutionDir).bin\$(Configuration)-$(Platform)\</OutputPath>
<PlatformTarget>x64</PlatformTarget>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<OutputPath>$(SolutionDir).bin\$(Configuration)-$(Platform)\</OutputPath>
<PlatformTarget>x64</PlatformTarget>
</PropertyGroup>
<ItemGroup>
<Reference Include="ScriptAPI">
<HintPath>..\.bin\Debug-x64\ScriptAPI.dll</HintPath>
<HintPath>..\.bin\Release-x64\ScriptAPI.dll</HintPath>
</Reference>
</ItemGroup>
</Project>
Notice that we added two <HintPath> blocks? Each one of them is for a specific build configuration of our engine, however, we can’t really tell the script’s build tool to select based on our engine’s build configuration so the build tool will pick the first one it finds. As you can imagine, this may cause problems with the wrong DLL being referenced if you have both configurations built. Hence, in a real project, you would dynamically generate this file based on the current engine’s build configuration.
Now that we are building the scripts separately, we will also need to remove the existing post-build steps which we added to our “Executable” project that help copy the built script assembly into our output directory.
Compile the Script Assembly
Preparation
The next step is to compile the scripts from our engine at runtime. To do this, we will need to invoke the .NET compiler to generate a DLL of our script assembly from the script assembly’s project files. For this project, we will assume that the .NET runtime’s executable (dotnet.exe) exists at it’s default location at C:/Program Files/dotnet/.
We will create a compileScriptAssembly() function and place our build command inside it:
// Application.h
...
namespace Core
{
...
class DLL_API Application
{
...
private:
void compileScriptAssembly();
...
};
}
// Application.cpp
...
namespace Core
{
...
void Application::compileScriptAssembly()
{
const char* PROJ_PATH =
"..\\..\\ManagedScripts\\ManagedScripts.csproj";
std::wstring buildCmd = L" build \"" +
std::filesystem::absolute(PROJ_PATH).wstring() +
L"\" -c Debug --no-self-contained " +
L"-o \"./tmp_build/\" -r \"win-x64\"";
}
}
The first argument is the relative path to the script assembly project file (ManagedScripts.csproj) from the application’s current directory. It is followed by 2 command line switches that you may change or add depending on your use case but the switches we have used here are as follows:
| Switch | Description |
c | Configuration of the project to build. By default, only Debug and Release are valid. |
-no-self-contained | Marks the build as a framework-dependent which is only allowed option for a DLL. |
o | Path to the output file. |
r | Configuration to build. |
Notice that in our case, we have built the files into a tmp_build folder. That is where the ManagedScripts.dll will be built to and we will have to copy it out later on.
Building
Now that we have the build command, we will pass this into dotnet.exe to execute it using the Win32 CreateProcess() function like this:
// Application.cpp
...
namespace Core
{
...
void Application::compileScriptAssembly()
{
...
// Define the struct to config the compiler process call
STARTUPINFOW startInfo;
PROCESS_INFORMATION pi;
ZeroMemory(&startInfo, sizeof(startInfo));
ZeroMemory(&pi, sizeof(pi));
startInfo.cb = sizeof(startInfo);
// Start compiler process
const auto SUCCESS = CreateProcess
(
L"C:\\Program Files\\dotnet\\dotnet.exe", buildCmd.data(),
nullptr, nullptr, true, NULL, nullptr, nullptr,
&startInfo, &pi
);
// Check that we launched the process
if (!SUCCESS)
{
auto err = GetLastError();
std::ostringstream oss;
oss << "Failed to launch compiler. Error code: "
<< std::hex << err;
throw std::runtime_error(oss.str());
}
// Wait for process to end
DWORD exitCode{};
while (true)
{
const auto EXEC_SUCCESS =
GetExitCodeProcess(pi.hProcess, &exitCode);
if (!EXEC_SUCCESS)
{
auto err = GetLastError();
std::ostringstream oss;
oss << "Failed to query process. Error code: "
<< std::hex << err;
throw std::runtime_error(oss.str());
}
if (exitCode != STILL_ACTIVE)
break;
}
}
}
We can then check the result of the operation to check if it is successful or unsuccessful. If the operation is successful, we’ll want to copy the built DLL into our engine’s directory.
// Application.cpp
...
namespace Core
{
...
void Application::compileScriptAssembly()
{
...
// Successful build
if (exitCode == 0)
{
// Copy out files
std::filesystem::copy_file
(
"./tmp_build/ManagedScripts.dll",
"ManagedScripts.dll",
std::filesystem::copy_options::overwrite_existing
);
}
// Failed build
else
{
throw std::runtime_error("Failed to build managed scripts!");
}
}
}
Reloading the Script Assembly
Now that we have the code for building the assembly, let’s put it all together. First, we will need to unload the existing assembly. We can only do this from the managed ScriptAPI project. Since the EngineInterface class is responsible for loading the script assembly, it only makes sense that it is responsible for unloading it. However, we’re going to have to take a small detour first.
Loading with an Assembly Load Context
.NET does not provide any means to simply unload an assembly, instead if we want to unload assemblies, we need to first load the assembly into an AssemblyLoadContext, we can then unload the context to unload all assemblies loaded into it. So let’s change how we load our assembly:
// EngineInterface.hxx
...
namespace ScriptAPI
{
public ref class EngineInterface
{
...
private:
...
static System::Runtime::Loader::AssemblyLoadContext^ loadContext;
...
};
}
// EngineInterface.cxx
...
namespace ScriptAPI
{
void EngineInterface::Init()
{
using namespace System::IO;
loadContext =
gcnew System::Runtime::Loader::AssemblyLoadContext(nullptr, true);
// Load assembly
FileStream^ managedLibFile = File::Open
(
"ManagedScripts.dll",
FileMode::Open, FileAccess::Read, FileShare::Read
);
loadContext->LoadFromStream(managedLibFile);
managedLibFile->Close();
// Create our script storage
scripts = gcnew System::Collections::Generic::List<ScriptList^>();
for (int i = 0; i < Core::Application::ENTITY_COUNT; ++i)
{
scripts->Add(gcnew ScriptList());
}
// Populate list of types of scripts
updateScriptTypeList();
}
}
Reloading the Assembly
Now let’s create a function that reloads the script assembly. We’ll need to take note of a few things, however. The most important thing is that before we reload the script assembly, we need to make sure that there are no longer any references to types from the script assembly still in memory. In our case, we will need to clear our scripts list and scriptTypeList list. We will then force the CLR’s garbage collector to dispose of them. Then, we’ll call our Init() function to load the scripts again.
// EngineInterface.hxx
...
namespace ScriptAPI
{
public ref class EngineInterface
{
public:
...
static void Reload();
...
};
}
// EngineInterface.cxx
...
namespace ScriptAPI
{
...
void EngineInterface::Reload()
{
// Clear all references to types in the script assembly we are going to unload
scripts->Clear();
scriptTypeList = nullptr;
// Unload
loadContext->Unload();
loadContext = nullptr;
// Wait for unloading to finish
System::GC::Collect();
System::GC::WaitForPendingFinalizers();
// Load the assembly again
Init();
}
...
}
I cannot state the importance of making sure that all references to types in the script assembly must be removed. When integrating into your engines, make sure you test that hot reloading is working properly when you add new lists or objects that hold types from the script assembly. This is because there are no exceptions thrown or errors printed when the assembly fails to unload. Essentially, it will fail silently and no one will be none the wiser until they find out much later.
In a future article, I will discuss a method which you can use to help debug these issues if you ever encounter them.
Build and Reload
With the functions written, let’s actually put them to use. We’ll modify our code to compile the script assembly at the start since Visual Studio no longer builds it for us.
// Application.cpp
namespace Core
{
...
void Application::Run()
{
startScriptEngine();
compileScriptAssembly();
...
}
....
}
Next, we’ll modify our Run() function to do a rebuild and reload when the space button is pressed:
// Application.cpp
namespace Core
{
...
void Application::Run()
{
...
auto reloadScripts = GetFunctionPtr<void(*)(void)>
(
"ScriptAPI",
"ScriptAPI.EngineInterface",
"Reload"
);
// Initialize
init();
addScript(0, "TestScript");
// Load
while (true)
{
if (GetKeyState(VK_ESCAPE) & 0x8000)
break;
if (GetKeyState(VK_SPACE) & 0x8000)
{
compileScriptAssembly();
reloadScripts();
addScript(0, "TestScript");
}
executeUpdate();
}
stopScriptEngine();
}
....
}
Notice that we added the script back? That’s because we removed all scripts when we reloaded the script assembly. For the purposes of this tutorial, we hard-coded the script’s re-addition. However, in a real application, what we would do is to serialize the scripts before reloading the assembly and then deserialize them after that. There’s many ways to serialize but that topic is beyond the scope of this tutorial.
Testing
To make it easier to check if hot reloading is working, let’s make a change to our TestScript.cs. Now that our ManagedScripts project is not within the same Visual Studio solution, we need to open it up separately. Navigate to the ManagedScripts folder in your Visual Studio solution’s root directory and open the ManagedScripts.csproj file.
Look for TestScript.cs in the Solution Explorer and open it up. We’ll get rid of the existing code in Update() to print the number “1” instead.
using ScriptAPI;
public class TestScript : Script
{
public override void Update()
{
Console.Write("1");
}
}
Now that we have all the code added in, let’s give it a spin. To test that hot reloading works, launch the application and give it a few seconds to compile the scripts. You should see a series of 1s being printed. To test this, let’s go into the TestScript.cs file again and change the "1" to a "2".
Press space and watch it recompile before showing a series of 2s being printed instead.
Conclusion
And that’s it! We’ve successfully demonstrated script reloading without too much work. This covers the bulk of the tutorial and you should have fairly workable although bare bones scripting engine. In the next post, we’ll take a look at techniques for debugging the scripting engine and enabling breakpoint debugging for our C# script programmers.
The repository of a working solution based on this tutorial can be found here.
0 Comments