Refactor asset pipeline: new registry, loader, and runtime

Major overhaul of asset system:
- Split assets into source, .gmeta (JSON), and cooked .imported binaries
- Replaced Asset base class; added TextureAsset, TextureLoader
- AssetManager now uses job-based, dependency-aware loading
- Unified IAssetHandler API; removed legacy handler interfaces
- Updated D3D12 allocator and graphics code for new resource model
- Improved error handling, memory management, and GPU upload logic
- Updated docs and removed obsolete code/interfaces
This commit is contained in:
2026-04-18 01:46:37 +09:00
parent 13bf1501e4
commit abd5ad74d5
32 changed files with 4348 additions and 570 deletions

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,310 @@
# GhostEngine Asset Registry — Design Analysis & Recommendations
## 1. Your Current Design at a Glance
Your current approach is **Unreal-style packed binary** (`.gasset`):
```
┌──────────────────────────────────────────────┐
│ AssetMetadata (128 bytes, fixed) │
│ FormatVersion ─ ID ─ TypeID ─ │
│ HandlerVersion ─ DependencyCount ─ │
│ DependenciesOffset ─ SettingsOffset/Size ─ │
│ ContentOffset/Size │
├──────────────────────────────────────────────┤
│ Settings blob (struct → raw bytes) │
├──────────────────────────────────────────────┤
│ Content blob (e.g. ImageContentHeader + raw) │
├──────────────────────────────────────────────┤
│ Dependencies (Guid[]) │
└──────────────────────────────────────────────┘
```
The AssetRegistry maintains an in-memory GUID↔path index by reading the first 20 bytes of every `.gasset` on startup, with a `FileSystemWatcher` for live updates. A planned SQLite backend (`AssetRegistry.Backend.cs`) would persist this catalog.
---
## 2. Unreal vs Unity — The Trade-Off Matrix
| Dimension | Unreal (Packed Binary `.uasset`) | Unity (Raw File + `.meta` sidecar) |
|---|---|---|
| **Source control** | Opaque blobs — merges impossible, diffs useless | Raw files are human-readable; `.meta` is text YAML — mergeable |
| **Import speed** | One file to open per asset | Two opens per asset (source + meta), but meta is tiny |
| **Runtime loading** | One `seek+read` → done (no re-import step) | Must "import" (cook) before runtime loading; raw files are editor-only |
| **Artist iteration** | Must re-import through editor | Can drop a PNG in Explorer & it auto-imports |
| **Dependency tracking** | Embedded in the binary — self-contained | External DB (`.meta` GUIDs + Library/) — can desync |
| **Asset settings versioning** | Binary struct layout is fragile | YAML/JSON → easy to add fields with defaults |
| **Corruption resilience** | One corrupted byte → whole asset lost | Source file is unaffected; re-import fixes derived data |
| **Build pipeline** | Already cooked (or close to it) | Separate cook step needed for builds |
| **Team discoverability** | "What is this .gasset?" → need editor to inspect | "It's a PNG, I can open it anywhere" |
### Key Insight
> Unreal doesn't actually store source data inside `.uasset` for most asset types. Unreal stores the **cooked/processed** representation. The source data (FBX, PSD, etc.) lives outside the engine's asset system — artists use a separate "source art" folder. The `.uasset` is a **derived artifact**, not the source of truth.
Unity's insight was: **leave source files alone, store metadata beside them, and derive everything else into a Library/ cache.** The `.meta` sidecar is tiny (GUID + import settings in YAML), version-control-friendly, and the actual imported data lives in `Library/` (a local, regenerable cache).
---
## 3. Current Design — Issues Found
### 3.1 Binary Settings Are a Versioning Nightmare
```csharp
// TextureAssetHandler — writes settings as raw struct bytes
Unsafe.WriteUnaligned(ref address, settings.Basic);
Unsafe.WriteUnaligned(ref Unsafe.Add(ref address, ...), settings.Advanced);
```
**Problem:** Adding a single field to `BasicSettings`, `AdvancedSettings`, or `SamplerSettings` changes the struct layout. Every existing `.gasset` file becomes unreadable because the byte offsets shift. You have `HandlerVersion` in the metadata, but no migration logic — and you'd need one per handler per version.
> [!CAUTION]
> This is the #1 pain point of the Unreal approach in practice. Epic has dedicated teams managing asset versioning with `FArchive` custom serialization + version tags. For a small team, this is a massive maintenance burden.
### 3.2 Source File Is Destroyed on Import
```csharp
// OnFileSystemOp — line 224
File.Delete(assetPath); // ← deletes the original source file!
```
After import, the source `.png` is deleted and only the `.gasset` remains. If the user wants to change import settings (e.g. switch from BC7 to BC5 for a normal map), they need to find the original source file elsewhere and re-import.
### 3.3 Handler Discovery Is O(N × M) per Call
```csharp
// GetAssetHandlerForExtension — line 326-338
foreach (var handlerType in AppDomain.CurrentDomain.GetAssemblies()
.SelectMany(assembly => assembly.GetTypes())
.Where(type => typeof(IAssetHandler).IsAssignableFrom(type) ...))
```
This scans **every type in every loaded assembly** on each call. It's called from `OnFileSystemOp` (FileSystemWatcher callback — frequent!) and `ImportAssetAsync`. The `_cachedHandler` dictionary helps for repeat loads, but the initial scan is expensive and runs every time a new extension is encountered.
### 3.4 `async void` in FileSystemWatcher Callback
```csharp
private async void OnFileSystemOp(object sender, FileSystemEventArgs e)
```
If `ImportAsync` throws, the exception is swallowed silently (unobserved). `FileSystemWatcher` callbacks should be synchronous (queue work to a channel/queue), or at minimum wrap the body in `try/catch`.
### 3.5 Race Conditions in Path Mapping
```csharp
// ConcurrentDictionary + lock(_pathLock)
_pathToGuid = new ConcurrentDictionary<...>(); // concurrent dict
lock (_pathLock) { _pathToGuid[relativePath] = guid; } // but manually locked
```
You're using `ConcurrentDictionary` but also taking a `Lock` for every access. These two strategies conflict — either use a plain `Dictionary<>` + lock, or use `ConcurrentDictionary` lock-free. Mixing them gives the worst of both: allocation overhead of `ConcurrentDictionary` with the contention of a lock.
### 3.6 Missing Content Hash for Cache Invalidation
The `TextureProcessor` hashes **settings** to build a cache key (`guid_settingsHash.dds`), but doesn't hash the **source content**. If you replace a PNG with a different image of the same name, the stale cache is served because only the settings hash changed (it didn't).
### 3.7 No Version Migration Path
The 128-byte `AssetMetadata` header reserves space for expansion — good! But there's no mechanism to detect "this `.gasset` was written by handler v1 and we're now at v3" and upgrade in place. Currently `HandlerVersion` is written but never read.
---
## 4. Recommendation: Hybrid Architecture
I recommend a **Unity-inspired hybrid** — keep source files untouched, use lightweight sidecar metadata, and produce a separate cooked cache. Here's the concrete design:
### 4.1 Three-Layer Architecture
```
ProjectRoot/
├── Assets/ ← Source files (PNG, FBX, HLSL, ...)
│ ├── Textures/
│ │ ├── hero_diffuse.png ← Source of truth (never modified)
│ │ └── hero_diffuse.png.gmeta ← Sidecar: GUID + import settings (YAML/JSON)
│ └── Models/
│ ├── character.fbx
│ └── character.fbx.gmeta
├── Library/ ← Derived data cache (local, .gitignore'd)
│ ├── AssetDB.sqlite ← Fast GUID↔path + dependency index
│ ├── Imports/ ← Cooked assets (DDS, compiled meshes, etc.)
│ │ ├── <guid>.imported ← Binary cooked data (current .gasset content section)
│ │ └── ...
│ └── Thumbnails/
│ └── <guid>.thumb
└── .ghostignore ← Patterns to exclude from asset scanning
```
### 4.2 `.gmeta` Sidecar File
```yaml
# hero_diffuse.png.gmeta
guid: 0906f4eb-c3f0-431b-bcea-132c88ab0c3f
handler: TextureAssetHandler
handlerVersion: 1
settings:
textureType: Default
textureShape: Texture2D
isSRGB: true
maxSize: 2048
filterMode: Anisotropic
wrapMode: Repeat
generateMipmaps: true
compressionLevel: Normal
# ... full settings tree
dependencies: []
labels: [environment, hero] # optional user tags
```
**Why this is better:**
| Concern | Current `.gasset` | Proposed `.gmeta` |
|---|---|---|
| Add a field | Binary layout breaks | YAML: missing keys → default values |
| Merge conflict | Impossible (binary) | Text merge, trivial |
| Inspect settings | Need editor | Open in any text editor |
| Source file recovery | Destroyed | Untouched, always available |
| Re-import | Need original file | `Library/` rebuild from source + `.gmeta` |
| `git diff` | `Binary files differ` | Readable YAML diff |
### 4.3 SQLite Catalog (`Library/AssetDB.sqlite`)
Replace the in-memory `ConcurrentDictionary<string, Guid>` mapping with an SQLite database (you already planned this in `AssetRegistry.Backend.cs`):
```sql
-- Core asset table
CREATE TABLE assets (
guid BLOB PRIMARY KEY, -- 16 bytes, exactly sizeof(Guid)
path TEXT NOT NULL, -- relative path to .gmeta
handler TEXT NOT NULL, -- handler type name
content_hash TEXT, -- xxHash64 of source file bytes
settings_hash TEXT, -- xxHash64 of import settings
imported_at INTEGER, -- unix timestamp of last successful import
UNIQUE(path)
);
-- Dependency edges (forward: asset → dependency)
CREATE TABLE dependencies (
from_guid BLOB NOT NULL REFERENCES assets(guid),
to_guid BLOB NOT NULL REFERENCES assets(guid),
PRIMARY KEY (from_guid, to_guid)
);
-- Reverse index for "what depends on me?" queries
CREATE INDEX idx_dep_reverse ON dependencies(to_guid);
-- Full-text search on asset paths and labels
CREATE VIRTUAL TABLE assets_fts USING fts5(path, labels);
```
**Startup becomes:**
1. Open SQLite DB → instant GUID↔path from indexed table
2. Diff `Assets/` tree vs DB → find stale/new/deleted `.gmeta` files
3. Queue incremental re-imports only for changed assets
This is **dramatically faster** than scanning every `.gasset` header on disk (your current `LoadExistingAssets`).
### 4.4 Import Pipeline
```
Source File Changed
FileSystemWatcher
├─── No .gmeta exists? → Generate one (new GUID, default settings)
Hash source + settings
├─── Hash matches DB? → Skip (already imported)
Queue ImportJob to background channel
ImportWorker (background thread pool)
├── Read source file
├── Run handler pipeline (e.g. NVTT compress)
├── Write Library/Imports/<guid>.imported
├── Update SQLite (content_hash, settings_hash, imported_at)
└── Fire AssetChanged event on main thread
```
### 4.5 Handler Registration — Build Once, Cache Forever
Replace the per-call assembly scan with a startup-once TypeCache approach (you already have this pattern in the engine):
```csharp
// Startup: build lookup tables once
Dictionary<string, Type> _extensionToHandler; // ".png" → typeof(TextureAssetHandler)
Dictionary<Guid, Type> _typeIdToHandler; // TypeGuid → handler type
// Populated once via TypeCache / assembly attribute scan at editor startup
foreach (var type in TypeCache.GetTypesWithAttribute<CustomAssetHandlerAttribute>())
{
var attr = type.GetCustomAttribute<CustomAssetHandlerAttribute>();
_typeIdToHandler[new Guid(attr.ID)] = type;
foreach (var ext in attr.SupportedExtensions)
_extensionToHandler[ext] = type;
}
```
---
## 5. What to Keep from Your Current Design
Your design has several things done well:
| Element | Verdict |
|---|---|
| `AssetMetadata` fixed-size header with offsets | ✅ Keep for the cooked `.imported` files — great for O(1) seeks |
| `Handle<GPUTexture>` on `TextureAsset` | ✅ Clean separation of asset data vs GPU resource handle |
| `WeakReference<Asset>` cache in registry | ✅ Elegant — auto-evicts when nothing holds the asset |
| `IAssetHandler` / `IImportableAssetHandler` split | ✅ Good separation (some assets are import-only, e.g. shaders compiled differently) |
| `AssetReference` with internal/external encoding | ✅ Clever — keeps sub-asset refs compact |
| `TextureProcessor` cache with settings hash | ✅ Great idea, just needs content hash too |
| `Result<T>` return pattern | ✅ Consistent with the rest of GhostEngine |
---
## 6. Summary Recommendation
```
┌────────────────────────────────────────────────────────────┐
│ RECOMMENDED APPROACH │
│ │
│ Source files → untouched, checked into git │
│ .gmeta sidecars → GUID + settings (YAML), in git │
│ Library/ → derived cache, .gitignored │
│ AssetDB.sqlite → fast GUID↔path index │
│ Imports/*.imported → cooked binary (your AssetMetadata │
│ header + content, no settings) │
│ │
│ Binary format → for cooked data only, not settings │
│ Settings format → YAML/JSON in .gmeta (human + VCS) │
│ Handler discovery → one-time TypeCache at startup │
│ Watcher callbacks → queue to Channel<T>, no async void │
└────────────────────────────────────────────────────────────┘
```
This gives you:
- **Unreal's runtime performance** (cooked binary in Library/ → single seek+read)
- **Unity's artist workflow** (drop files in Assets/, settings are readable text)
- **Clean version control** (text `.gmeta` files merge cleanly)
- **Resilient re-import** (source is never touched; Library/ is regenerable)
- **Zero startup cost** (SQLite index instead of scanning thousands of file headers)
---
## 7. Open Questions for You
1. **Do you want `.gmeta` in YAML, JSON, or a custom text format?** YAML is more compact and human-friendly, but adds a parser dependency. JSON is built into .NET but more verbose. A custom format is more work.
2. **Should the cooked `.imported` files keep the 128-byte `AssetMetadata` header?** It's useful for validation on load, but since SQLite already knows the GUID and handler, you could simplify the binary format.
3. **Do you want hot-reload of import settings?** (Changing `.gmeta` → auto re-import and refresh live asset in editor.) Your current `WeakReference<Asset>` + `RefreshAsync` already supports this.
4. **How do you want to handle the `Library/` on first clone?** Options: (a) full re-import from source, (b) share a pre-built Library via LFS, (c) asset server that caches imports.

File diff suppressed because it is too large Load Diff

View File

@@ -1,69 +0,0 @@
using Ghost.Editor.Core.Contracts;
namespace Ghost.Editor.Core.AssetHandler;
public abstract class Asset
{
public Guid ID
{
get;
}
public abstract Guid TypeID
{
get;
}
protected Asset(Guid id)
{
ID = id;
}
public virtual ValueTask RefreshAsync(IAssetRegistry db, CancellationToken token = default)
{
return ValueTask.CompletedTask;
}
}
public readonly struct AssetReference : IEquatable<AssetReference>
{
private readonly int _value;
/// <summary>
/// The index of the asset in the dependency list.
/// </summary>
public int Index
{
get => Math.Abs(_value) - 1;
}
public static AssetReference Null => default;
public readonly bool IsInternal => _value >= 0;
public readonly bool IsExternal => _value < 0;
public bool Equals(AssetReference other)
{
return _value == other._value;
}
public override int GetHashCode()
{
return _value.GetHashCode();
}
public override bool Equals(object? obj)
{
return obj is AssetReference reference && Equals(reference);
}
public static bool operator ==(AssetReference left, AssetReference right)
{
return left.Equals(right);
}
public static bool operator !=(AssetReference left, AssetReference right)
{
return !(left == right);
}
}

View File

@@ -1,5 +1,4 @@
using Ghost.Core; using Ghost.Core;
using Ghost.Editor.Core.Contracts;
namespace Ghost.Editor.Core.AssetHandler; namespace Ghost.Editor.Core.AssetHandler;
@@ -15,40 +14,9 @@ public interface IAssetExportOptions;
public interface IAssetHandler public interface IAssetHandler
{ {
ValueTask<Result<Asset>> LoadAsync(Stream sourceStream, Guid id, IAssetRegistry assetRegistry, CancellationToken token = default); bool CanExport => false;
ValueTask<Result> SaveAsync(Asset asset, Stream targetStream, IAssetRegistry assetRegistry, CancellationToken token = default);
}
public interface IImportableAssetHandler : IAssetHandler
{
IAssetSettings? CreateDefaultSettings(); IAssetSettings? CreateDefaultSettings();
ValueTask<Result> ImportAsync(Stream sourceStream, Stream targetStream, Guid id, IAssetSettings? settings, CancellationToken token = default); ValueTask<Result> ImportAsync(Stream sourceStream, Stream targetStream, Guid id, IAssetSettings? settings, CancellationToken token = default);
}
public interface IExportableAssetHandler : IAssetHandler
{
ValueTask<Result> ExportAsync(Stream assetStream, Stream targetStream, IAssetExportOptions? options, CancellationToken token = default); ValueTask<Result> ExportAsync(Stream assetStream, Stream targetStream, IAssetExportOptions? options, CancellationToken token = default);
} }
public static class AssetHandlerExtensions
{
public static async ValueTask<Result> ImportAsync(this IImportableAssetHandler handler, string sourceFilePath, string targetFilePath, Guid id, IAssetSettings? settings = null, CancellationToken token = default)
{
await using var sourceStream = new FileStream(sourceFilePath, FileMode.Open, FileAccess.Read, FileShare.Read);
await using var targetStream = new FileStream(targetFilePath, FileMode.Create, FileAccess.Write, FileShare.None);
return await handler.ImportAsync(sourceStream, targetStream, id, settings, token);
}
public static async ValueTask<Result> ExportAsync(this IExportableAssetHandler handler, string assetFilePath, string targetFilePath, IAssetExportOptions? options, CancellationToken token = default)
{
await using var assetStream = new FileStream(assetFilePath, FileMode.Open, FileAccess.Read, FileShare.Read);
await using var targetStream = new FileStream(targetFilePath, FileMode.Create, FileAccess.Write, FileShare.None);
return await handler.ExportAsync(assetStream, targetStream, options, token);
}
public static async ValueTask<Result<Asset>> LoadAsync(this IAssetHandler handler, string assetFilePath, Guid id, IAssetRegistry assetDatabase, CancellationToken token = default)
{
await using var sourceStream = new FileStream(assetFilePath, FileMode.Open, FileAccess.Read, FileShare.Read);
return await handler.LoadAsync(sourceStream, id, assetDatabase, token);
}
}

View File

@@ -1,4 +1,5 @@
using Ghost.Editor.Core.Contracts; using Ghost.Editor.Core.Contracts;
using Ghost.Engine.AssetLoader;
namespace Ghost.Editor.Core.AssetHandler; namespace Ghost.Editor.Core.AssetHandler;

View File

@@ -1,5 +1,5 @@
using Ghost.Core; using Ghost.Core;
using Ghost.Editor.Core.Contracts; using Ghost.Engine.AssetLoader;
using Ghost.Graphics.RHI; using Ghost.Graphics.RHI;
using ImageMagick; using ImageMagick;
using System.Runtime.InteropServices; using System.Runtime.InteropServices;
@@ -46,55 +46,6 @@ public enum MipmapFilter : uint
MitchellNetravali MitchellNetravali
} }
public class TextureAsset : Asset
{
internal const string _TYPE_ID = "0906F4EB-C3F0-431B-BCEA-132C88AB0C3F";
internal static readonly Guid s_typeGuid = Guid.Parse(_TYPE_ID);
private readonly byte[] _textureData;
private readonly uint _width;
private readonly uint _height;
private readonly uint _depth;
private readonly uint _colorComponents;
public override Guid TypeID => s_typeGuid;
/// <summary>
/// Gets the raw texture data in a compressed format.
/// </summary>
public ReadOnlyMemory<byte> TextureData => _textureData;
/// <summary>
/// Gets the width of the texture in pixels.
/// </summary>
public uint Width => _width;
/// <summary>
/// Gets the height of the texture in pixels.
/// </summary>
public uint Height => _height;
/// <summary>
/// Gets the bit depth of the texture.
/// </summary>
public uint Depth => _depth;
/// <summary>
/// Gets the number of color components in the texture.
/// </summary>
public uint ColorComponents => _colorComponents;
internal TextureAsset(byte[] data, ImageContentHeader header, Guid id)
: base(id)
{
_textureData = data;
_width = header.width;
_height = header.height;
_depth = header.depth;
_colorComponents = header.colorComponents;
}
}
public class TextureAssetSettings : IAssetSettings public class TextureAssetSettings : IAssetSettings
{ {
public struct BasicSettings() public struct BasicSettings()
@@ -242,65 +193,14 @@ public class TextureAssetSettings : IAssetSettings
} = new SamplerSettings(); } = new SamplerSettings();
} }
[StructLayout(LayoutKind.Sequential, Size = 64)] // Leave extra space for future expansion without breaking compatibility [CustomAssetHandler(TextureAsset.TYPE_ID, [".png", ".jpg", ".jpeg", ".tga", ".bmp", ".hdr"], 1)]
internal struct ImageContentHeader internal class TextureAssetHandler : IAssetHandler
{
public uint width;
public uint height;
public uint depth;
public uint colorComponents;
}
[CustomAssetHandler(TextureAsset._TYPE_ID, [ ".png", ".jpg", ".jpeg", ".tga", ".bmp", ".hdr" ], 1)]
internal class TextureAssetHandler : IImportableAssetHandler
{ {
public IAssetSettings? CreateDefaultSettings() public IAssetSettings? CreateDefaultSettings()
{ {
return new TextureAssetSettings(); return new TextureAssetSettings();
} }
public async ValueTask<Result<Asset>> LoadAsync(Stream sourceStream, Guid id, IAssetRegistry assetRegistry, CancellationToken token = default)
{
try
{
// FIX: Should the sourceStream be the stream of the imported file or the raw asset file?
// Or should we change our paramemters to inlcude more information and let each handler decide how to load the asset?
// The problem of a single sourceStream is, for example, for texture assets, we don't even need to read the ".png" file at all,
// but for some other asset types, we may don't even have imported intermediate files at all.
// var path = assetRegistry.GetAssetPath(id);
// if (string.IsNullOrEmpty(path))
// {
// return Result.Failure("Asset path not found in registry.");
// }
//
// var metadataPath = AssetMetaIO.GetMetaPath(path);
// var meta = await AssetMetaIO.ReadAsync(metadataPath, token).ConfigureAwait(false);
// Logger.DebugAssert(meta != null, $"Missing or invalid metadata for asset at {path}");
var header = new ImageContentHeader();
sourceStream.ReadExactly(MemoryMarshal.AsBytes(new Span<ImageContentHeader>(ref header)));
var imageDataSize = (int)(sourceStream.Length - sourceStream.Position);
var imageData = new byte[imageDataSize];
await sourceStream.ReadExactlyAsync(imageData, token).ConfigureAwait(false);
var asset = new TextureAsset(imageData, header, id);
return asset;
}
catch (Exception ex)
{
return Result.Failure($"Failed to load texture asset: {ex.Message}");
}
}
public ValueTask<Result> SaveAsync(Asset asset, Stream targetStream, IAssetRegistry assetRegistry, CancellationToken token = default)
{
throw new NotImplementedException();
}
public async ValueTask<Result> ImportAsync(Stream sourceStream, Stream targetStream, Guid id, IAssetSettings? settings, CancellationToken token = default) public async ValueTask<Result> ImportAsync(Stream sourceStream, Stream targetStream, Guid id, IAssetSettings? settings, CancellationToken token = default)
{ {
try try
@@ -309,20 +209,22 @@ internal class TextureAssetHandler : IImportableAssetHandler
using var image = new MagickImage(sourceStream); using var image = new MagickImage(sourceStream);
var bytes = image.ToByteArray(); var bytes = image.ToByteArray();
await TextureProcessor.CompressToCacheAsync(EditorApplication.LibraryFolderPath, id, bytes, image.Width, image.Height, image.Depth, textureSettings, token) var (path, mip) = await TextureProcessor.CompressToCacheAsync(EditorApplication.ImportsFolderPath, id, bytes, image.Width, image.Height, image.Depth, textureSettings, token)
.ConfigureAwait(false); .ConfigureAwait(false);
targetStream.Seek(0, SeekOrigin.Begin); targetStream.Seek(0, SeekOrigin.Begin);
var contentHeader = new ImageContentHeader var contentHeader = new TextureContentHeader
{ {
width = image.Width, width = image.Width,
height = image.Height, height = image.Height,
depth = image.Depth, depth = image.Depth,
colorComponents = image.ChannelCount colorComponents = image.ChannelCount,
mipLevels = (uint)mip,
dimension = (int)TextureDimension.Texture2D // TODO: Implement dimension calculation
}; };
targetStream.Write(MemoryMarshal.AsBytes(new Span<ImageContentHeader>(ref contentHeader))); targetStream.Write(MemoryMarshal.AsBytes(new Span<TextureContentHeader>(ref contentHeader)));
await targetStream.WriteAsync(bytes, token).ConfigureAwait(false); await targetStream.WriteAsync(bytes, token).ConfigureAwait(false);
await targetStream.FlushAsync(token).ConfigureAwait(false); await targetStream.FlushAsync(token).ConfigureAwait(false);
@@ -334,4 +236,9 @@ internal class TextureAssetHandler : IImportableAssetHandler
return Result.Failure($"Failed to import texture asset: {ex.Message}"); return Result.Failure($"Failed to import texture asset: {ex.Message}");
} }
} }
public ValueTask<Result> ExportAsync(Stream assetStream, Stream targetStream, IAssetExportOptions? options, CancellationToken token = default)
{
return ValueTask.FromResult(Result.Failure("Exporting texture assets is not supported yet."));
}
} }

View File

@@ -33,6 +33,8 @@ internal static class TextureProcessor
private readonly TextureAssetSettings _settings; private readonly TextureAssetSettings _settings;
private readonly TaskCompletionSource _completionSource; private readonly TaskCompletionSource _completionSource;
public int mipmapCount;
public Task Task => _completionSource.Task; public Task Task => _completionSource.Task;
public NvttPipelineTask(string outputPath, byte[] image, uint width, uint height, uint depth, TextureAssetSettings settings) public NvttPipelineTask(string outputPath, byte[] image, uint width, uint height, uint depth, TextureAssetSettings settings)
@@ -119,7 +121,6 @@ internal static class TextureProcessor
var nvttFilter = SelectMipmapFilter(_settings.Advanced.MipmapFilter); var nvttFilter = SelectMipmapFilter(_settings.Advanced.MipmapFilter);
int mipmapCount;
if (!_settings.Advanced.GenerateMipmaps) if (!_settings.Advanced.GenerateMipmaps)
{ {
mipmapCount = 1; mipmapCount = 1;
@@ -162,23 +163,19 @@ internal static class TextureProcessor
} }
} }
private const string _TEXTURE_CACHE_SUBFOLDER = "TextureCache"; public static async ValueTask<(string cachePath, int mipmapCount)> CompressToCacheAsync(string cachesFolderPath, Guid assetId, byte[] image, uint width, uint height, uint depth, TextureAssetSettings settings, CancellationToken cancellationToken)
public static async ValueTask<string> CompressToCacheAsync(string cachesFolderPath, Guid assetId, byte[] image, uint width, uint height, uint depth, TextureAssetSettings settings, CancellationToken cancellationToken)
{ {
var cacheDir = Path.Combine(cachesFolderPath, _TEXTURE_CACHE_SUBFOLDER);
Directory.CreateDirectory(cacheDir);
var settingsHash = ComputeSettingsHash(settings); var settingsHash = ComputeSettingsHash(settings);
var cacheFileName = $"{assetId:N}_{settingsHash:X16}.dds"; var cacheFileName = $"texturecache_{assetId:N}_{settingsHash:X16}.dds";
var cachePath = Path.Combine(cacheDir, cacheFileName); var cachePath = Path.Combine(cachesFolderPath, cacheFileName);
if (File.Exists(cachePath)) if (File.Exists(cachePath))
{ {
return cachePath; // TODO: Implement mipmap count retrieval from existing cache file
return (cachePath, 0);
} }
foreach (var stale in Directory.EnumerateFiles(cacheDir, $"{assetId:N}_*.dds")) foreach (var stale in Directory.EnumerateFiles(cachesFolderPath, $"texturecache_{assetId:N}_*.dds"))
{ {
File.Delete(stale); File.Delete(stale);
} }
@@ -187,7 +184,7 @@ internal static class TextureProcessor
ThreadPool.UnsafeQueueUserWorkItem(workItem, true); ThreadPool.UnsafeQueueUserWorkItem(workItem, true);
await workItem.Task.WaitAsync(cancellationToken).ConfigureAwait(false); await workItem.Task.WaitAsync(cancellationToken).ConfigureAwait(false);
return cachePath; return (cachePath, workItem.mipmapCount);
} }
private static NvttFormat SelectFormat(TextureAssetSettings settings) private static NvttFormat SelectFormat(TextureAssetSettings settings)

View File

@@ -1,6 +1,6 @@
using Ghost.Core; using Ghost.Core;
using Ghost.Editor.Core.AssetHandler;
using Ghost.Editor.Core.Services; using Ghost.Editor.Core.Services;
using Ghost.Engine.AssetLoader;
namespace Ghost.Editor.Core.Contracts; namespace Ghost.Editor.Core.Contracts;

View File

@@ -1,8 +1,8 @@
using System.Collections.Concurrent;
using System.Reflection;
using Ghost.Core; using Ghost.Core;
using Ghost.Editor.Core.AssetHandler; using Ghost.Editor.Core.AssetHandler;
using Ghost.Editor.Core.Contracts; using Ghost.Editor.Core.Contracts;
using Ghost.Engine.AssetLoader;
using System.Collections.Concurrent;
namespace Ghost.Editor.Core.Services; namespace Ghost.Editor.Core.Services;
@@ -157,7 +157,7 @@ internal sealed class AssetRegistry : IAssetRegistry, IDisposable
var ext = Path.GetExtension(relativePath); var ext = Path.GetExtension(relativePath);
var handler = _handlerRegistry.GetByExtension(ext); var handler = _handlerRegistry.GetByExtension(ext);
var importable = handler as IImportableAssetHandler; var importable = handler as IAssetHandler;
var metaPath = AssetMetaIO.GetMetaPath(fullPath); var metaPath = AssetMetaIO.GetMetaPath(fullPath);
if (File.Exists(metaPath)) if (File.Exists(metaPath))
@@ -169,7 +169,7 @@ internal sealed class AssetRegistry : IAssetRegistry, IDisposable
var meta = new AssetMeta var meta = new AssetMeta
{ {
Guid = Guid.NewGuid(), Guid = Guid.NewGuid(),
HandlerTypeId = handlerTypeId is string str? Guid.Parse(str) : null, HandlerTypeId = handlerTypeId is string str ? Guid.Parse(str) : null,
HandlerVersion = 1, HandlerVersion = 1,
Settings = importable?.CreateDefaultSettings() Settings = importable?.CreateDefaultSettings()
}; };

View File

@@ -1,8 +1,8 @@
using System.Threading.Channels;
using Ghost.Core; using Ghost.Core;
using Ghost.Editor.Core.AssetHandler; using Ghost.Editor.Core.AssetHandler;
using System.Security.Cryptography; using System.Security.Cryptography;
using System.Text.Json; using System.Text.Json;
using System.Threading.Channels;
namespace Ghost.Editor.Core.Services; namespace Ghost.Editor.Core.Services;
@@ -115,7 +115,7 @@ internal sealed class ImportCoordinator : IDisposable
} }
var importResult = Result.Success(); var importResult = Result.Success();
if (handler is IImportableAssetHandler importable) if (handler is IAssetHandler importable)
{ {
// TODO: This should be handled by EditorApplication. // TODO: This should be handled by EditorApplication.
var importsDir = Path.Combine(_libraryRoot, "Imports"); var importsDir = Path.Combine(_libraryRoot, "Imports");

View File

@@ -20,7 +20,7 @@
<ItemGroup> <ItemGroup>
<PackageReference Include="Misaki.HighPerformance" Version="1.0.7" /> <PackageReference Include="Misaki.HighPerformance" Version="1.0.7" />
<PackageReference Include="Misaki.HighPerformance.Jobs" Version="1.5.9" /> <PackageReference Include="Misaki.HighPerformance.Jobs" Version="1.6.1" />
<PackageReference Include="Misaki.HighPerformance.LowLevel" Version="1.6.13"> <PackageReference Include="Misaki.HighPerformance.LowLevel" Version="1.6.13">
<PrivateAssets>all</PrivateAssets> <PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets> <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>

View File

@@ -1,9 +1,25 @@
using Misaki.HighPerformance.LowLevel; using Misaki.HighPerformance.LowLevel;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
namespace Ghost.Core; namespace Ghost.Core;
public enum Error
{
None,
NotFound,
InvalidArgument,
InvalidState,
InternalError,
PermissionDenied,
NotSupported,
OutOfMemory,
Timeout,
Cancelled,
UnknownError,
Success = None,
}
public readonly struct Result public readonly struct Result
{ {
private readonly string? _message; private readonly string? _message;
@@ -126,25 +142,8 @@ public readonly struct Result<T>
public static implicit operator bool(Result<T> result) => result.IsSuccess; public static implicit operator bool(Result<T> result) => result.IsSuccess;
} }
public enum Error : byte
{
None,
NotFound,
InvalidArgument,
InvalidState,
InternalError,
PermissionDenied,
NotSupported,
OutOfMemory,
Timeout,
Cancelled,
UnknownError,
Success = None,
}
public readonly struct Result<T, E> public readonly struct Result<T, E>
where E : struct, Enum where E : struct
{ {
private readonly T _value; private readonly T _value;
private readonly E _error; private readonly E _error;
@@ -203,7 +202,7 @@ public readonly struct Result<T, E>
} }
public readonly ref struct RefResult<T, E> public readonly ref struct RefResult<T, E>
where E : struct, Enum where E : struct
{ {
private readonly ref T _value; private readonly ref T _value;
private readonly E _error; private readonly E _error;
@@ -261,70 +260,30 @@ public readonly ref struct RefResult<T, E>
public static implicit operator bool(RefResult<T, E> result) => result.IsSuccess; public static implicit operator bool(RefResult<T, E> result) => result.IsSuccess;
} }
[AsyncMethodBuilder(typeof(AsyncValueTaskMethodBuilder))]
public readonly struct ResultTask
{
private readonly ValueTask<Result> _task;
public ResultTask(ValueTask<Result> task)
{
_task = task;
}
public ValueTaskAwaiter<Result> GetAwaiter() => _task.GetAwaiter();
public ValueTask<Result> AsValueTask() => _task;
public Task<Result> AsTask() => _task.AsTask();
public static implicit operator ResultTask(ValueTask<Result> task) => new ResultTask(task);
public static implicit operator ValueTask<Result>(ResultTask resultTask) => resultTask._task;
}
[AsyncMethodBuilder(typeof(AsyncValueTaskMethodBuilder))]
public readonly struct ResultTask<T>
{
private readonly ValueTask<Result<T>> _task;
public ResultTask(ValueTask<Result<T>> task)
{
_task = task;
}
public ValueTaskAwaiter<Result<T>> GetAwaiter() => _task.GetAwaiter();
public ValueTask<Result<T>> AsValueTask() => _task;
public Task<Result<T>> AsTask() => _task.AsTask();
public static implicit operator ResultTask<T>(ValueTask<Result<T>> task) => new ResultTask<T>(task);
public static implicit operator ValueTask<Result<T>>(ResultTask<T> resultTask) => resultTask._task;
}
[AsyncMethodBuilder(typeof(AsyncValueTaskMethodBuilder))]
public readonly struct ResultTask<T, E>
where E : struct, Enum
{
private readonly ValueTask<Result<T, E>> _task;
public ResultTask(ValueTask<Result<T, E>> task)
{
_task = task;
}
public ValueTaskAwaiter<Result<T, E>> GetAwaiter() => _task.GetAwaiter();
public ValueTask<Result<T, E>> AsValueTask() => _task;
public Task<Result<T, E>> AsTask() => _task.AsTask();
public static implicit operator ResultTask<T, E>(ValueTask<Result<T, E>> task) => new ResultTask<T, E>(task);
public static implicit operator ValueTask<Result<T, E>>(ResultTask<T, E> resultTask) => resultTask._task;
}
public static class ResultExtensions public static class ResultExtensions
{ {
extension(Error error) extension(Error error)
{ {
public bool IsSuccess => error == Error.None; public bool IsSuccess => error == Error.None;
public bool IsFailure => error != Error.None; public bool IsFailure => error != Error.None;
public static Error FromHResult(int hr)
{
return hr switch
{
0 => Error.None,
unchecked((int)0x80070002) => Error.NotFound,
unchecked((int)0x80070057) => Error.InvalidArgument,
unchecked((int)0x8007139F) => Error.InvalidState,
unchecked((int)0x80004005) => Error.InternalError,
unchecked((int)0x80070005) => Error.PermissionDenied,
unchecked((int)0x80004001) => Error.NotSupported,
unchecked((int)0x8007000E) => Error.OutOfMemory,
unchecked((int)0x800705B4) => Error.Timeout,
unchecked((int)0x800704C7) => Error.Cancelled,
_ => Error.UnknownError
};
}
} }
public static void ThrowIfFailed(this Error result, [CallerArgumentExpression(nameof(result))] string? op = null) public static void ThrowIfFailed(this Error result, [CallerArgumentExpression(nameof(result))] string? op = null)
@@ -354,7 +313,7 @@ public static class ResultExtensions
} }
public static T GetValueOrThrow<T, E>(this Result<T, E> result, [CallerArgumentExpression(nameof(result))] string? op = null) public static T GetValueOrThrow<T, E>(this Result<T, E> result, [CallerArgumentExpression(nameof(result))] string? op = null)
where E : struct, Enum where E : struct
{ {
if (!result.IsSuccess) if (!result.IsSuccess)
{ {
@@ -365,7 +324,7 @@ public static class ResultExtensions
} }
public static ref T GetValueOrThrow<T, E>(this RefResult<T, E> result, [CallerArgumentExpression(nameof(result))] string? op = null) public static ref T GetValueOrThrow<T, E>(this RefResult<T, E> result, [CallerArgumentExpression(nameof(result))] string? op = null)
where E : struct, Enum where E : struct
{ {
if (!result.IsSuccess) if (!result.IsSuccess)
{ {
@@ -381,13 +340,13 @@ public static class ResultExtensions
} }
public static T? GetValueOrDefault<T, E>(this Result<T, E> result, T? defaultValue = default) public static T? GetValueOrDefault<T, E>(this Result<T, E> result, T? defaultValue = default)
where E : struct, Enum where E : struct
{ {
return result.IsSuccess ? result.Value : defaultValue; return result.IsSuccess ? result.Value : defaultValue;
} }
public static ref T GetValueOrDefault<T, E>(this RefResult<T, E> result) public static ref T GetValueOrDefault<T, E>(this RefResult<T, E> result)
where E : struct, Enum where E : struct
{ {
return ref result.IsSuccess ? ref result.Value : ref Unsafe.NullRef<T>(); return ref result.IsSuccess ? ref result.Value : ref Unsafe.NullRef<T>();
} }
@@ -438,7 +397,7 @@ public static class ResultExtensions
} }
public static Result<T, E> OnSuccess<T, E>(this Result<T, E> result, Action<T> action) public static Result<T, E> OnSuccess<T, E>(this Result<T, E> result, Action<T> action)
where E : struct, Enum where E : struct
{ {
if (result.IsSuccess) if (result.IsSuccess)
{ {
@@ -469,7 +428,7 @@ public static class ResultExtensions
} }
public static Result<T, E> OnFailed<T, E>(this Result<T, E> result, Action<E> action) public static Result<T, E> OnFailed<T, E>(this Result<T, E> result, Action<E> action)
where E : struct, Enum where E : struct
{ {
if (result.IsFailure) if (result.IsFailure)
{ {
@@ -500,7 +459,7 @@ public static class ResultExtensions
} }
public static Result<U, E> Then<T, U, E>(this Result<T, E> result, Func<T, Result<U, E>> func) public static Result<U, E> Then<T, U, E>(this Result<T, E> result, Func<T, Result<U, E>> func)
where E : struct, Enum where E : struct
{ {
if (result.IsFailure) if (result.IsFailure)
{ {
@@ -535,7 +494,7 @@ public static class ResultExtensions
} }
public static U Match<T, U, E>(this Result<T, E> result, Func<T, U> onSuccess, Func<E, U> onFailure) public static U Match<T, U, E>(this Result<T, E> result, Func<T, U> onSuccess, Func<E, U> onFailure)
where E : struct, Enum where E : struct
{ {
if (result.IsSuccess) if (result.IsSuccess)
{ {

View File

@@ -98,6 +98,58 @@ public unsafe ref struct SpanWriter
} }
} }
public unsafe struct BufferReader
{
private readonly byte* _buffer;
private readonly nuint _size;
private byte* _position;
public readonly byte* Position => _position;
public nuint Offset
{
readonly get => (nuint)(_buffer + (_position - _buffer));
set => _position = _buffer + value;
}
public BufferReader(byte* buffer, nuint size)
{
_buffer = buffer;
_size = size;
_position = _buffer;
}
public T Read<T>()
where T : unmanaged
{
var value = *(T*)_position;
_position += (nuint)sizeof(T);
return value;
}
public ReadOnlySpan<T> ReadSpan<T>(int length)
where T : unmanaged
{
length = Math.Min(length, (int)((nuint)(_buffer + _size - _position) / (nuint)sizeof(T)));
var size = sizeof(T) * length;
var span = new ReadOnlySpan<T>(_position, length);
_position += (nuint)size;
return span;
}
public ReadOnlySpan<T> ReadToEnd<T>()
where T : unmanaged
{
var span = new ReadOnlySpan<T>(_position, (int)(_buffer + _size - _position));
_position += (nuint)(span.Length * sizeof(T));
return span;
}
}
public unsafe ref struct SpanReader public unsafe ref struct SpanReader
{ {
private readonly Span<byte> _buffer; private readonly Span<byte> _buffer;

View File

@@ -1,5 +1,7 @@
using Misaki.HighPerformance.LowLevel.Buffer;
using Misaki.HighPerformance.LowLevel.Collections.Contracts; using Misaki.HighPerformance.LowLevel.Collections.Contracts;
using System.Buffers; using System.Buffers;
using MemoryHandle = System.Buffers.MemoryHandle;
namespace Ghost.Core.Utilities; namespace Ghost.Core.Utilities;
@@ -26,6 +28,11 @@ public unsafe class NativeMemoryManager<T> : MemoryManager<T>
return new NativeMemoryManager<T>((T*)collection.GetUnsafePtr(), collection.Count); return new NativeMemoryManager<T>((T*)collection.GetUnsafePtr(), collection.Count);
} }
public static NativeMemoryManager<T> FromMemoryBlock(MemoryBlock memoryBlock, int start, int length)
{
return new NativeMemoryManager<T>((T*)memoryBlock.GetUnsafePtr() + start, length);
}
public override Span<T> GetSpan() public override Span<T> GetSpan()
{ {
return new Span<T>(_pointer, _length); return new Span<T>(_pointer, _length);

View File

@@ -0,0 +1,154 @@
using Ghost.Core;
using Ghost.Graphics.RHI;
using Misaki.HighPerformance.LowLevel;
using Misaki.HighPerformance.LowLevel.Buffer;
using System.Runtime.InteropServices;
namespace Ghost.Engine.AssetLoader;
public abstract class Asset : IResourceReleasable
{
private bool _disposed;
public Guid ID
{
get;
}
public abstract AssetType Type
{
get;
}
protected Asset(Guid id)
{
ID = id;
}
protected virtual void Release(IResourceDatabase resourceDatabase)
{
}
public void ReleaseResource(IResourceDatabase database)
{
if (_disposed)
{
return;
}
Release(database);
_disposed = true;
}
}
public readonly struct AssetReference : IEquatable<AssetReference>
{
private readonly int _value;
/// <summary>
/// The index of the asset in the dependency list.
/// </summary>
public int Index
{
get => Math.Abs(_value) - 1;
}
public static AssetReference Null => default;
public readonly bool IsInternal => _value >= 0;
public readonly bool IsExternal => _value < 0;
public bool Equals(AssetReference other)
{
return _value == other._value;
}
public override int GetHashCode()
{
return _value.GetHashCode();
}
public override bool Equals(object? obj)
{
return obj is AssetReference reference && Equals(reference);
}
public static bool operator ==(AssetReference left, AssetReference right)
{
return left.Equals(right);
}
public static bool operator !=(AssetReference left, AssetReference right)
{
return !(left == right);
}
}
[StructLayout(LayoutKind.Sequential, Size = 64)] // Leave extra space for future expansion without breaking compatibility
public struct TextureContentHeader
{
public uint width;
public uint height;
public uint depth;
public uint mipLevels;
public uint dimension; // 1 for 1D, 2 for 2D, 3 for 3D
public uint colorComponents;
}
public class TextureAsset : Asset
{
private MemoryBlock _textureData;
private readonly uint _width;
private readonly uint _height;
private readonly uint _depth;
private readonly uint _colorComponents;
private readonly uint _mipLevels;
private readonly uint _dimension;
private Handle<GPUTexture> _textureHandle;
public override AssetType Type => AssetType.Texture;
public uint Width => _width;
public uint Height => _height;
public uint Depth => _depth;
public uint MipLevels => _mipLevels;
public uint Dimension => _dimension;
public uint ColorComponents => _colorComponents;
public Handle<GPUTexture> TextureHandle => _textureHandle;
internal TextureAsset([OwnershipTransfer] ref MemoryBlock data, TextureContentHeader header, Guid id)
: base(id)
{
_textureData = data;
_width = header.width;
_height = header.height;
_depth = header.depth;
_mipLevels = header.mipLevels;
_dimension = header.dimension;
_colorComponents = header.colorComponents;
}
internal void SetTextureHandle(Handle<GPUTexture> handle, bool disposeCPUData = true)
{
_textureHandle = handle;
if (disposeCPUData)
{
_textureData.Dispose();
}
}
public ReadOnlySpan<T> GeData<T>()
where T : unmanaged
{
return _textureData.AsSpan<T>();
}
protected override void Release(IResourceDatabase resourceDatabase)
{
_textureData.Dispose();
resourceDatabase.ReleaseResource(_textureHandle.AsResource());
}
}

View File

@@ -0,0 +1,42 @@
using Ghost.Core;
using Ghost.Core.Utilities;
using Misaki.HighPerformance.LowLevel.Buffer;
using Misaki.HighPerformance.LowLevel.Utilities;
using System.Runtime.InteropServices;
namespace Ghost.Engine.AssetLoader;
internal sealed class TextureLoader : IRuntimeAssetLoader
{
public static readonly AssetType AssetType = AssetType.Texture;
public async ValueTask<Result<Asset>> LoadAsync(Stream cookedData, Guid id, CancellationToken token)
{
var header = new TextureContentHeader();
cookedData.ReadExactly(MemoryMarshal.AsBytes(new Span<TextureContentHeader>(ref header)));
var alignment = header.depth switch
{
8 => MemoryUtility.AlignOf<byte>(),
16 => MemoryUtility.AlignOf<ushort>(),
32 => MemoryUtility.AlignOf<float>(),
_ => MemoryUtility.AlignOf<float>()
};
var data = new MemoryBlock((nuint)(cookedData.Length - cookedData.Position), alignment, AllocationHandle.Persistent);
// C# built-in collections use int for indexing, so we need to ensure that the buffer size does not exceed int.MaxValue
var maxBufferSize = (int)Math.Min(0x7effffffu, header.width * header.height * header.depth / 8u * header.colorComponents);
var offset = 0u;
while (offset < data.Size)
{
using var memoryManager = NativeMemoryManager<byte>.FromMemoryBlock(data, (int)offset, maxBufferSize);
await cookedData.ReadExactlyAsync(memoryManager.Memory, token);
offset += (uint)memoryManager.Memory.Length;
}
return new TextureAsset(ref data, header, id);
}
}

View File

@@ -0,0 +1,100 @@
using Ghost.Core;
using Ghost.Core.Utilities;
using Ghost.Engine.AssetLoader;
using Ghost.Graphics.RHI;
using Ghost.Graphics.Utilities;
namespace Ghost.Engine;
public partial class AssetManager
{
private Handle<GPUTexture> AllocateTextureHandle()
{
// This will create a new slot in the database, but not allocation any GPU resource.
// Everything in the slot will have the same value as the fallback texture, expect the slot will be marked as shared.
return _resourceDatabase.CreateShared(_fallbackTexture.AsResource()).AsTexture();
}
private static TextureFormat GetTextureFormat(uint depth, uint colorComponents)
{
return colorComponents switch
{
1 => depth switch
{
8 => TextureFormat.R8_UNorm,
16 => TextureFormat.R16_UNorm,
32 => TextureFormat.R32_UInt,
_ => TextureFormat.Unknown,
},
2 => depth switch
{
8 => TextureFormat.R8G8_UNorm,
16 => TextureFormat.R16G16_UNorm,
32 => TextureFormat.R32G32_Float,
_ => TextureFormat.Unknown,
},
3 or 4 => depth switch
{
8 => TextureFormat.R8G8B8A8_UNorm,
16 => TextureFormat.R16G16B16A16_Float,
32 => TextureFormat.R32G32B32A32_Float,
_ => TextureFormat.Unknown,
},
_ => TextureFormat.Unknown,
};
}
private unsafe Result UploadTexture(AssetEntry entry)
{
var pData = (byte*)entry.rawData.GetUnsafePtr();
var reader = new BufferReader(pData, entry.rawData.Size);
var header = reader.Read<TextureContentHeader>();
var textureDesc = new TextureDesc
{
Width = header.width,
Height = header.height,
MipLevels = header.mipLevels,
Slice = 1,
Format = GetTextureFormat(header.depth, header.colorComponents),
Dimension = (TextureDimension)header.dimension,
Usage = TextureUsage.ShaderResource,
};
var newHandle = RenderingUtility.CreateTexture(
_resourceManager,
_resourceDatabase,
_resourceAllocator,
_uploadedBatch.CommandBuffer,
reader.Position,
in textureDesc);
if (newHandle.IsInvalid)
{
return Result.Failure("Failed to create GPU texture.");
}
// FIX: We can not Swap right now, we must wait on the GPU to finish the upload.
var oldHandle = entry.GetStorage<Handle<GPUTexture>>();
_resourceDatabase.Swap(oldHandle.AsResource(), newHandle.AsResource());
// Release the new handle since it now contains the old handle's resource.
// Because the old handle is shared, it will only release the slot in the database, not the actuall GPU resource, which is the fallback texture in this case.
_resourceDatabase.ReleaseResource(newHandle.AsResource());
return Result.Success();
}
public Handle<GPUTexture> ResolveTexture(Guid assetID)
{
if (assetID == Guid.Empty)
{
return _fallbackTexture;
}
var entry = GetOrCreateEntry(assetID);
Logger.DebugAssert(entry.assetType == AssetType.Texture);
return entry.GetStorage<Handle<GPUTexture>>();
}
}

View File

@@ -1,44 +1,429 @@
using Ghost.Core; using Ghost.Core;
using System.Runtime.InteropServices; using Ghost.Core.Utilities;
using Ghost.Engine.AssetLoader;
using Ghost.Graphics.Core;
using Ghost.Graphics.RHI;
using Ghost.Graphics.Services;
using Ghost.Graphics.Utilities;
using Misaki.HighPerformance.Buffer;
using Misaki.HighPerformance.Jobs;
using Misaki.HighPerformance.LowLevel.Buffer;
using Misaki.HighPerformance.LowLevel.Utilities;
using System.Collections.Concurrent;
using System.Diagnostics;
using System.Runtime.CompilerServices;
namespace Ghost.Engine; namespace Ghost.Engine;
internal abstract class RuntimeAsset; public enum AssetState : byte
internal interface IRuntimeAssetLoader
{ {
ValueTask<Result<RuntimeAsset>> LoadAsync(Stream cookedData, Guid id, CancellationToken token = default); Unloaded = 0,
Scheduled = 1,
Loading = 2,
Loaded = 3,
Ready = 4,
Failed = 5,
} }
internal sealed class RuntimeLoaderRegistry public enum AssetType : byte
{ {
private readonly Dictionary<Guid, IRuntimeAssetLoader> _loaders = new(); Texture = 0,
public void Register(Guid cookedTypeId, IRuntimeAssetLoader loader) Mesh = 1,
Material = 2,
Audio = 3,
Scene = 4,
Video = 5,
Json = 6,
Unknown = 255,
}
internal interface IContentProvider
{
bool HasAsset(Guid guid);
Result<Stream> OpenRead(Guid guid, CancellationToken token = default);
Guid[] GetDependencies(Guid guid);
AssetType GetAssetType(Guid guid);
}
// TODO: Support DirectStorage.
public partial class AssetManager : IDisposable
{
private unsafe class AssetEntry : IDisposable
{ {
_loaders[cookedTypeId] = loader; private static readonly ObjectPool<AssetEntry> s_pool = new ObjectPool<AssetEntry>(() => new AssetEntry(), (entry) => entry.Reset());
public struct __storage
{
public fixed byte data[64];
}
public Guid assetId;
public __storage storage;
public MemoryBlock rawData;
public JobHandle loadJobHandle;
public AssetType assetType;
public int state;
public int refCount;
public static AssetEntry Create()
{
return s_pool.Rent();
}
private AssetEntry()
{
}
private void Reset()
{
assetId = Guid.Empty;
assetType = AssetType.Unknown;
storage = default;
rawData = default;
state = (int)AssetState.Unloaded;
refCount = 0;
loadJobHandle = default;
}
public void SetStorage<T>(T asset)
where T : unmanaged
{
Unsafe.WriteUnaligned(ref storage.data[0], asset);
}
public T GetStorage<T>()
where T : unmanaged
{
return Unsafe.ReadUnaligned<T>(ref storage.data[0]);
}
public void Dispose()
{
s_pool.Return(this);
}
} }
public IRuntimeAssetLoader? GetLoader(Guid cookedTypeId)
private struct LoadAssetJob : IJob
{ {
_loaders.TryGetValue(cookedTypeId, out var loader); public Guid assetID;
return loader; public AssetType assetType;
public void Execute(ref readonly JobExecutionContext ctx)
{
var assetManager = ctx.State as AssetManager;
Debug.Assert(assetManager is not null);
Debug.Assert(assetManager._contentProvider.GetAssetType(assetID) == assetType);
if (!assetManager._entries.TryGetValue(assetID, out var entry))
{
Logger.Error($"Asset entry not found for {assetID}");
return;
}
var result = assetManager.LoadRawData(entry);
if (result.IsFailure)
{
Volatile.Write(ref entry.state, (int)AssetState.Failed);
Logger.Error($"Failed to load asset {assetID}: {result.Message}");
return;
}
Volatile.Write(ref entry.state, (int)AssetState.Loaded);
// Ensure the buffer inside the resource database does not move.
assetManager._resourceDatabase.EnterParallelRead();
try
{
switch (assetType)
{
case AssetType.Texture:
result = assetManager.UploadTexture(entry);
break;
case AssetType.Mesh:
break;
case AssetType.Material:
break;
case AssetType.Audio:
break;
case AssetType.Scene:
break;
case AssetType.Video:
break;
case AssetType.Json:
break;
case AssetType.Unknown:
default:
break;
}
if (result.IsFailure)
{
Logger.Error($"Failed to upload asset {assetID}: {result.Message}");
return;
}
}
finally
{
assetManager._resourceDatabase.ExitParallelRead();
}
}
}
private readonly IContentProvider _contentProvider;
private readonly ResourceManager _resourceManager;
private readonly IResourceAllocator _resourceAllocator;
private readonly IResourceDatabase _resourceDatabase;
private readonly ResourceUploadBatch _uploadedBatch; // Upload via copy queue.
private readonly JobScheduler _jobScheduler;
private readonly ConcurrentDictionary<Guid, AssetEntry> _entries;
private readonly ConcurrentQueue<Guid> _pendingUploads;
// TODO
private Handle<GPUTexture> _fallbackTexture;
private Handle<GPUTexture> _fallbackNormalMap;
private Handle<Mesh> _fallbackMesh;
private Handle<Material> _fallbackMaterial;
internal AssetManager(IContentProvider contentProvider, ResourceManager resourceManager, IResourceAllocator resourceAllocator, IResourceDatabase resourceDatabase, ResourceUploadBatch uploadBatch)
{
_contentProvider = contentProvider;
_resourceManager = resourceManager;
_resourceAllocator = resourceAllocator;
_resourceDatabase = resourceDatabase;
_uploadedBatch = uploadBatch;
// Ideally we should use a single JobScheduler across the entire engine, and schedule the streaming jobs to that scheduler as low priority background jobs.
// But how can we get the reference to the AssetManager? Is force job types to be unmanaged a wrong decision? Because we don't have burst compiler at all.
var threadCount = Environment.ProcessorCount < 8 ? 1 : 2;
_jobScheduler = new JobScheduler(threadCount, ThreadPriority.BelowNormal, this);
_entries = new ConcurrentDictionary<Guid, AssetEntry>();
_pendingUploads = new ConcurrentQueue<Guid>();
}
private JobHandle EnsureScheduled(Guid assetID)
{
if (_entries.TryGetValue(assetID, out var existing) && existing.state >= (int)AssetState.Scheduled)
{
return existing.loadJobHandle;
}
// Resolve dependencies (in-memory manifest/catalog lookup — instant)
var deps = _contentProvider.GetDependencies(assetID);
// Schedule all dependencies first (recursive, depth-first)
JobHandle dependency = default;
if (deps.Length > 0)
{
var depHandles = deps.Length <= 8
? stackalloc JobHandle[deps.Length]
: new JobHandle[deps.Length];
for (int i = 0; i < deps.Length; i++)
{
var depEntry = GetOrCreateEntry(deps[i]);
depHandles[i] = depEntry.loadJobHandle;
}
dependency = _jobScheduler.CombineDependencies(depHandles);
}
if (_entries.TryGetValue(assetID, out var entry))
{
var job = new LoadAssetJob
{
assetID = assetID,
assetType = entry.assetType,
};
entry.loadJobHandle = _jobScheduler.Schedule(ref job, dependency);
return entry.loadJobHandle;
}
// This should not happen, because GetOrCreateEntry should have created the entry and scheduled the job.
Debug.Fail($"Entry for {assetID} should have been created by GetOrCreateEntry");
return JobHandle.Invalid;
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private AssetEntry GetOrCreateEntry(Guid guid)
{
return _entries.GetOrAdd(guid, static (id, self) =>
{
var entry = AssetEntry.Create();
entry.assetId = id;
entry.assetType = self._contentProvider.GetAssetType(id);
entry.state = (int)AssetState.Scheduled;
switch (entry.assetType)
{
case AssetType.Texture:
entry.SetStorage(self.AllocateTextureHandle());
break;
case AssetType.Mesh:
break;
case AssetType.Material:
break;
case AssetType.Audio:
break;
case AssetType.Scene:
break;
case AssetType.Video:
break;
case AssetType.Json:
break;
case AssetType.Unknown:
default:
break;
}
entry.loadJobHandle = self.EnsureScheduled(entry.assetId);
return entry;
}, this);
}
private Result LoadRawData(AssetEntry entry)
{
try
{
using var stream = _contentProvider.OpenRead(entry.assetId).GetValueOrThrow();
var data = new MemoryBlock((nuint)stream.Length, MemoryUtility.AlignOf<IntPtr>(), AllocationHandle.Persistent);
// C# built-in collections use int for indexing, so we need to ensure that the buffer size does not exceed int.MaxValue
var maxChunkSize = (int)Math.Min(0x7fffffffu, data.Size);
var offset = 0u;
while (offset < data.Size)
{
using var mem = NativeMemoryManager<byte>.FromMemoryBlock(data, (int)offset, maxChunkSize);
stream.ReadExactly(mem.Memory.Span);
offset += (uint)mem.Memory.Length;
}
entry.rawData = data;
return Result.Success();
}
catch (Exception ex)
{
return Result.Failure(ex.Message);
}
}
// ── Render thread only — single-threaded by design ──
// TODO: Does this must be called on render thread? Can you just create a dedicated thred or a worker in thread pool for uploading?
// Also, I think we may don't need this RenderContext at all, because the CommandBuffer is from the ResourceUploadBatch (via async upload), and the ResourceManager/Database/Allocator can be passed in the constructor.
public void ProcessUploads(RenderContext ctx, int maxPerFrame = 4)
{
_uploadedBatch.Begin();
for (var i = 0; i < maxPerFrame; i++)
{
if (!_pendingUploads.TryDequeue(out var guid))
{
break;
}
if (!_entries.TryGetValue(guid, out var entry) || entry.asset is null)
{
continue;
}
var error = Error.Success;
switch (entry.assetType)
{
case AssetType.Texture:
var textureDesc = new TextureDesc
{
Width = textureAsset.Width,
Height = textureAsset.Height,
MipLevels = textureAsset.MipLevels,
Slice = 1,
Format = GetTextureFormat(textureAsset.Depth, textureAsset.ColorComponents),
Dimension = GetTextureDimension(textureAsset.Dimension),
Usage = TextureUsage.ShaderResource,
};
// NOTE: We use Color128 here to avoid that c# span can't hold 16k x 16k x sizeof(float) x 4 textures, because the max span length is int.MaxValue.
// Internal method will cast the data to void* so the type does not matter as long as the format and size are correct.
var handle = RenderingUtility.CreateTexture(
ctx.ResourceManager,
ctx.ResourceDatabase,
ctx.ResourceAllocator,
_uploadedBatch.CommandBuffer,
textureAsset.GeData<Color128>(),
in textureDesc);
textureAsset.SetTextureHandle(handle);
break;
default:
error = Error.NotSupported;
break;
}
if (error.IsSuccess)
{
Volatile.Write(ref entry.state, (int)AssetState.Ready);
}
else
{
_pendingUploads.Enqueue(guid); // retry next frame
}
}
_uploadedBatch.End();
// TODO: Do we need to wait?
// await _uploadedBatch.WaitAsync(); // WaitIdle();
}
/// <summary>
/// Blocking load. Returns when the asset reaches at least Loaded state.
/// GPU upload still happens via ProcessUploads on the render thread.
/// Use for loading screens or synchronous initialization.
/// </summary>
public async ValueTask<T?> LoadAsync<T>(AssetRef<T> assetRef, CancellationToken token = default)
where T : Asset
{
if (!assetRef.IsValid)
{
return null;
}
var entry = _entries.GetOrAdd(assetRef.guid, static (guid, self) =>
{
var e = new AssetEntryOld { assetId = guid, state = (int)AssetState.Loading };
e.loadTask = Task.Run(() => self.ExecuteLoadAsync(e));
return e;
}, this);
if (Volatile.Read(ref entry.state) >= (int)AssetState.Loaded)
{
return entry.asset as T;
}
var loadTask = entry.loadTask;
if (loadTask is not null)
{
await loadTask.WaitAsync(token).ConfigureAwait(false);
}
return _entries.TryGetValue(assetRef.guid, out var e) ? e.asset as T : null;
}
public void Dispose()
{
throw new NotImplementedException();
} }
} }
internal sealed class CookedTextureLoader : IRuntimeAssetLoader
{
public static readonly Guid TYPE_ID = TextureAsset.s_typeGuid;
public async ValueTask<Result<RuntimeAsset>> LoadAsync(Stream cookedData, Guid id, CancellationToken token)
{
// Read the ImageContentHeader you wrote during import
var header = new ImageContentHeader();
cookedData.ReadExactly(MemoryMarshal.AsBytes(new Span<ImageContentHeader>(ref header)));
// Read the rest as raw GPU data (DDS/BC compressed bytes)
var data = new byte[cookedData.Length - cookedData.Position];
await cookedData.ReadExactlyAsync(data, token);
return new TextureAsset(data, header, id);
}
}
public class AssetManager
{
}

View File

@@ -480,7 +480,7 @@ internal sealed unsafe partial class D3D12ResourceAllocator : IResourceAllocator
Dispose(); Dispose();
} }
private HRESULT CreateResource(D3D12MA_ALLOCATION_DESC* pAllocationDesc, D3D12_RESOURCE_DESC* pResourceDesc, D3D12_RESOURCE_STATES initialState, CreationOptions options, Guid* riid, void** ppv) private HRESULT CreateResource(D3D12MA_ALLOCATION_DESC* pAllocationDesc, D3D12_RESOURCE_DESC1* pResourceDesc, D3D12_BARRIER_LAYOUT initialLayout, CreationOptions options, uint numCapatableFormats, DXGI_FORMAT* pCastableFormats, Guid* riid, void** ppv)
{ {
HRESULT hr; HRESULT hr;
@@ -493,12 +493,14 @@ internal sealed unsafe partial class D3D12ResourceAllocator : IResourceAllocator
return E.E_NOTFOUND; return E.E_NOTFOUND;
} }
hr = _d3d12MA.Get()->CreateAliasingResource(result.Value.resource.allocation.Get(), options.Offset, pResourceDesc, initialState, null, riid, ppv); hr = _d3d12MA.Get()->CreateAliasingResource2(result.Value.resource.allocation.Get(), options.Offset, pResourceDesc, initialLayout, null, numCapatableFormats, pCastableFormats, riid, ppv);
} }
else else
{ {
Logger.DebugAssert(*riid == __uuidof<D3D12MA_Allocation>());
var iid_null = IID.IID_NULL; var iid_null = IID.IID_NULL;
hr = _d3d12MA.Get()->CreateResource(pAllocationDesc, pResourceDesc, initialState, null, (D3D12MA_Allocation**)ppv, &iid_null, null); hr = _d3d12MA.Get()->CreateResource3(pAllocationDesc, pResourceDesc, initialLayout, null, numCapatableFormats, pCastableFormats, (D3D12MA_Allocation**)ppv, &iid_null, null);
} }
return hr; return hr;
@@ -506,21 +508,23 @@ internal sealed unsafe partial class D3D12ResourceAllocator : IResourceAllocator
public ResourceSizeInfo GetSizeInfo(ResourceDesc desc) public ResourceSizeInfo GetSizeInfo(ResourceDesc desc)
{ {
D3D12_RESOURCE_DESC d3d12Desc; D3D12_RESOURCE_DESC1 d3d12Desc;
if (desc.Type == ResourceType.Texture) if (desc.Type == ResourceType.Texture)
{ {
d3d12Desc = desc.TextureDescriptor.ToD3D12ResourceDesc(); d3d12Desc = desc.TextureDescriptor.ToD3D12ResourceDesc1();
} }
else else
{ {
d3d12Desc = desc.BufferDescriptor.ToD3D12ResourceDesc(); d3d12Desc = desc.BufferDescriptor.ToD3D12ResourceDesc1();
} }
var info = _device.NativeObject.Get()->GetResourceAllocationInfo(0, 1, &d3d12Desc); D3D12_RESOURCE_ALLOCATION_INFO1 info1;
var info = _device.NativeObject.Get()->GetResourceAllocationInfo2(0, 1, &d3d12Desc, &info1);
return new ResourceSizeInfo return new ResourceSizeInfo
{ {
Size = info.SizeInBytes, Size = info.SizeInBytes,
Alignment = info.Alignment Alignment = info.Alignment,
Offset = info1.Offset,
}; };
} }
@@ -556,13 +560,13 @@ internal sealed unsafe partial class D3D12ResourceAllocator : IResourceAllocator
return _resourceDatabase.AddAllocation(alloc, barrierData, ResourceViewGroup.Invalid, default, name); return _resourceDatabase.AddAllocation(alloc, barrierData, ResourceViewGroup.Invalid, default, name);
} }
public Handle<GPUTexture> CreateTexture(ref readonly TextureDesc desc, string? name = null, CreationOptions options = default) public Handle<GPUTexture> CreateTexture(ref readonly TextureDesc desc, string? name = null, CreationOptions options = default, AdditionalTextureDesc additionalDesc = default)
{ {
Logger.DebugAssert(!_disposed); Logger.DebugAssert(!_disposed);
CheckTexture2DSize(desc.Width, desc.Height); CheckTexture2DSize(desc.Width, desc.Height);
var resourceDesc = desc.ToD3D12ResourceDesc(); var resourceDesc = desc.ToD3D12ResourceDesc1();
var allocationDesc = new D3D12MA_ALLOCATION_DESC var allocationDesc = new D3D12MA_ALLOCATION_DESC
{ {
HeapType = D3D12_HEAP_TYPE_DEFAULT, HeapType = D3D12_HEAP_TYPE_DEFAULT,
@@ -574,13 +578,19 @@ internal sealed unsafe partial class D3D12ResourceAllocator : IResourceAllocator
ID3D12Resource* pResource = default; ID3D12Resource* pResource = default;
HRESULT hr; HRESULT hr;
var pCastableFormats = stackalloc DXGI_FORMAT[additionalDesc.CastableFormat.Length];
for ( var i = 0; i < additionalDesc.CastableFormat.Length; i++)
{
pCastableFormats[i] = additionalDesc.CastableFormat[i].ToDXGIFormat();
}
if (isSubAllocation) if (isSubAllocation)
{ {
hr = CreateResource(&allocationDesc, &resourceDesc, D3D12_RESOURCE_STATE_COMMON, options, __uuidof(pResource), (void**)&pResource); hr = CreateResource(&allocationDesc, &resourceDesc, D3D12_BARRIER_LAYOUT_COMMON, options, (uint)additionalDesc.CastableFormat.Length, pCastableFormats, __uuidof(pResource), (void**)&pResource);
} }
else else
{ {
hr = CreateResource(&allocationDesc, &resourceDesc, D3D12_RESOURCE_STATE_COMMON, options, null, (void**)&pAllocation); hr = CreateResource(&allocationDesc, &resourceDesc, D3D12_BARRIER_LAYOUT_COMMON, options, (uint)additionalDesc.CastableFormat.Length, pCastableFormats, __uuidof(pAllocation), (void**)&pAllocation);
if (hr.SUCCEEDED) if (hr.SUCCEEDED)
{ {
pResource = pAllocation->GetResource(); pResource = pAllocation->GetResource();
@@ -638,8 +648,8 @@ internal sealed unsafe partial class D3D12ResourceAllocator : IResourceAllocator
var barrierData = new ResourceBarrierData var barrierData = new ResourceBarrierData
{ {
access = BarrierAccess.NoAccess,
layout = BarrierLayout.Common, layout = BarrierLayout.Common,
access = BarrierAccess.Common,
sync = BarrierSync.None sync = BarrierSync.None
}; };
@@ -656,20 +666,12 @@ internal sealed unsafe partial class D3D12ResourceAllocator : IResourceAllocator
return resource.AsTexture(); return resource.AsTexture();
} }
public Handle<GPUTexture> CreateRenderTarget(ref readonly RenderTargetDesc desc, string? name = null, CreationOptions options = default)
{
Logger.DebugAssert(!_disposed);
var textureDesc = desc.ToTextureDescription();
return CreateTexture(in textureDesc, name, options);
}
public Handle<GPUBuffer> CreateBuffer(ref readonly BufferDesc desc, string? name = null, CreationOptions options = default) public Handle<GPUBuffer> CreateBuffer(ref readonly BufferDesc desc, string? name = null, CreationOptions options = default)
{ {
Logger.DebugAssert(!_disposed); Logger.DebugAssert(!_disposed);
CheckBufferSize(desc.Size); CheckBufferSize(desc.Size);
var resourceDesc = desc.ToD3D12ResourceDesc(); var resourceDesc = desc.ToD3D12ResourceDesc1();
var isRaw = desc.Usage.HasFlag(BufferUsage.Raw); var isRaw = desc.Usage.HasFlag(BufferUsage.Raw);
var allocationDesc = new D3D12MA_ALLOCATION_DESC var allocationDesc = new D3D12MA_ALLOCATION_DESC
@@ -683,21 +685,13 @@ internal sealed unsafe partial class D3D12ResourceAllocator : IResourceAllocator
ID3D12Resource* pResource = default; ID3D12Resource* pResource = default;
HRESULT hr; HRESULT hr;
var initialState = desc.HeapType switch
{
HeapType.Default => D3D12_RESOURCE_STATE_COMMON,
HeapType.Upload => D3D12_RESOURCE_STATE_GENERIC_READ,
HeapType.Readback => D3D12_RESOURCE_STATE_COPY_DEST,
_ => D3D12_RESOURCE_STATE_COMMON
};
if (isSubAllocation) if (isSubAllocation)
{ {
hr = CreateResource(&allocationDesc, &resourceDesc, initialState, options, __uuidof(pResource), (void**)&pResource); hr = CreateResource(&allocationDesc, &resourceDesc, D3D12_BARRIER_LAYOUT_UNDEFINED, options, 0u, null, __uuidof(pResource), (void**)&pResource);
} }
else else
{ {
hr = CreateResource(&allocationDesc, &resourceDesc, initialState, options, null, (void**)&pAllocation); hr = CreateResource(&allocationDesc, &resourceDesc, D3D12_BARRIER_LAYOUT_UNDEFINED, options, 0u, null, __uuidof(pAllocation), (void**)&pAllocation);
if (hr.SUCCEEDED) if (hr.SUCCEEDED)
{ {
pResource = pAllocation->GetResource(); pResource = pAllocation->GetResource();
@@ -750,8 +744,8 @@ internal sealed unsafe partial class D3D12ResourceAllocator : IResourceAllocator
var barrierData = new ResourceBarrierData var barrierData = new ResourceBarrierData
{ {
access = BarrierAccess.NoAccess,
layout = BarrierLayout.Undefined, layout = BarrierLayout.Undefined,
access = BarrierAccess.Common,
sync = BarrierSync.None sync = BarrierSync.None
}; };

View File

@@ -7,6 +7,7 @@ using Misaki.HighPerformance.LowLevel.Collections;
using System.Diagnostics; using System.Diagnostics;
using System.Runtime.InteropServices; using System.Runtime.InteropServices;
using TerraFX.Interop.DirectX; using TerraFX.Interop.DirectX;
using TerraFX.Interop.Windows;
namespace Ghost.Graphics.D3D12; namespace Ghost.Graphics.D3D12;
@@ -39,7 +40,8 @@ internal unsafe class D3D12ResourceDatabase : IResourceDatabase
public ResourceBarrierData barrierData; public ResourceBarrierData barrierData;
public readonly bool isExternal; public bool isExternal;
public bool isShared;
public readonly bool Allocated => isExternal ? resource.resource.Get() != null : resource.allocation.Get() != null; public readonly bool Allocated => isExternal ? resource.resource.Get() != null : resource.allocation.Get() != null;
public readonly SharedPtr<ID3D12Resource> ResourcePtr => isExternal ? resource.resource.Get() : resource.allocation.Get()->GetResource(); public readonly SharedPtr<ID3D12Resource> ResourcePtr => isExternal ? resource.resource.Get() : resource.allocation.Get()->GetResource();
@@ -48,6 +50,7 @@ internal unsafe class D3D12ResourceDatabase : IResourceDatabase
{ {
this.resource = new __resource_union(allocation); this.resource = new __resource_union(allocation);
this.isExternal = false; this.isExternal = false;
this.isShared = false;
this.viewGroup = viewGroup; this.viewGroup = viewGroup;
this.barrierData = barrierData; this.barrierData = barrierData;
@@ -58,6 +61,7 @@ internal unsafe class D3D12ResourceDatabase : IResourceDatabase
{ {
this.resource = new __resource_union(resource); this.resource = new __resource_union(resource);
this.isExternal = true; this.isExternal = true;
this.isShared = false;
this.viewGroup = viewGroup; this.viewGroup = viewGroup;
this.barrierData = barrierData; this.barrierData = barrierData;
@@ -66,6 +70,11 @@ internal unsafe class D3D12ResourceDatabase : IResourceDatabase
public readonly uint Release(D3D12DescriptorAllocator descriptorAllocator) public readonly uint Release(D3D12DescriptorAllocator descriptorAllocator)
{ {
if (isShared)
{
return 0;
}
var refCount = 0u; var refCount = 0u;
if (Allocated) if (Allocated)
{ {
@@ -400,6 +409,39 @@ internal unsafe class D3D12ResourceDatabase : IResourceDatabase
return Error.None; return Error.None;
} }
public Handle<GPUResource> CreateShared(Handle<GPUResource> src)
{
if (src.IsInvalid)
{
return Handle<GPUResource>.Invalid;
}
var spinner = new SpinWait();
while (Interlocked.CompareExchange(ref _writeLock, 1, 0) != 0)
{
spinner.SpinOnce();
}
try
{
var (srcRecord, error) = GetResourceRecord(src);
if (error.IsFailure)
{
return Handle<GPUResource>.Invalid;
}
var newRecord = srcRecord.Get();
newRecord.isShared = true;
var id = _resources.Add(newRecord, out var generation);
return new Handle<GPUResource>(id, generation);
}
finally
{
Interlocked.Exchange(ref _writeLock, 0);
}
}
public void* MapResource(Handle<GPUResource> handle, uint subResource, ResourceRange? readRange) public void* MapResource(Handle<GPUResource> handle, uint subResource, ResourceRange? readRange)
{ {
var r = GetResourceRecord(handle); var r = GetResourceRecord(handle);

View File

@@ -134,10 +134,31 @@ internal static unsafe class D3D12Utility
return format switch return format switch
{ {
TextureFormat.Unknown => DXGI_FORMAT_UNKNOWN, TextureFormat.Unknown => DXGI_FORMAT_UNKNOWN,
TextureFormat.R8_UNorm => DXGI_FORMAT_R8_UNORM,
TextureFormat.R8_SNorm => DXGI_FORMAT_R8_SNORM,
TextureFormat.R16_UNorm => DXGI_FORMAT_R16_UNORM,
TextureFormat.R16_SNorm => DXGI_FORMAT_R16_SNORM,
TextureFormat.R16_Float => DXGI_FORMAT_R16_FLOAT,
TextureFormat.R32_UInt => DXGI_FORMAT_R32_UINT,
TextureFormat.R32_SInt => DXGI_FORMAT_R32_SINT,
TextureFormat.R8G8_UNorm => DXGI_FORMAT_R8G8_UNORM,
TextureFormat.R8G8_SNorm => DXGI_FORMAT_R8G8_SNORM,
TextureFormat.R16G16_UNorm => DXGI_FORMAT_R16G16_UNORM,
TextureFormat.R16G16_SNorm => DXGI_FORMAT_R16G16_SNORM,
TextureFormat.R16G16_Float => DXGI_FORMAT_R16G16_FLOAT,
TextureFormat.R32G32_Float => DXGI_FORMAT_R32G32_FLOAT,
TextureFormat.R8G8B8A8_UNorm => DXGI_FORMAT_R8G8B8A8_UNORM, TextureFormat.R8G8B8A8_UNorm => DXGI_FORMAT_R8G8B8A8_UNORM,
TextureFormat.R8G8B8A8_SNorm => DXGI_FORMAT_R8G8B8A8_SNORM,
TextureFormat.B8G8R8A8_UNorm => DXGI_FORMAT_B8G8R8A8_UNORM, TextureFormat.B8G8R8A8_UNorm => DXGI_FORMAT_B8G8R8A8_UNORM,
TextureFormat.R10G10B10A2_UNorm => DXGI_FORMAT_R10G10B10A2_UNORM,
TextureFormat.R16G16B16A16_Float => DXGI_FORMAT_R16G16B16A16_FLOAT, TextureFormat.R16G16B16A16_Float => DXGI_FORMAT_R16G16B16A16_FLOAT,
TextureFormat.R32G32B32A32_Float => DXGI_FORMAT_R32G32B32A32_FLOAT, TextureFormat.R32G32B32A32_Float => DXGI_FORMAT_R32G32B32A32_FLOAT,
TextureFormat.D24_UNorm_S8_UInt => DXGI_FORMAT_D24_UNORM_S8_UINT, TextureFormat.D24_UNorm_S8_UInt => DXGI_FORMAT_D24_UNORM_S8_UINT,
TextureFormat.D32_Float => DXGI_FORMAT_D32_FLOAT, TextureFormat.D32_Float => DXGI_FORMAT_D32_FLOAT,
TextureFormat.R32_Typeless => DXGI_FORMAT_R32_TYPELESS, TextureFormat.R32_Typeless => DXGI_FORMAT_R32_TYPELESS,
@@ -150,10 +171,32 @@ internal static unsafe class D3D12Utility
{ {
return format switch return format switch
{ {
DXGI_FORMAT_UNKNOWN => TextureFormat.Unknown,
DXGI_FORMAT_R8_UNORM => TextureFormat.R8_UNorm,
DXGI_FORMAT_R8_SNORM => TextureFormat.R8_SNorm,
DXGI_FORMAT_R16_UNORM => TextureFormat.R16_UNorm,
DXGI_FORMAT_R16_SNORM => TextureFormat.R16_SNorm,
DXGI_FORMAT_R16_FLOAT => TextureFormat.R16_Float,
DXGI_FORMAT_R32_UINT => TextureFormat.R32_UInt,
DXGI_FORMAT_R32_SINT => TextureFormat.R32_SInt,
DXGI_FORMAT_R8G8_UNORM => TextureFormat.R8G8_UNorm,
DXGI_FORMAT_R8G8_SNORM => TextureFormat.R8G8_SNorm,
DXGI_FORMAT_R16G16_UNORM => TextureFormat.R16G16_UNorm,
DXGI_FORMAT_R16G16_SNORM => TextureFormat.R16G16_SNorm,
DXGI_FORMAT_R16G16_FLOAT => TextureFormat.R16G16_Float,
DXGI_FORMAT_R32G32_FLOAT => TextureFormat.R32G32_Float,
DXGI_FORMAT_R8G8B8A8_UNORM => TextureFormat.R8G8B8A8_UNorm, DXGI_FORMAT_R8G8B8A8_UNORM => TextureFormat.R8G8B8A8_UNorm,
DXGI_FORMAT_R8G8B8A8_SNORM => TextureFormat.R8G8B8A8_SNorm,
DXGI_FORMAT_B8G8R8A8_UNORM => TextureFormat.B8G8R8A8_UNorm, DXGI_FORMAT_B8G8R8A8_UNORM => TextureFormat.B8G8R8A8_UNorm,
DXGI_FORMAT_R10G10B10A2_UNORM => TextureFormat.R10G10B10A2_UNorm,
DXGI_FORMAT_R16G16B16A16_FLOAT => TextureFormat.R16G16B16A16_Float, DXGI_FORMAT_R16G16B16A16_FLOAT => TextureFormat.R16G16B16A16_Float,
DXGI_FORMAT_R32G32B32A32_FLOAT => TextureFormat.R32G32B32A32_Float, DXGI_FORMAT_R32G32B32A32_FLOAT => TextureFormat.R32G32B32A32_Float,
DXGI_FORMAT_D24_UNORM_S8_UINT => TextureFormat.D24_UNorm_S8_UInt, DXGI_FORMAT_D24_UNORM_S8_UINT => TextureFormat.D24_UNorm_S8_UInt,
DXGI_FORMAT_D32_FLOAT => TextureFormat.D32_Float, DXGI_FORMAT_D32_FLOAT => TextureFormat.D32_Float,
DXGI_FORMAT_R32_TYPELESS => TextureFormat.R32_Typeless, DXGI_FORMAT_R32_TYPELESS => TextureFormat.R32_Typeless,
@@ -501,6 +544,67 @@ internal static unsafe class D3D12Utility
}; };
} }
public static D3D12_RESOURCE_DESC1 ToD3D12ResourceDesc1(this in TextureDesc desc)
{
var dxgiFormat = desc.Format.ToDXGIFormat();
if (desc.Usage.HasFlag(TextureUsage.DepthStencil) && desc.Usage.HasFlag(TextureUsage.ShaderResource))
{
if (dxgiFormat == DXGI_FORMAT_D32_FLOAT)
{
dxgiFormat = DXGI_FORMAT_R32_TYPELESS;
}
else if (dxgiFormat == DXGI_FORMAT_D24_UNORM_S8_UINT)
{
dxgiFormat = DXGI_FORMAT_R24G8_TYPELESS;
}
}
var maxDimension = Math.Max(desc.Width, Math.Max(desc.Height, desc.Slice));
var mipLevels = desc.MipLevels == 0
? (ushort)(1 + Math.Floor(Math.Log2(maxDimension)))
: (ushort)desc.MipLevels;
var resourceFlags = desc.Usage.ToD3D12ResourceFlag();
return desc.Dimension switch
{
TextureDimension.Texture2D => D3D12_RESOURCE_DESC1.Tex2D(
dxgiFormat,
desc.Width,
desc.Height,
mipLevels: mipLevels,
flags: resourceFlags),
TextureDimension.Texture3D => D3D12_RESOURCE_DESC1.Tex3D(
dxgiFormat,
desc.Width,
desc.Height,
(ushort)desc.Slice,
flags: resourceFlags),
TextureDimension.TextureCube => D3D12_RESOURCE_DESC1.Tex2D(
dxgiFormat,
desc.Width,
desc.Height,
mipLevels: mipLevels,
arraySize: 6,
flags: resourceFlags),
TextureDimension.Texture2DArray => D3D12_RESOURCE_DESC1.Tex2D(
dxgiFormat,
desc.Width,
desc.Height,
mipLevels: mipLevels,
arraySize: (ushort)desc.Slice,
flags: resourceFlags),
TextureDimension.TextureCubeArray => D3D12_RESOURCE_DESC1.Tex2D(
dxgiFormat,
desc.Width,
desc.Height,
mipLevels: mipLevels,
arraySize: (ushort)(desc.Slice * 6),
flags: resourceFlags),
_ => throw new ArgumentException($"Unsupported texture dimension: {desc.Dimension}"),
};
}
public static D3D12_RESOURCE_FLAGS ToD3D12ResourceFlag(this BufferUsage usage) public static D3D12_RESOURCE_FLAGS ToD3D12ResourceFlag(this BufferUsage usage)
{ {
var flags = D3D12_RESOURCE_FLAG_NONE; var flags = D3D12_RESOURCE_FLAG_NONE;
@@ -526,6 +630,19 @@ internal static unsafe class D3D12Utility
return D3D12_RESOURCE_DESC.Buffer(alignedSize, resourceFlags); return D3D12_RESOURCE_DESC.Buffer(alignedSize, resourceFlags);
} }
public static D3D12_RESOURCE_DESC1 ToD3D12ResourceDesc1(this in BufferDesc desc)
{
var alignedSize = desc.Size;
if (desc.Usage.HasFlag(BufferUsage.Constant))
{
// D3D12 CBV size must be 256-byte aligned
alignedSize = (uint)(desc.Size + 255) & ~255u;
}
var resourceFlags = desc.Usage.ToD3D12ResourceFlag();
return D3D12_RESOURCE_DESC1.Buffer(alignedSize, resourceFlags);
}
public static ResourceDesc GetResourceDesc(ID3D12Resource* pResource, ResourceViewGroup viewGroup) public static ResourceDesc GetResourceDesc(ID3D12Resource* pResource, ResourceViewGroup viewGroup)
{ {
D3D12_HEAP_PROPERTIES heapProperties; D3D12_HEAP_PROPERTIES heapProperties;

View File

@@ -757,150 +757,6 @@ public record struct ResourceDesc
} }
} }
/// <summary>
/// Render Target description
/// Supports either color OR depth rendering, not both
/// </summary>
public struct RenderTargetDesc
{
/// <summary>
/// Width of the render Target
/// </summary>
public uint Width
{
get; set;
}
/// <summary>
/// Height of the render Target
/// </summary>
public uint Height
{
get; set;
}
/// <summary>
/// Slice of the render Target
/// </summary>
public uint Slice
{
get; set;
}
/// <summary>
/// Type of render Target
/// </summary>
public RenderTargetType Type
{
get; set;
}
/// <summary>
/// Target texture Format
/// </summary>
public TextureFormat Format
{
get; set;
}
/// <summary>
/// Texture dimension
/// </summary>
public TextureDimension Dimension
{
get; set;
}
/// <summary>
/// Creation flags for the render Target
/// </summary>
public RenderTargetCreationFlags CreationFlags
{
get; set;
}
/// <summary>
/// Number of mip levels. 0 to generate full mip chain
/// </summary>
public uint MipLevels
{
get; set;
}
/// <summary>
/// Number of samples for MSAA
/// </summary>
public uint SampleCount
{
get; set;
}
/// <summary>
/// Creates a color render Target
/// </summary>
public static RenderTargetDesc Color(uint width, uint height, uint slice = 1,
TextureFormat format = TextureFormat.R8G8B8A8_UNorm, TextureDimension dimension = TextureDimension.Texture2D,
RenderTargetCreationFlags creationFlags = RenderTargetCreationFlags.AllowUAV | RenderTargetCreationFlags.DynamicallyResolution | RenderTargetCreationFlags.GenerateMips,
uint mipLevels = 0u, uint sampleCount = 1)
{
return new RenderTargetDesc
{
Width = width,
Height = height,
Slice = slice,
Type = RenderTargetType.Color,
Format = format,
Dimension = dimension,
CreationFlags = creationFlags,
MipLevels = mipLevels,
SampleCount = sampleCount
};
}
/// <summary>
/// Creates a depth render Target
/// </summary>
public static RenderTargetDesc Depth(uint width, uint height, uint slice = 1,
TextureFormat format = TextureFormat.D24_UNorm_S8_UInt, TextureDimension dimension = TextureDimension.Texture2D,
RenderTargetCreationFlags creationFlags = RenderTargetCreationFlags.AllowUAV | RenderTargetCreationFlags.DynamicallyResolution,
uint mipLevels = 0u, uint sampleCount = 1)
{
return new RenderTargetDesc
{
Width = width,
Height = height,
Slice = slice,
Type = RenderTargetType.Depth,
Format = format,
Dimension = dimension,
CreationFlags = creationFlags,
MipLevels = mipLevels,
SampleCount = sampleCount
};
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public TextureDesc ToTextureDescription()
{
var usage = Type == RenderTargetType.Color ? TextureUsage.RenderTarget : TextureUsage.DepthStencil;
if (CreationFlags.HasFlag(RenderTargetCreationFlags.AllowUAV))
{
usage |= TextureUsage.UnorderedAccess;
}
return new TextureDesc
{
Width = Width,
Height = Height,
Slice = Slice,
Format = Format,
Dimension = Dimension,
MipLevels = MipLevels,
Usage = usage,
};
}
}
/// <summary> /// <summary>
/// Texture description /// Texture description
/// </summary> /// </summary>
@@ -964,6 +820,14 @@ public record struct TextureDesc
} }
} }
public ref struct AdditionalTextureDesc
{
public ReadOnlySpan<TextureFormat> CastableFormat
{
get; set;
}
}
public record struct SamplerDesc public record struct SamplerDesc
{ {
public TextureFilterMode FilterMode public TextureFilterMode FilterMode
@@ -1564,10 +1428,31 @@ public enum RenderTargetType
public enum TextureFormat public enum TextureFormat
{ {
Unknown, Unknown,
R8_UNorm,
R8_SNorm,
R16_UNorm,
R16_SNorm,
R16_Float,
R32_UInt,
R32_SInt,
R8G8_UNorm,
R8G8_SNorm,
R16G16_UNorm,
R16G16_SNorm,
R16G16_Float,
R32G32_Float,
R8G8B8A8_UNorm, R8G8B8A8_UNorm,
R8G8B8A8_SNorm,
B8G8R8A8_UNorm, B8G8R8A8_UNorm,
R10G10B10A2_UNorm,
R16G16B16A16_Float, R16G16B16A16_Float,
R32G32B32A32_Float, R32G32B32A32_Float,
D24_UNorm_S8_UInt, D24_UNorm_S8_UInt,
D32_Float, D32_Float,

View File

@@ -77,6 +77,11 @@ public readonly struct ResourceSizeInfo
{ {
get; init; get; init;
} }
public ulong Offset
{
get; init;
}
} }
public interface IResourceAllocator : IDisposable public interface IResourceAllocator : IDisposable
@@ -97,17 +102,9 @@ public interface IResourceAllocator : IDisposable
/// <param name="desc">Texture description</param> /// <param name="desc">Texture description</param>
/// <param name="name">Debug name of the resource</param> /// <param name="name">Debug name of the resource</param>
/// <param name="options">Additional options of the resource allocation</param> /// <param name="options">Additional options of the resource allocation</param>
/// <param name="additionalDesc">Additional texture description for some specific texture types</param>
/// <returns>An <see cref="Handle{Texture}"/> point to the resource</returns> /// <returns>An <see cref="Handle{Texture}"/> point to the resource</returns>
Handle<GPUTexture> CreateTexture(ref readonly TextureDesc desc, string? name = null, CreationOptions options = default); Handle<GPUTexture> CreateTexture(ref readonly TextureDesc desc, string? name = null, CreationOptions options = default, AdditionalTextureDesc additionalDesc = default);
/// <summary>
/// Creates a render Target for off-screen rendering
/// </summary>
/// <param name="desc">Render Target description</param>
/// <param name="name">Debug name of the resource</param>
/// <param name="options">Additional options of the resource allocation</param>
/// <returns>An <see cref="Handle{Texture}"/> point to the resource</returns>
Handle<GPUTexture> CreateRenderTarget(ref readonly RenderTargetDesc desc, string? name = null, CreationOptions options = default);
/// <summary> /// <summary>
/// Creates a buffer resource /// Creates a buffer resource

View File

@@ -132,6 +132,18 @@ public unsafe interface IResourceDatabase : IDisposable
/// <returns>An Error indicating the success or failure of the swap operation.</returns> /// <returns>An Error indicating the success or failure of the swap operation.</returns>
Error Swap(Handle<GPUResource> handleA, Handle<GPUResource> handleB); Error Swap(Handle<GPUResource> handleA, Handle<GPUResource> handleB);
/// <summary>
/// Creates a new GPU resource that is a share of the specified source resource, including all its properties and data.
/// The new resource will have the same description and content as the source resource, but will be a distinct entity in the resource database with its own handle.
/// </summary>
/// <remarks>
/// The shared resource created by this method will have the same description and content as the source resource, but will be a distinct entity in the resource database with its own handle.
/// However, it is important to note that modifications to the shared resource through one handle will affect all other handles that reference the same underlying resource, as they all point to the same GPU memory.
/// </remarks>
/// <param name="src">The handle to the source resource.</param>
/// <returns>The handle to the newly created shared resource.</returns>
Handle<GPUResource> CreateShared(Handle<GPUResource> src);
/// <summary> /// <summary>
/// Maps a subresource of a GPU resource for CPU access, specifying read and write ranges. /// Maps a subresource of a GPU resource for CPU access, specifying read and write ranges.
/// </summary> /// </summary>

View File

@@ -1,7 +1,6 @@
using Ghost.Core; using Ghost.Core;
using Ghost.Core.Utilities; using Ghost.Core.Utilities;
using Ghost.Graphics.RHI; using Ghost.Graphics.RHI;
using System.Diagnostics;
using System.Runtime.InteropServices; using System.Runtime.InteropServices;
namespace Ghost.Graphics.RenderGraphModule; namespace Ghost.Graphics.RenderGraphModule;

View File

@@ -60,6 +60,7 @@ public class RenderSystem : IDisposable
get; init; get; init;
} }
// TODO: Thread local?
public required ICommandAllocator CommandAllocator public required ICommandAllocator CommandAllocator
{ {
get; init; get; init;

View File

@@ -1,6 +1,5 @@
using Ghost.Core; using Ghost.Core;
using Ghost.Graphics.RHI; using Ghost.Graphics.RHI;
using Ghost.Graphics.Services;
namespace Ghost.Graphics.Services; namespace Ghost.Graphics.Services;
@@ -11,6 +10,8 @@ public class ResourceUploadBatch
private readonly ICommandAllocator _commandAllocator; private readonly ICommandAllocator _commandAllocator;
private readonly ICommandBuffer _commandBuffer; private readonly ICommandBuffer _commandBuffer;
public ICommandBuffer CommandBuffer => _commandBuffer;
internal ResourceUploadBatch(IGraphicsEngine engine) internal ResourceUploadBatch(IGraphicsEngine engine)
{ {
_device = engine.Device; _device = engine.Device;
@@ -21,6 +22,7 @@ public class ResourceUploadBatch
public void Begin() public void Begin()
{ {
_commandAllocator.Reset();
_commandBuffer.Begin(_commandAllocator); _commandBuffer.Begin(_commandAllocator);
} }
@@ -32,7 +34,7 @@ public class ResourceUploadBatch
return r; return r;
} }
_device.GraphicsQueue.Submit(_commandBuffer); _device.CopyQueue.Submit(_commandBuffer);
return Result.Success(); return Result.Success();
} }

View File

@@ -1,34 +1,29 @@
using Ghost.Core; using Ghost.Core;
using Ghost.Graphics.RHI; using Ghost.Graphics.RHI;
using Misaki.HighPerformance.LowLevel.Utilities;
using Ghost.Graphics.Services; using Ghost.Graphics.Services;
using Misaki.HighPerformance.LowLevel.Utilities;
namespace Ghost.Graphics.Utilities; namespace Ghost.Graphics.Utilities;
public static unsafe class RenderingUtility public static unsafe class RenderingUtility
{ {
public static void UploadBuffer<T>(ResourceManager resourceManager, IResourceDatabase resourceDatabase, ICommandBuffer cmd, Handle<GPUBuffer> buffer, params ReadOnlySpan<T> data) public static Error UploadBuffer(ResourceManager resourceManager, IResourceDatabase resourceDatabase, ICommandBuffer cmd, Handle<GPUBuffer> buffer, void* pData, nuint sizeInBytes)
where T : unmanaged
{ {
var r = resourceDatabase.GetResourceDescription(buffer.AsResource()); var (desc, error) = resourceDatabase.GetResourceDescription(buffer.AsResource());
if (r.IsFailure) if (error.IsFailure)
{ {
return; return error;
} }
Logger.DebugAssert(r.Value.Type == ResourceType.Buffer); Logger.DebugAssert(desc.Type == ResourceType.Buffer);
var sizeInBytes = (nuint)(data.Length * sizeof(T)); var memoryType = desc.BufferDescriptor.HeapType;
var memoryType = r.Value.BufferDescriptor.HeapType;
if (memoryType == HeapType.Upload) if (memoryType == HeapType.Upload)
{ {
fixed (T* pData = data) var mappedData = resourceDatabase.MapResource(buffer.AsResource(), 0, null);
{ MemoryUtility.MemCpy(mappedData, pData, sizeInBytes);
var mappedData = resourceDatabase.MapResource(buffer.AsResource(), 0, null); resourceDatabase.UnmapResource(buffer.AsResource(), 0, null);
MemoryUtility.MemCpy(mappedData, pData, sizeInBytes);
resourceDatabase.UnmapResource(buffer.AsResource(), 0, null);
}
} }
else else
{ {
@@ -42,24 +37,66 @@ public static unsafe class RenderingUtility
var uploadHandle = resourceManager.CreateTransientBuffer(in uploadDesc); var uploadHandle = resourceManager.CreateTransientBuffer(in uploadDesc);
if (uploadHandle.IsInvalid) if (uploadHandle.IsInvalid)
{ {
throw new OutOfMemoryException("Failed to create upload buffer for buffer data."); return Error.OutOfMemory;
} }
fixed (T* pData = data) var mappedData = resourceDatabase.MapResource(uploadHandle.AsResource(), 0, null);
{ MemoryUtility.MemCpy(mappedData, pData, sizeInBytes);
var mappedData = resourceDatabase.MapResource(uploadHandle.AsResource(), 0, null); resourceDatabase.UnmapResource(uploadHandle.AsResource(), 0, null);
MemoryUtility.MemCpy(mappedData, pData, sizeInBytes);
resourceDatabase.UnmapResource(uploadHandle.AsResource(), 0, null);
}
cmd.Barrier(BarrierDesc.Buffer(buffer.AsResource(), BarrierSync.Copy, BarrierAccess.CopyDest));
cmd.CopyBuffer(buffer, uploadHandle, 0, 0, sizeInBytes); cmd.CopyBuffer(buffer, uploadHandle, 0, 0, sizeInBytes);
cmd.Barrier(BarrierDesc.Buffer(buffer.AsResource(), BarrierSync.None, BarrierAccess.Common));
}
return Error.None;
}
public static Error UploadBuffer<T>(ResourceManager resourceManager, IResourceDatabase resourceDatabase, ICommandBuffer cmd, Handle<GPUBuffer> buffer, params ReadOnlySpan<T> data)
where T : unmanaged
{
fixed (T* pData = data)
{
return UploadBuffer(resourceManager, resourceDatabase, cmd, buffer, pData, (nuint)(data.Length * sizeof(T)));
} }
} }
public static void UploadTexture<T>(ResourceManager resourceManager, IResourceDatabase resourceDatabase, ICommandBuffer cmd, Handle<GPUTexture> texture, ReadOnlySpan<T> data) public static Handle<GPUBuffer> CreateBuffer(ResourceManager resourceManager, IResourceDatabase resourceDatabase, IResourceAllocator resourceAllocator, ICommandBuffer cmd, void* pData, nuint sizeInBytes, ref readonly BufferDesc desc, string? name = null)
{
var error = Error.UnknownError;
var bufferHandle = resourceAllocator.CreateBuffer(in desc, name);
if (!bufferHandle.IsInvalid)
{
error = UploadBuffer(resourceManager, resourceDatabase, cmd, bufferHandle, pData, sizeInBytes);
}
if (error.IsSuccess)
{
return bufferHandle;
}
Logger.DebugAssert(error.IsSuccess);
return Handle<GPUBuffer>.Invalid;
}
public static Handle<GPUBuffer> CreateBuffer<T>(ResourceManager resourceManager, IResourceAllocator resourceAllocator, IResourceDatabase resourceDatabase, ICommandBuffer cmd, ReadOnlySpan<T> data, ref readonly BufferDesc desc, string? name = null)
where T : unmanaged where T : unmanaged
{ {
var desc = resourceDatabase.GetResourceDescription(texture.AsResource()).GetValueOrThrow(); fixed (T* pData = data)
{
return CreateBuffer(resourceManager, resourceDatabase, resourceAllocator, cmd, pData, (nuint)(data.Length * sizeof(T)), in desc, name);
}
}
public static Error UploadTexture(ResourceManager resourceManager, IResourceDatabase resourceDatabase, ICommandBuffer cmd, Handle<GPUTexture> texture, void* pData)
{
var (desc, error) = resourceDatabase.GetResourceDescription(texture.AsResource());
if (error.IsFailure)
{
return error;
}
desc.TextureDescriptor.Format.GetSurfaceInfo(desc.TextureDescriptor.Width, desc.TextureDescriptor.Height, out var rowPitch, out var slicePitch, out _); desc.TextureDescriptor.Format.GetSurfaceInfo(desc.TextureDescriptor.Width, desc.TextureDescriptor.Height, out var rowPitch, out var slicePitch, out _);
var requiredSize = resourceDatabase.GetIntermediateResourceSize(texture.AsResource(), 0, 1); var requiredSize = resourceDatabase.GetIntermediateResourceSize(texture.AsResource(), 0, 1);
@@ -73,21 +110,58 @@ public static unsafe class RenderingUtility
var uploadHandle = resourceManager.CreateTransientBuffer(in uploadDesc); var uploadHandle = resourceManager.CreateTransientBuffer(in uploadDesc);
if (uploadHandle.IsInvalid) if (uploadHandle.IsInvalid)
{ {
throw new OutOfMemoryException("Failed to create upload buffer for texture data."); return Error.OutOfMemory;
} }
cmd.Barrier(BarrierDesc.Texture(texture.AsResource(), BarrierSync.Copy, BarrierAccess.CopyDest, BarrierLayout.CopyDest)); cmd.Barrier(BarrierDesc.Texture(texture.AsResource(), BarrierSync.Copy, BarrierAccess.CopyDest, BarrierLayout.CopyDest));
var subresourceData = new SubResourceData
{
pData = pData,
rowPitch = rowPitch,
slicePitch = slicePitch
};
cmd.UpdateSubResources(texture.AsResource(), uploadHandle.AsResource(), subresourceData);
cmd.Barrier(BarrierDesc.Texture(texture.AsResource(), BarrierSync.None, BarrierAccess.Common, BarrierLayout.Common));
return Error.None;
}
public static Error UploadTexture<T>(ResourceManager resourceManager, IResourceDatabase resourceDatabase, ICommandBuffer cmd, Handle<GPUTexture> texture, ReadOnlySpan<T> data)
where T : unmanaged
{
fixed (T* pData = data) fixed (T* pData = data)
{ {
var subresourceData = new SubResourceData return UploadTexture(resourceManager, resourceDatabase, cmd, texture, pData);
{ }
pData = pData, }
rowPitch = rowPitch,
slicePitch = slicePitch
};
cmd.UpdateSubResources(texture.AsResource(), uploadHandle.AsResource(), subresourceData); public static Handle<GPUTexture> CreateTexture(ResourceManager resourceManager, IResourceDatabase resourceDatabase, IResourceAllocator resourceAllocator, ICommandBuffer cmd, void* pData, ref readonly TextureDesc desc, string? name = null)
{
var error = Error.UnknownError;
var textureHandle = resourceAllocator.CreateTexture(in desc, name);
if (!textureHandle.IsInvalid)
{
error = UploadTexture(resourceManager, resourceDatabase, cmd, textureHandle, pData);
}
if (error.IsSuccess)
{
return textureHandle;
}
Logger.DebugAssert(error.IsSuccess);
return Handle<GPUTexture>.Invalid;
}
public static Handle<GPUTexture> CreateTexture<T>(ResourceManager resourceManager, IResourceDatabase resourceDatabase, IResourceAllocator resourceAllocator, ICommandBuffer cmd, ReadOnlySpan<T> data, ref readonly TextureDesc desc, string? name = null)
where T : unmanaged
{
fixed (T* pData = data)
{
return CreateTexture(resourceManager, resourceDatabase, resourceAllocator, cmd, pData, in desc, name);
} }
} }
} }

View File

@@ -2,6 +2,7 @@ using Ghost.Core;
using Ghost.Core.Attributes; using Ghost.Core.Attributes;
using Ghost.Editor.Core.AssetHandler; using Ghost.Editor.Core.AssetHandler;
using Ghost.Editor.Core.Contracts; using Ghost.Editor.Core.Contracts;
using Ghost.Engine.AssetLoader;
using Microsoft.VisualStudio.TestTools.UnitTesting; using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace Ghost.UnitTest.AssetSystem; namespace Ghost.UnitTest.AssetSystem;