forked from Misaki/GhostEngine
Add high-performance material/shader system (Ghost.Shader.Concept)
Introduces a new Ghost.Shader.Concept project implementing a modern, data-oriented material and shader system with: - Global/local keyword bitsets (fast O(1) ops, 64 bytes) - Multi-pass shader program and per-pass render state overrides - Thread-safe, 16-byte aligned material property blocks - Material pooling to reduce GC pressure - Batch renderer for efficient PSO grouping and async variant warmup - Full demo (Program.cs) and extensive documentation (ARCHITECTURE.md, README.md, PROJECT_SUMMARY.md) - Minor integration: new enums, doc updates, and keyword handling in existing code No breaking changes to the existing engine; all new code is isolated. This serves as a reference implementation for high-performance, extensible material/shader architectures.
This commit is contained in:
356
Ghost.Shader.Concept/README.md
Normal file
356
Ghost.Shader.Concept/README.md
Normal file
@@ -0,0 +1,356 @@
|
||||
# Ghost Shader Concept - High Performance Material System
|
||||
|
||||
A modern, high-performance material and shader system designed for maximum efficiency and flexibility. Built with data-oriented design principles inspired by Unity DOTS, Unreal Engine 5, and modern rendering engines.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
### Core Design Principles
|
||||
|
||||
1. **Data-Oriented Design**: Cache-friendly memory layouts for optimal performance
|
||||
2. **Lock-Free Where Possible**: Concurrent collections and atomic operations minimize contention
|
||||
3. **Zero-Allocation Hot Paths**: Struct-based keys and value types reduce GC pressure
|
||||
4. **Compile-Time Variants**: Shader permutations compiled ahead or on-demand
|
||||
5. **Batch-Friendly**: Automatic PSO batching for minimal state changes
|
||||
|
||||
---
|
||||
|
||||
## System Components
|
||||
|
||||
### 1. Keyword System (`ShaderKeyword.cs`, `KeywordSet.cs`)
|
||||
|
||||
**Keywords** enable/disable shader features at compile time, creating variants.
|
||||
|
||||
- **Global Keywords**: Engine-wide settings (HDR, shadow quality, platform features)
|
||||
- **Local Keywords**: Per-material settings (normal mapping, alpha test, etc.)
|
||||
|
||||
**KeywordSet**: Compact bitset (256 global + 256 local keywords) using unsafe fixed buffers
|
||||
- O(1) enable/disable/query operations
|
||||
- Fast hash computation for variant key generation
|
||||
- Supports merging global + local keywords
|
||||
|
||||
```csharp
|
||||
var keywords = new KeywordSet();
|
||||
keywords.Enable(alphaTestKeyword);
|
||||
keywords.Enable(normalMapKeyword);
|
||||
ulong hash = keywords.ComputeHash(); // For variant lookup
|
||||
```
|
||||
|
||||
### 2. Shader Variant System (`ShaderKeys.cs`)
|
||||
|
||||
**ShaderVariantKey**: Uniquely identifies a compiled shader variant
|
||||
- Combines shader program ID + keyword hash
|
||||
- Used as cache key for `IShaderCompiler`
|
||||
|
||||
**GraphicsPipelineKey**: Uniquely identifies a complete PSO
|
||||
- Combines shader variant + render state hash + pass ID
|
||||
- Used as cache key for `IPipelineLibrary`
|
||||
|
||||
### 3. Render State (`RenderState.cs`)
|
||||
|
||||
Immutable, hashable pipeline state:
|
||||
- Rasterizer (cull mode, fill mode, depth bias)
|
||||
- Depth-Stencil (test/write enable, compare func, stencil ops)
|
||||
- Blend State (per-RT blend factors and operations)
|
||||
- Topology
|
||||
|
||||
```csharp
|
||||
var state = RenderState.Default;
|
||||
state.BlendEnable = true;
|
||||
state.SrcBlend = BlendFactor.SrcAlpha;
|
||||
ulong hash = state.ComputeHash();
|
||||
```
|
||||
|
||||
### 4. Shader Programs (`ShaderProgram.cs`)
|
||||
|
||||
A **ShaderProgram** represents a complete shader with multiple passes.
|
||||
|
||||
**ShaderPass**: Single rendering pass with:
|
||||
- Name and ID
|
||||
- Default render state
|
||||
- Entry point functions (vertex/pixel)
|
||||
|
||||
**Builder Pattern** for clean creation:
|
||||
|
||||
```csharp
|
||||
var shader = new ShaderProgramBuilder()
|
||||
.WithName("StandardPBR")
|
||||
.AddPass("ForwardBase", RenderState.Default)
|
||||
.AddPass("ShadowCaster", shadowState)
|
||||
.DeclareKeywords(alphaTest, normalMap)
|
||||
.Build();
|
||||
```
|
||||
|
||||
### 5. Material Properties (`MaterialPropertyBlock.cs`)
|
||||
|
||||
**Thread-safe**, **linear memory layout** for GPU upload efficiency.
|
||||
|
||||
Supports:
|
||||
- Scalars (float, int)
|
||||
- Vectors (float2, float3, float4)
|
||||
- Matrices (4x4)
|
||||
- Textures (planned)
|
||||
|
||||
Properties are **16-byte aligned** for GPU compatibility and stored contiguously.
|
||||
|
||||
```csharp
|
||||
var props = new MaterialPropertyBlock();
|
||||
props.SetFloat("_Metallic", 0.5f);
|
||||
props.SetVector4("_Color", 1, 0, 0, 1);
|
||||
unsafe {
|
||||
props.CopyTo(gpuBufferPtr, bufferSize);
|
||||
}
|
||||
```
|
||||
|
||||
### 6. Materials (`Material.cs`)
|
||||
|
||||
High-level material instance combining:
|
||||
- **Shader program** reference
|
||||
- **Property block** for per-material data
|
||||
- **Local keywords** for variant selection
|
||||
- **Per-pass render state overrides**
|
||||
|
||||
**Thread-safe** for property updates. **Dirty tracking** for efficient GPU updates.
|
||||
|
||||
```csharp
|
||||
var material = new Material(shaderProgram);
|
||||
material.SetFloat("_Metallic", 0.8f);
|
||||
material.EnableKeyword(normalMapKeyword);
|
||||
material.SetPassRenderState("ForwardBase", transparentState);
|
||||
|
||||
// Get pipeline key for rendering
|
||||
var psoKey = material.GetPipelineKey(passIndex, globalKeywords);
|
||||
```
|
||||
|
||||
**Cloning** for material instances:
|
||||
```csharp
|
||||
var clone = material.Clone(); // Deep copy of properties and state
|
||||
```
|
||||
|
||||
### 7. Global State (`GlobalKeywordState.cs`)
|
||||
|
||||
Singleton managing **engine-wide keywords**.
|
||||
- Thread-safe keyword enable/disable
|
||||
- Version tracking for cache invalidation
|
||||
- Automatic merging with local keywords during rendering
|
||||
|
||||
```csharp
|
||||
GlobalKeywordState.Instance.EnableKeyword(hdrKeyword);
|
||||
var keywords = GlobalKeywordState.Instance.GetKeywordSet();
|
||||
```
|
||||
|
||||
### 8. Batch Renderer (`MaterialBatchRenderer.cs`)
|
||||
|
||||
**Core rendering system** that:
|
||||
1. Groups draw calls by PSO (shader variant + render state + pass)
|
||||
2. Ensures shader variants are compiled
|
||||
3. Gets/creates PSOs from pipeline library
|
||||
4. Returns sorted batches for minimal state changes
|
||||
|
||||
```csharp
|
||||
var batches = batchRenderer.BatchDrawCalls(drawCalls);
|
||||
foreach (var batch in batches) {
|
||||
SetPipeline(batch.Pipeline);
|
||||
foreach (var draw in batch.DrawCommands) {
|
||||
// Upload material properties
|
||||
// Issue draw call
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Async Warmup** for pre-compiling variants:
|
||||
```csharp
|
||||
await batchRenderer.WarmupVariantsAsync(shader, variantConfigs);
|
||||
```
|
||||
|
||||
### 9. Material Pooling (`MaterialPool.cs`)
|
||||
|
||||
Object pool for material instances to reduce allocations.
|
||||
|
||||
```csharp
|
||||
var material = pool.Rent(shaderProgram);
|
||||
// Use material...
|
||||
pool.Return(material);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
### Memory Layout
|
||||
- **KeywordSet**: 64 bytes (fixed size, stack-allocated)
|
||||
- **RenderState**: ~60 bytes (stack-allocated)
|
||||
- **MaterialPropertyBlock**: Variable, contiguous, 16-byte aligned
|
||||
|
||||
### Complexity
|
||||
- **Keyword enable/disable**: O(1)
|
||||
- **Hash computation**: O(1) - fixed iterations
|
||||
- **Pipeline key generation**: O(1)
|
||||
- **Batch sorting**: O(N log N) where N = unique PSOs (typically << draw calls)
|
||||
|
||||
### Concurrency
|
||||
- **Lock-free**: Keyword queries, hash computation, key generation
|
||||
- **Concurrent**: Variant compilation cache, PSO cache
|
||||
- **Thread-safe**: Material property updates, global keyword changes
|
||||
|
||||
---
|
||||
|
||||
## Usage Example
|
||||
|
||||
```csharp
|
||||
// 1. Setup
|
||||
var registry = ShaderKeywordRegistry.Instance;
|
||||
var normalMap = registry.GetOrRegister("NORMAL_MAP", KeywordScope.Local);
|
||||
var hdr = registry.GetOrRegister("HDR", KeywordScope.Global);
|
||||
|
||||
GlobalKeywordState.Instance.EnableKeyword(hdr);
|
||||
|
||||
// 2. Create shader
|
||||
var shader = new ShaderProgramBuilder()
|
||||
.WithName("PBR")
|
||||
.AddPass("Forward", RenderState.Default)
|
||||
.AddPass("Shadow", shadowState)
|
||||
.DeclareKeywords(normalMap)
|
||||
.Build();
|
||||
|
||||
// 3. Create material
|
||||
var material = new Material(shader);
|
||||
material.SetVector4("_BaseColor", 1, 0, 0, 1);
|
||||
material.SetFloat("_Metallic", 0.8f);
|
||||
material.EnableKeyword(normalMap);
|
||||
|
||||
// 4. Render
|
||||
var batchRenderer = new MaterialBatchRenderer(compiler, pipelineLib);
|
||||
var batches = batchRenderer.BatchDrawCalls(drawCommands);
|
||||
|
||||
foreach (var batch in batches) {
|
||||
commandList.SetPipeline(batch.Pipeline);
|
||||
foreach (var draw in batch.DrawCommands) {
|
||||
unsafe {
|
||||
draw.Material.CopyPropertiesTo(cbufferPtr, cbufferSize);
|
||||
}
|
||||
commandList.DrawIndexed(draw.IndexCount, ...);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Advanced Features
|
||||
|
||||
### Per-Pass State Overrides
|
||||
|
||||
Materials can override render state per-pass:
|
||||
|
||||
```csharp
|
||||
var transparentState = RenderState.Default;
|
||||
transparentState.BlendEnable = true;
|
||||
transparentState.SrcBlend = BlendFactor.SrcAlpha;
|
||||
transparentState.DestBlend = BlendFactor.InvSrcAlpha;
|
||||
|
||||
material.SetPassRenderState("Forward", transparentState);
|
||||
material.SetPassRenderState("Shadow", RenderState.Default); // Opaque shadow
|
||||
```
|
||||
|
||||
### Shader Variant Warmup
|
||||
|
||||
Pre-compile common variants to avoid runtime hitches:
|
||||
|
||||
```csharp
|
||||
var variants = new[] {
|
||||
keywordSet1, // No features
|
||||
keywordSet2, // Normal map only
|
||||
keywordSet3, // Normal map + alpha test
|
||||
// ...
|
||||
};
|
||||
|
||||
await batchRenderer.WarmupVariantsAsync(shader, variants);
|
||||
```
|
||||
|
||||
### Material Property Inheritance
|
||||
|
||||
Clone materials with shared base properties:
|
||||
|
||||
```csharp
|
||||
var baseMaterial = new Material(shader);
|
||||
baseMaterial.SetVector4("_BaseColor", 1, 1, 1, 1);
|
||||
|
||||
var redVariant = baseMaterial.Clone();
|
||||
redVariant.SetVector4("_BaseColor", 1, 0, 0, 1);
|
||||
|
||||
var blueVariant = baseMaterial.Clone();
|
||||
blueVariant.SetVector4("_BaseColor", 0, 0, 1, 1);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Extension Points
|
||||
|
||||
### Custom Property Types
|
||||
|
||||
Extend `MaterialPropertyBlock` for custom data:
|
||||
|
||||
```csharp
|
||||
public void SetCustomStruct<T>(string name, T value) where T : unmanaged
|
||||
{
|
||||
// Implementation
|
||||
}
|
||||
```
|
||||
|
||||
### Material Property Validation
|
||||
|
||||
Add validation in `Material` setters:
|
||||
|
||||
```csharp
|
||||
public void SetFloat(string name, float value)
|
||||
{
|
||||
if (value < 0 || value > 1)
|
||||
throw new ArgumentOutOfRangeException();
|
||||
_propertyBlock.SetFloat(name, value);
|
||||
}
|
||||
```
|
||||
|
||||
### Custom Batching Strategies
|
||||
|
||||
Subclass or compose with `MaterialBatchRenderer`:
|
||||
|
||||
```csharp
|
||||
public class DepthSortedBatchRenderer : MaterialBatchRenderer
|
||||
{
|
||||
public override MaterialBatch[] BatchDrawCalls(...)
|
||||
{
|
||||
var batches = base.BatchDrawCalls(...);
|
||||
// Custom depth sorting logic
|
||||
return batches;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Comparison to Other Engines
|
||||
|
||||
| Feature | Ghost | Unity URP | Unreal 5 | Godot 4 |
|
||||
|---------|-------|-----------|----------|---------|
|
||||
| Keyword System | Global + Local | Global + Local | Static + Dynamic | Static |
|
||||
| Multi-pass | Native | SubShader | Material Functions | Multi-pass |
|
||||
| Per-pass Override | ✓ | Limited | ✓ | ✓ |
|
||||
| Variant Caching | Auto | Auto | Auto | Auto |
|
||||
| Batch Optimization | PSO-based | SRP Batcher | Nanite/VSM | Clustered |
|
||||
| Unsafe/Native | ✓ | ✓ (Jobs) | ✓ (C++) | Limited |
|
||||
|
||||
---
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
1. **GPU-Driven Rendering**: Indirect draws, culling on GPU
|
||||
2. **Material Graphs**: Node-based shader authoring
|
||||
3. **Hot Reload**: Runtime shader recompilation
|
||||
4. **Texture Support**: Bindless textures, virtual texturing
|
||||
5. **Compute Shaders**: Material property animation on GPU
|
||||
6. **Serialization**: Material asset loading/saving
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
This is a concept/demonstration project. Adapt as needed for your engine.
|
||||
Reference in New Issue
Block a user