Commit Graph

25 Commits

Author SHA1 Message Date
d2c165bbe5 Centralize memory ops via MemoryUtility, add VM support
Refactor all memory allocation/deallocation to use MemoryUtility, replacing direct calls with unified methods. Introduce cross-platform virtual memory management (Mmap, Munmap, Decommit, Recommit). Switch to NativeMemory for standard allocations. Enhance FreeList with global free buckets and thread safety. Standardize alignment/size calculations. Remove global usings for memory utils. Bump version to 1.6.24. Includes minor cleanups and improved docs.
2026-05-07 21:34:25 +09:00
d6b4074281 Refactor collections to use 'scoped in T' parameters
Updated Add/Remove/Enqueue/Push/etc. methods in core unsafe collections to accept parameters as 'scoped in T' for improved performance and safety. Bumped assembly versions in both csproj files.
2026-05-02 13:52:45 +09:00
9c4faa107a feat(memory): transition to AllocationHandle API
Replaced the deprecated Allocator API with the new AllocationHandle API across the codebase. Updated constructors, methods, and tests to use AllocationHandle for memory management. Marked Allocator-based methods as [Obsolete] and provided alternatives.

Added OwnershipTransferAnalyzer to detect ownership transfer issues and introduced OwnershipTransferAttribute for marking parameters. Enhanced DefensiveCopyAnalyzer with additional checks for readonly and ValueType instances.

Refactored internal memory management in AllocationManager and updated benchmarks, utilities, and documentation to reflect the changes.

BREAKING CHANGE: Deprecated Allocator API in favor of AllocationHandle. Updated constructors and methods to use AllocationHandle. Users must migrate to the new API.
2026-04-12 17:50:12 +09:00
dea8de60d0 feat(collections)!: switch to ref struct enumerators
Refactor all unsafe collection enumerators to use ref struct types,
removing support for boxing and standard .NET enumeration interfaces.
GetEnumerator methods now return stack-only, more efficient enumerators
with [UnscopedRef] and inlining attributes. IEnumerable<T> and
IEnumerable implementations are removed from affected types. Interfaces
now require unmanaged types. Also includes minor doc and bug fixes.

BREAKING CHANGE: Enumerators are no longer compatible with LINQ, and collections no longer implement IEnumerable/IEnumerator.
2026-04-10 02:51:37 +09:00
208e1aa975 feat(allocator): add unified Reallocate to all allocators
Introduce a unified Reallocate method to all memory allocator types (Arena, Stack, FreeList, VirtualArena, VirtualStack, DynamicArena) and require it in the IMemoryAllocator interface. This enables efficient resizing of memory blocks, with fast-path optimizations for stack-like allocators.

Update AllocationManager and MemoryPool to use the new Reallocate method, simplifying and optimizing memory resizing logic. Add public properties for buffer pointers, sizes, and offsets to allocator structs for easier diagnostics.

Set FreeList's default concurrency level to 1 and make its allocation method return null on dispose instead of throwing. Clean up vector types for formatting, fix UnsafeList's RemoveRangeSwapBack logic, and simplify RemoveAtSwapBack.

Simplify Program.cs to only run SPMDBenchmark. Add new unit tests for FixedString, UnsafeList, UnsafeHashMap, and UnsafeHashSet. Apply minor test code cleanups for consistency in TestUnsafeQueue.

BREAKING CHANGE: IMemoryAllocator now requires a Reallocate method, and allocator APIs have changed accordingly.
2026-04-04 14:16:52 +09:00
78c565e428 feat(buffer): add allocation tracking and diagnostics
Add AllocationManager.GetTotalAllocatedMemory() to track total allocations when safety checks are enabled. Improve diagnostics by calling Debug.Fail in Dispose methods of HashMapHelper, UnTypedArray, and UnsafeArray when disposing uninitialized or already disposed arrays. Remove AddRange(ReadOnlyUnsafeCollection<T>) from UnsafeList<T>. Increment assembly version to 1.6.4. Ensure arrcpy is disposed in Program.cs.
2026-04-02 19:11:37 +09:00
abb0cd88ea feat(core): improve memory management and API safety
- JobScheduler now explicitly frees unmanaged memory in _jobInfoPool before disposal.
- Removed the unused TempJobAllocator struct and implementation.
- Refactored AllocationManager to use conditional compilation for safety checks, making MemoryHandle usage conditional.
- Improved documentation in AllocationManager for clarity on allocation size and safety check behavior.
- Added UnsafeSetCount method to UnsafeList<T> for direct count manipulation with validation.
- Bumped AssemblyVersion in Jobs and LowLevel projects.
2026-03-31 19:58:47 +09:00
aae8e2826f feat(core)!: refactor safety/debug defines, remove FreeList in JobSchedular
Refactor to use MHP_ENABLE_SAFETY_CHECKS, MHP_ENABLE_STACKTRACE, and MHP_ENABLE_MIMALLOC for feature toggling. Remove FreeList allocator in JobSchedular and debug-layer code, simplifying memory management. Improve memory leak detection and reporting, update memory allocation API, and guard all safety/debug features with new defines. Update csproj files, README, and code samples to match new API and toggles. Fix and improve collection types for correct behavior with and without safety checks. The codebase is now more modular and easier to configure for different build environments.

BREAKING CHANGE: Old defines (ENABLE_SAFETY_CHECKS, ENABLE_DEBUG_LAYER, ENABLE_MIMALLOC) are replaced with MHP_* equivalents. FreeList allocator and related debug features are removed. Some APIs and behaviors have changed for safety/debug configuration.
2026-03-30 15:21:09 +09:00
04dd7222d9 feat(unsafe-collections)!: move span/array copy methods to core
CopyTo, CopyFrom, ToArray, and ToList are now implemented as instance methods on UnsafeArray<T> and UnsafeList<T>, replacing the previous extension methods. This change enables more direct and efficient copying between unsafe collections and managed spans, arrays, and lists, with overloads for ranges and resizing as needed. Extension methods for these operations have been removed. Minor cleanup and usage updates are included.

BREAKING CHANGE: Extension methods for copying and converting between unsafe collections and spans/arrays/lists have been removed. Use the new instance methods instead.
2026-03-28 11:28:50 +09:00
a32b0668de feat(collections): add implicit conversions and AddRange overloads
Added implicit conversion operators to UnsafeArray<T> and UnsafeList<T> for easier conversion to ReadOnlyUnsafeCollection<T> and Span<T>. Introduced new AddRange overloads in UnsafeList<T> for ReadOnlyUnsafeCollection<T> and pointer-based sources. Updated AssemblyVersion in all projects. Improved MemoryLeakException stack trace output and fixed a documentation comment.
2026-03-25 17:37:21 +09:00
69b054e81d feat(lowlevel): add VirtualStack, update allocators, docs
Introduce VirtualStack allocator, refactor memory management to use virtual memory stacks, and update documentation.

Added VirtualStack as a new stack allocator using virtual memory, replaced Stack with VirtualStack in allocation manager and related APIs, and updated TempJobAllocator to use VirtualArena. Introduced AllocationManagerInitOpts for allocator configuration. Replaced ENABLE_COLLECTION_CHECKS with ENABLE_SAFETY_CHECKS for safety checks. Removed Result.cs and updated project files and examples. Added comprehensive README files for all major packages and improved root documentation.

BREAKING CHANGE: Stack allocator replaced by VirtualStack; TempJobAllocator and AllocationManager initialization signatures changed; Result types removed.
2026-03-19 15:38:23 +09:00
641b459997 feat(memory): improve oversized alloc handling & cleanup
Enhanced FreeList to handle oversized allocations by bypassing chunk management and allocating directly from the OS, with proper header assignment and immediate release on free. Updated IUnsafeCollection<T> documentation, removed unnecessary array clearing in Clear() methods, refined UnsafeSlotMap initialization logic, and removed redundant default element addition. Bumped assembly version to 1.5.1.
2026-03-18 20:12:41 +09:00
7326c83948 feat(allocator): add VirtualArena and FreeList allocators
Introduce VirtualArena for large, thread-safe virtual memory allocation and FreeList allocator for efficient persistent allocations. Update AllocationManager to support new allocators, add cross-platform virtual memory utilities, and improve thread-safety and performance in existing allocators. Bump version to 1.5.0 and update project configuration.

BREAKING CHANGE: AllocationManager initialization now requires explicit parameters for arena and FreeList capacities. Existing allocator usage may require code changes.
2026-03-18 19:26:16 +09:00
21e755a56e Added UnsafeMultiHashMap 2026-03-08 15:38:37 +09:00
ffac1c643e Fix source only package issue in nuget 2026-02-22 12:36:51 +09:00
fb31fd8ca8 Reserve index 0 in SlotMap, improve unsafe collections
- Reserve index 0 as always invalid in SlotMap, ConcurrentSlotMap, UnsafeSlotMap, and UnsafeSparseSet; update all index checks and slot operations accordingly
- Refactor SlotMap to use parallel arrays and BitArray for occupancy
- Double capacity on resize for all major unsafe collections
- Add debugger display support for unsafe collections
- Improve NuGet publishing workflow to skip existing versions
- Increment package versions (LowLevel: 1.3.1, main: 1.0.2)
- Add comprehensive unit tests for SlotMap and ConcurrentSlotMap
- Update main program and documentation for new slot map behavior
2025-12-12 16:10:49 +09:00
f3b0f295a8 Added new TempJobAllocator
Added new AllocationHandle property in Stack.Scope.

Changed the ref AllocationHandle constructor parameter to AllocationHandle on of all UnsafeCollection types
Removed Allocator.Stack. Use Stack.Scope.AllocationHandle to allocate on stack instead.
2025-12-06 22:16:39 +09:00
b67569aa14 Improve AsSpan, CopyTo, and CopyFrom for IUnsafeCollection 2025-11-27 14:04:46 +09:00
bf4dd5670e Refactor unsafe collections and improve memory handling
Some checks failed
Publish NuGet Packages / publish (pull_request) Has been cancelled
Refactored enumerators across multiple unsafe collections to use
`ref` returns for `Current`, improving performance and reducing
memory usage. Enhanced memory management with `AllocationOption`
support and optimized resizing logic for collections like
`UnsafeBitSet`, `UnsafeSlotMap`, and `UnsafeSparseSet`.

Updated `publish-nuget.yaml` to support manual workflow dispatch
and trigger on `push`/`pull_request` events. Incremented project
version to `1.1.2` and ensured NuGet package generation on build.
2025-11-11 21:20:33 +09:00
fbe72e33f7 Refactor AllocationManager and enhance debug tracking
Refactored `AllocationManager` to introduce intrusive allocation tracking with `AllocationHeader` structs for debug mode. Added lightweight allocation counters for non-debug mode. Enhanced memory leak detection with detailed stack traces and `MemoryLeakException`.

Simplified `AllocationInfo` by removing the `Allocator` property. Updated `AllocationOption` enum to remove `UnTracked` and clarified documentation.

Improved unsafe collections (`UnsafeArray`, `UnsafeStack`, etc.) with strongly-typed enumerators and better compatibility with `IEnumerable<T>`. Enhanced `UnsafeStack` with a dedicated `Enumerator` struct and consistent constructor parameters.

Refactored `MemoryLeakException` to support detailed allocation info and improved stack trace formatting. Simplified `MemoryUtility` by removing redundant null checks.

Added unit tests for `AllocationManager`, `UnsafeArray`, and `UnsafeStack` to validate memory management and functionality. Updated `Program.cs` with new examples.

Cleaned up namespaces, removed redundant `using` directives, and improved XML documentation. Applied `MethodImplOptions.AggressiveInlining` to performance-critical methods.
2025-11-06 01:28:43 +09:00
49e1171781 Refactor and enhance math and utility libraries
Some checks failed
Publish NuGet Packages / publish (push) Failing after 3m12s
Refactored `sincos` usage across `quaternion` and `random` to use `out` parameters for improved performance. Enhanced `random` struct with updated random direction generation methods.

Added new benchmarks in `MathematicsBenchmark` for vector operations, including SIMD-based `f4` struct. Downgraded target framework to `net9.0` for compatibility.

Introduced `ReadOnlyUnsafeCollection` for low-level memory management. Added utility methods in `CollectionUtility` for span creation and optimized list operations.

Renamed `MemoryUtilities` to `MemoryUtility` and updated all references. Enhanced `ObjectPool` with `Rent` and `TryRent` methods. Enabled `AllowUnsafeBlocks` and AOT compatibility in project configuration.

Performed general code cleanup, including removal of unused methods, improved formatting, and alignment with modern coding practices.
2025-11-04 14:53:01 +09:00
081103372f Refactor memory management and improve allocation APIs
Updated `ReallocFunc` to support `oldSize`, `newSize`, and `AllocationOption`, enabling more granular control over memory reallocation. Simplified `AllocationInfo` by removing the `FreeHandler` property. Enhanced `Reallocate` and `Allocate` methods in `AllocationManager` to handle memory clearing and tracking more effectively.

Refactored `UnsafeSparseSet` to use `UnsafeArray<T>` directly, added a `generations` array, and replaced the `freeList` with `UnsafeStack<int>` for better performance and simplicity. Updated `Resize`, `Add`, and `Remove` methods to improve memory handling and code clarity.

Introduced `AllocationOption` support in `Resize` methods across `IUnsafeCollection` implementations for flexible resizing behavior. Added `GetUnsafePtr` extension methods in `UnsafeUtilities` for direct access to span data.

Added new tests for `UnsafeSparseSet` to validate resizing, clearing, enumeration, and memory compaction. These changes improve memory management, enhance performance, and ensure correctness.
2025-10-08 15:40:49 +09:00
ac73e28f26 Refactor and enhance math, memory, and utilities
Refactored `sincos` usage in `quaternion` to use tuple-based
returns for improved readability. Introduced a `random` struct
with methods for generating random values of various types
and dimensions, including ranges and directions. Added a
`DynamicArray` class for dynamic resizing and manipulation
of collections. Enhanced `SlotMap` with new methods for
safe access and updates.

Updated `uint` vector types with `NumericConvertable`
attributes for better type interoperability. Removed the
`MathUtilities` class and refactored `adj` and `adjInverse`
methods for encapsulation. Improved memory management
with `StackAllocator` and `UnsafeArray` enhancements.

Added geometry utilities like `AABB`, `OBB`, `Plane`, and
`SphereBounds` for 3D operations. Updated project
configuration for versioning and NuGet packaging.
Performed general code cleanup, improved validation, and
aligned with modern C# practices.
2025-09-23 22:57:12 +09:00
a2a760594e Enhance mathematical capabilities and job system
Added new numeric types for unsigned integers, including uint2, uint3, and uint4, along with their matrix types.
Added a new `quaternion` struct with constructors and methods for creating and manipulating quaternions.
Added methods for projecting and reflecting vectors, enhancing geometric operations.
Added utility functions for generating orthonormal bases and changing vector signs.
Added comprehensive unit tests for new mathematical functions and quaternion operations.
Added a high-performance job scheduling system with job management features and worker thread management.
Added new structs for job execution, allowing efficient job scheduling and execution.
Added utility functions for job execution, including methods for obtaining unique job IDs.

Changed access modifiers and property definitions in several files for improved clarity and maintainability.
Changed property definitions and method implementations in `ImageInfo.cs`, `ImageResult.cs`, and `ImageResultFloat.cs` for better readability.
Changed memory management functions in `CRuntime.cs` and improved memory allocation tracking in `MemoryStats.cs`.
Changed the project file to include references to necessary projects and enable unsafe code blocks.

Removed the `WorkerThreadPool.cs` file, integrating worker thread management directly into the `JobScheduler`.
Removed the `float4` struct and its associated methods and properties, transitioning to a new code generation strategy.
Removed the `float4.tt` template and other related files, indicating a shift in code generation approach.
Removed the `Vectorize.cs` file, indicating a change in how vector operations are handled.

Updated the `.gitignore` file to include IDE-specific settings.
Updated various XML files to define project components and structure.
Updated the `AllocationManager.cs` to improve memory allocation management and introduce new strategies.
Updated the `UnsafeArray.cs`, `UnsafeHashMap.cs`, and `UnsafeList.cs` to enhance performance and safety in unsafe contexts.
Updated error handling and function pointer management in `MemoryLeakException.cs` and `FunctionPointer.cs`.
Updated the `AssemblyInfo.cs` file to include global using directives for better code organization.
2025-09-06 12:07:02 +09:00
eeff3313b5 Add image processing and memory management features
Added new namespace `Misaki.HighPerformance.Image` for image processing, including classes for animated GIF handling and memory management.
Added `AnimatedFrameResult` class for individual frames in animated images.
Added `AnimatedGifEnumerator` class for enumerating frames in animated GIFs.
Added `ColorComponents` enum for different color formats.
Added `ImageInfo` struct for image dimensions and color components.
Added `CRuntime` class for low-level memory management functions.
Added `MemoryStats` class to track memory allocation statistics.
Added utility functions for creating multi-dimensional arrays.
Added new structures for fixed-size UTF-8 encoded strings.
Added benchmarking classes to test new memory management features.

Changed `StbImage.cs` to include new namespaces and functionality for image data manipulation.
Changed project files to target .NET 9.0 and enable new features.
Changed `Arena.cs` and `DynamicArena.cs` to use `nuint` for size parameters.
Changed `BitSet.cs` to enhance bit manipulation methods.
Changed `Program.cs` to run `FunctionPtrBenchmark` for performance testing.

Removed memory tracking code from `AllocationManager.cs`, including the `_allocated` dictionary and related logic.
Removed `Free` method from `IAllocator.cs` interface.
Removed `UNSAFE_COLLECTION_CHECK` preprocessor directive from the codebase.

Refactored various files to improve organization, moving from `Unsafe` to `LowLevel` namespace.
Refactored `MemoryUtilities` class to include new memory operation methods.
Refactored `UnsafeUtilities.cs` to support new collection structures.
2025-07-12 19:48:42 +09:00