[FSU] Inference FSU with Shared memory #2969
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Update FSU forwarding logic - FSU will handle look ahead tensor inside of pool - so we don't need to call Loadtensor for f + i **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghak PARK <donghak.park@samsung.com>
Add memory ptr for allocate shared mem - add mem_ptr - add unmap - array for manage unmapped ptr **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghak PARK <donghak.park@samsung.com>
I have changed the method from using dynamic memory allocation to using static memory allocation. In order to prevent multiple frees, I added a map to check whether the mem_address has already been processed. Previously, memory was allocated through buf, but now it is being allocated directly. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Co-authored-by: jijoong.moon <jijoong.moon@samsung.com> Signed-off-by: Donghak PARK <donghak.park@samsung.com>
make neuralnet can pass path to the swap_device & weight offset (file offset) it can make calculate weight file's offset **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Co-authored-by: hyeonseok <hs89.lee@samsung.com> Signed-off-by: Donghak PARK <donghak.park@samsung.com>
This was referenced Feb 25, 2025
3b395aa to
1629bad
Compare
Apply Shared mem & FSU - when inference mode : read from weight bin ( weight offset ) - when train mode : same logic with swap **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghak PARK <donghak.park@samsung.com>
1629bad to
769c28a
Compare
Fix Unittest Fail bug at Training Case Swap - There are some issue on PutBuffer that can not free ptr **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghak PARK <donghak.park@samsung.com>
ec044b7 to
d328785
Compare
Apply clang format at changed File **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghak PARK <donghak.park@samsung.com>
d328785 to
995c37c
Compare
Update FSU Unitter - For now, we should set our weight & input size as pagesize * N - For later i will add Page Align Algorithm **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghak PARK <donghak.park@samsung.com>
1ebd358 to
e41af8c
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[FSU] Inference FSU with Shared memory
To reduce memory usage during inference by utilizing FSU, and to minimize speed degradation by performing loading during forwarding, changed to use shared memory. and ensure the existing swap in training mode is also performed normally.
Commit 1 : [FSU] Update FSU Forwarding (Load) Logis
Commit 2 : [FSU] Update swap device & cache element
Commit 3 : [FSU] Update FSU mem allocate Logic
Commit 4 : [FSU] add FSU file offset info
Commit 5 : [FSU] Apply Shared Mem & FSU
This PR was include #2957 #2927 #2949, So i will close previous PRs