Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Map disk cache #132

Merged
merged 63 commits into from
Mar 28, 2022
Merged
Changes from 1 commit
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
6206368
Added a very basic disk cache to the bifrost.map() function to help s…
jaycedowell Feb 19, 2019
52ac3ed
Added basic CUDA runtime/driver version checking to the map disk cache.
jaycedowell Feb 19, 2019
3ffd92a
Small cleanup of the valitdate_cache() method of the DiskCacheMgr. U…
jaycedowell Feb 19, 2019
0cae416
Force a cleanup to map function cache when 'make' is called.
jaycedowell Feb 20, 2019
e209a80
Moved away from two files that store all of the cached kernels to a p…
jaycedowell May 16, 2019
accc1e5
Added a clear method to DiskCacheMgr in map.cpp. It's not currently …
jaycedowell May 16, 2019
6b11b8b
Added a new bfMapClearCache function to map.cpp and linked it in with…
jaycedowell May 16, 2019
b1ede2f
Updated test_map.py and test_transpose.py to always clear the map dis…
jaycedowell May 16, 2019
63b03a1
Added a flag to user.mk to disable the map disk cache at compile time.
jaycedowell Aug 30, 2019
90291f0
Fixed the dates in the various files modified.
jaycedowell Oct 2, 2019
2874bd7
Pulled in master.
jaycedowell Jul 23, 2020
f6b075e
Merge remote-tracking branch 'upstream/master' into map-disk-cache
jaycedowell Feb 13, 2021
609cf12
Merge remote-tracking branch 'upstream/master' into map-disk-cache
jaycedowell Apr 20, 2021
38071e7
Merge branch 'master' into map-disk-cache
jaycedowell Dec 3, 2021
ce0e5b5
Move the map cache directory into the user's HOME.
jaycedowell Dec 3, 2021
89a3c45
The cache is not longer in /dev/shm.
jaycedowell Dec 3, 2021
ab4ece0
Formatting cleanup.
jaycedowell Dec 3, 2021
6c7dd5f
Merge remote-tracking branch 'upstream/master' into map-disk-cache
jaycedowell Mar 22, 2022
c038eee
Formatting and cleanup.
jaycedowell Mar 22, 2022
5a0fff8
Rebuild configure.
jaycedowell Mar 22, 2022
6572c97
Rebuild configure correctly.
jaycedowell Mar 22, 2022
550f9dc
Move from CPPFLAGS to config.h for map cache control.
jaycedowell Mar 22, 2022
a87bf74
Add the map cache to the listing.
jaycedowell Mar 22, 2022
c93acbf
Rebuild configure.
jaycedowell Mar 22, 2022
6596cf9
Formatting.
jaycedowell Mar 22, 2022
d47a544
Added list_cache to bifrost.map to show the cache contents. Fixed a …
jaycedowell Mar 22, 2022
76eb87a
Clearing the cache always works if it does not exist.
jaycedowell Mar 24, 2022
f24402f
clear_cache -> clear_map_cache, plus import from __init__.py.
jaycedowell Mar 24, 2022
9408003
Clear the map cache only once per test suite.
jaycedowell Mar 24, 2022
1c8d598
list_map_cache cleanup/fixes.
jaycedowell Mar 24, 2022
6cdcbc5
Remove an old comment.
jaycedowell Mar 24, 2022
7a5d290
Remove debugging prints.
jaycedowell Mar 24, 2022
c2779e7
inline process_exists() for now.
jaycedowell Mar 24, 2022
865bef6
HAVE_MAP_CACHE follows HAVE_CUDA if the cache is requested.
jaycedowell Mar 24, 2022
4fe1d71
Simplify.
jaycedowell Mar 24, 2022
f235b21
Fix a few define problems.
jaycedowell Mar 24, 2022
7302af0
list_map_cache() formatting.
jaycedowell Mar 24, 2022
316806f
Also tag the cache with a 'map cache' version so that we can invalida…
jaycedowell Mar 24, 2022
ac1bd23
Update for the map cache.
jaycedowell Mar 24, 2022
fb30280
Make a note about using setUpClass.
jaycedowell Mar 24, 2022
e1d6b78
Small formatting cleanup.
jaycedowell Mar 24, 2022
a57921f
Style.
jaycedowell Mar 25, 2022
8851fb7
Not really a value for config.h.
jaycedowell Mar 25, 2022
3547518
More descriptive name.
jaycedowell Mar 25, 2022
1f8b80f
Match what is in #135.
jaycedowell Mar 25, 2022
50b5277
Ugh.
jaycedowell Mar 25, 2022
87ac1ab
Switch to a global control variable.
jaycedowell Mar 25, 2022
599ac73
Switch to a global control variable.
jaycedowell Mar 25, 2022
d300068
More specific.
jaycedowell Mar 25, 2022
c137507
In fileutils, catch polymorphic exceptions as references
league Mar 26, 2022
d050c34
Separate fileutils.cpp, to avoid inline duplication
league Mar 26, 2022
41f8798
If using the map cache always target the lowest arch. that the librar…
jaycedowell Mar 26, 2022
fc6ffa1
Merge branch 'map-disk-cache' of https://github.com/jaycedowell/bifro…
jaycedowell Mar 26, 2022
5724141
Added a TODO with caching options to consider.
jaycedowell Mar 26, 2022
44567fa
Expose list_map_cache
league Mar 26, 2022
3ccab0f
Constants for cache filenames; notes on fileutils system and flock.
league Mar 27, 2022
902c653
Move BF_MAP_KERNEL_CACHE_SIZE into map.h.
jaycedowell Mar 28, 2022
a65af24
Formatting.
jaycedowell Mar 28, 2022
9ddeec9
Formatting.
jaycedowell Mar 28, 2022
25c4610
Variable fix.
jaycedowell Mar 28, 2022
d75cf87
Add in a test for the cache listing function.
jaycedowell Mar 28, 2022
573cce2
Formatting.
jaycedowell Mar 28, 2022
a8f65f9
Merge branch 'master' into map-disk-cache
jaycedowell Mar 28, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Formatting cleanup.
  • Loading branch information
jaycedowell committed Dec 3, 2021
commit ab4ece0055409b93ba36696c72dcc36f6b78c077
6 changes: 3 additions & 3 deletions src/map.cpp
Original file line number Diff line number Diff line change
@@ -638,8 +638,8 @@ BFstatus bfMap(int ndim,
int const block_axes[2]) {
// Map containing compiled kernels and basic_indexing_only flag
thread_local static ObjectCache<std::string,std::pair<CUDAKernel,bool> >
kernel_cache(BF_MAP_KERNEL_CACHE_SIZE);
BF_ASSERT(ndim >= 0, BF_STATUS_INVALID_ARGUMENT);
kernel_cache(BF_MAP_KERNEL_CACHE_SIZE);
BF_ASSERT(ndim >= 0, BF_STATUS_INVALID_ARGUMENT);
//BF_ASSERT(!ndim || shape, BF_STATUS_INVALID_POINTER);
//BF_ASSERT(!ndim || axis_names, BF_STATUS_INVALID_POINTER);
BF_ASSERT(narg >= 0, BF_STATUS_INVALID_ARGUMENT);
@@ -725,7 +725,7 @@ BFstatus bfMap(int ndim,
basic_indexing_only,
&ptx, &kernel_name));
}
CUDAKernel kernel;
CUDAKernel kernel;
BF_TRY(kernel.set(kernel_name.c_str(), ptx.c_str()));
kernel_cache.insert(cache_key,
std::make_pair(kernel, basic_indexing_only));