Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
5 changes: 5 additions & 0 deletions catch/ABM/AddKernels/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# Copyright (c) Advanced Micro Devices, Inc., or its affiliates.
#
# SPDX-License-Identifier: MIT

# Common Tests - Test independent of all platforms
set(TEST_SRC
add.cc
Expand All @@ -6,3 +10,4 @@ set(TEST_SRC
hip_add_exe_to_target(NAME ABMAddKernels
TEST_SRC ${TEST_SRC}
TEST_TARGET_NAME build_tests)

8 changes: 7 additions & 1 deletion catch/ABM/AddKernels/add.cc
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
/*
* Copyright (c) Advanced Micro Devices, Inc., or its affiliates.
*
* SPDX-License-Identifier: MIT
*/

#include <hip_test_common.hh>
#include <iostream>

Expand All @@ -6,7 +12,7 @@ template <typename T> __global__ void add(T* a, T* b, T* c, size_t size) {
if (i < size) c[i] = a[i] + b[i];
}

TEMPLATE_TEST_CASE("ABM_AddKernel_MultiTypeMultiSize", "", int, long, float, long long, double) {
TEMPLATE_TEST_CASE(ABM_AddKernel_MultiTypeMultiSize, int, long, float, long long, double) {
auto size = GENERATE(as<size_t>{}, 100, 500, 1000);
TestType *d_a, *d_b, *d_c;
auto res = hipMalloc(&d_a, sizeof(TestType) * size);
Expand Down
4 changes: 4 additions & 0 deletions catch/ABM/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1 +1,5 @@
# Copyright (c) Advanced Micro Devices, Inc., or its affiliates.
#
# SPDX-License-Identifier: MIT

add_subdirectory(AddKernels)
313 changes: 172 additions & 141 deletions catch/CMakeLists.txt

Large diffs are not rendered by default.

66 changes: 40 additions & 26 deletions catch/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,6 @@ Tests in Catch2 are declared via ```TEST_CASE```.
Running Subtest: ctest –R “...” (Regex to match the subtest name)

## New Features
- Skip test without recompiling tests, by addition of a json file. Default name is ```config.json``` , this can be overridden by using the variable ```HIP_CATCH_EXCLUDE_FILE=some_config.json```.
- Json file supports regex. Ex: All tests which has the word ‘Memset’ can be skipped using ‘*Memset*’
- Support multiple skip test list which can be set via environment variable, so you can have multiple files containing different skip test lists and can pick and choose among them depending on your platform and os.
- Better CI integration via xunit compatible output
- hip-tests can be built in SPIRV, where all kernels are compiled in SPIRV, enable with cmake flag `-DENABLE_SPIRV=ON`. By default its `OFF`.

Expand Down Expand Up @@ -51,31 +48,42 @@ TEST_CASE("hipExtAPIs") {
#endif
```

## Config file schema
Some tests can be skipped using a config file placed in hipTestMain/config folder. Multiple config files can be defined for different configurations.
The naming convention for the file needs to be "config_platform_os_archname.json"
Platform and os are mandatory.
Arch name is optional and takes precedence while loading the json file.
Currently the json files need to be manually chosen by the executor for the architecture of choice.

example:
config_amd_windows.json
config_nvidia_windows.json

The schema of the json file is as follows:
```json
{
"DisabledTests":
[
"TestName1",
"TestName2",
...
]
}
## Configuration

All configuration for the test cases is done through the `hip_tests_config.yaml` file.
Every test case has its own entry. Currently supported options are:
- level : Specify to which level the case belongs to (e.g. Level_2 is a standard test)
- tags : List all Catch2 tags that the case is associated with
- disabled : List all platforms where the case should be disabled
The group name is automatically added as a tag for every case.
Changing the configuration file will retrigger the build, so we have an up to date configuration every time.

Example:
```yaml
unit:
atomics:
Unit_atomicExch_system_Positive_Peer_GPUs:
<<: *level_2
tags: [multigpu]
# SWDEV-435667: Below tests failing randomly in stress test on 01/12/23
disabled: [amd_wsl]
```
will be generated as (on an AMD linux machine) as:
```cpp
#define Unit_atomicExch_system_Positive_Peer_GPUs "Unit_atomicExch_system_Positive_Peer_GPUs", "[multigpu][level_2][atomics]"
```
and on an AMD WSL machine as (skipping the test):
```cpp
#define Unit_atomicExch_system_Positive_Peer_GPUs "Unit_atomicExch_system_Positive_Peer_GPUs", "[.]"
```
and this macro is expanded in the actual test case definition
```cpp
TEMPLATE_TEST_CASE(Unit_atomicExch_system_Positive_Peer_GPUs, int, float, double)
```

<b>Every newly added test should mandate adding an entry to the configuration file.</b>

## Environment Variables
- `HIP_CATCH_EXCLUDE_FILE` : This variable can be set to the config file name or full path. Disabled tests will be read from this.
- `HT_LOG_ENABLE` : This is for debugging the HIP Test Framework itself. Setting it to 1, all `LogPrintf` will be printed on screen

## Test Macros
Expand Down Expand Up @@ -174,8 +182,13 @@ The process must be a standalone exe inside the same folder as other tests.
Initially, the new tests can be enabled via using ```-DHIP_CATCH_TEST=1```. After porting existing tests, this will be turned on by default.

## Building a single test
Generate `hip_tests_config.hh` before compilation
```bash
python3 <HIP_SRC_DIR>/catch/config/parse_config.py <HIP_SRC_DIR>/catch/config/configs <PLATFORM> <OS> <ARCH> <CONFIG_TARGET_DIR>/hip_tests_config.hh
```

```bash
hipcc <path_to_test.cpp> -I<HIP_SRC_DIR>/tests/catch/include <HIP_SRC_DIR>/tests/catch/hipTestMain/standalone_main.cc -I<HIP_SRC_DIR>/tests/catch/external/Catch2 -g -o <out_file_name>
hipcc -I <HIP_SRC_DIR>/catch/include -I <HIP_SRC_DIR>/catch/external/picojson -I <CONFIG_TARGET_DIR> <HIP_SRC_DIR>/catch/hipTestMain/main.cc <HIP_SRC_DIR>/catch/hipTestMain/hip_test_context.cc -I <CATCH2_INCLUDE_DIR> -L <CATCH2_LIB_DIR> -lCatch2 -I $HIP_PATH/include -O1 -ggdb <PATH_TO_TEST.cc> -o <OUT_FILE_NAME>
```

## Debugging support
Expand All @@ -188,6 +201,7 @@ Catch2 allows multiple ways in which you can debug the test case.
## External Libs being used
- [Catch2](https://github.com/catchorg/Catch2) - Testing framework
- [picojson](https://github.com/kazuho/picojson) - For config file parsing
- [PyYAML](https://pyyaml.org/) - For config file parsing (Python script)

# Testing Guidelines
Tests fall in 5 categories and its file name prefix are as follows:
Expand Down
4 changes: 4 additions & 0 deletions catch/TypeQualifiers/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# Copyright (c) Advanced Micro Devices, Inc., or its affiliates.
#
# SPDX-License-Identifier: MIT

# Common Tests
set(TEST_SRC
hipManagedKeyword.cc
Expand Down
34 changes: 16 additions & 18 deletions catch/TypeQualifiers/hipManagedKeyword.cc
Original file line number Diff line number Diff line change
@@ -1,20 +1,7 @@
/*
Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANNTY OF ANY KIND, EXPRESS OR
IMPLIED, INNCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANNY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER INN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR INN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
* Copyright (c) Advanced Micro Devices, Inc., or its affiliates.
*
* SPDX-License-Identifier: MIT
*/

/*
Expand All @@ -39,6 +26,17 @@ static __global__ void managed_add(size_t size) {
static __global__ void managed_inc() { atomicAdd(&m_X, 1.0f); }

TEST_CASE("Unit_hipManagedKeyword_SingleGpu") {
int numDevices = 0;
HIP_CHECK(hipGetDeviceCount(&numDevices));
for (int i = 0; i < numDevices; i++) {
int managed_memory = 0;
HIP_CHECK(hipDeviceGetAttribute(&managed_memory, hipDeviceAttributeManagedMemory, i));
if (!managed_memory) {
HipTest::HIP_SKIP_TEST("managed memory access not supported on device");
return;
}
}

for (size_t i = 0; i < N; i++) {
m_A[i] = 1.0f;
m_B[i] = 2.0f;
Expand All @@ -58,13 +56,13 @@ TEST_CASE("Unit_hipManagedKeyword_SingleGpu") {
}
}

TEST_CASE("Unit_hipManagedKeyword_MultiGpu") {
TEST_CASE(Unit_hipManagedKeyword_MultiGpu) {
int numDevices = 0;
HIP_CHECK(hipGetDeviceCount(&numDevices));

for (int i = 0; i < numDevices; i++) {
int managed_memory = 0;
HIPCHECK(hipDeviceGetAttribute(&managed_memory, hipDeviceAttributeManagedMemory, i));
HIP_CHECK(hipDeviceGetAttribute(&managed_memory, hipDeviceAttributeManagedMemory, i));
if (!managed_memory) {
HipTest::HIP_SKIP_TEST("managed memory access not supported on device");
return;
Expand Down
21 changes: 13 additions & 8 deletions catch/cmake/hip-tests.cmake
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# Copyright (c) Advanced Micro Devices, Inc., or its affiliates.
#
# SPDX-License-Identifier: MIT

include(Catch)

###############################################################################
Expand Down Expand Up @@ -66,8 +70,8 @@ function(hip_gen_exe_target)
endif()

# Create shared lib of all tests
set_source_files_properties(${SRC_NAME} PROPERTIES LANGUAGE HIP)
set_source_files_properties(${COMMON_SHARED_SRC} PROPERTIES LANGUAGE HIP)
set_source_files_properties(${SRC_NAME} PROPERTIES LANGUAGE ${GPGPU_LANGUAGE})
set_source_files_properties(${COMMON_SHARED_SRC} PROPERTIES LANGUAGE ${GPGPU_LANGUAGE})
if(NOT RTC_TESTING)
add_executable(${_EXE_NAME} EXCLUDE_FROM_ALL ${SRC_NAME} ${COMMON_SHARED_SRC} $<TARGET_OBJECTS:Main_Object> $<TARGET_OBJECTS:KERNELS>)
else ()
Expand All @@ -78,7 +82,7 @@ function(hip_gen_exe_target)
target_link_libraries(${_EXE_NAME} nvrtc)
endif()
endif()
set_target_properties(${_EXE_NAME} PROPERTIES LINKER_LANGUAGE HIP)
set_target_properties(${_EXE_NAME} PROPERTIES LINKER_LANGUAGE ${GPGPU_LANGUAGE})

if (DEFINED _PROPERTY)
set_property(TARGET ${_EXE_NAME} PROPERTY ${_PROPERTY})
Expand Down Expand Up @@ -108,20 +112,21 @@ function(hip_gen_exe_target)
if (DEFINED _COMPILE_OPTIONS)
target_compile_options(${_EXE_NAME} PUBLIC ${_COMPILE_OPTIONS})
endif()
target_link_libraries(${_EXE_NAME} Catch2::Catch2)
target_link_libraries(${_EXE_NAME} hip::host hip::device)
target_link_libraries(${_EXE_NAME} Catch2::Catch2 ${GPGPU_LINKER_LIBRARIES})

foreach(arg IN LISTS _UNPARSED_ARGUMENTS)
message(WARNING "Unparsed arguments: ${arg}")
message(WARNING "Unparsed arguments: ${arg}")
endforeach()
# add binary to global list of binaries to install
set_property(GLOBAL APPEND PROPERTY G_INSTALL_EXE_TARGETS ${_EXE_NAME})
catch_discover_tests("${_EXE_NAME}" DISCOVERY_MODE PRE_TEST PROPERTIES SKIP_REGULAR_EXPRESSION "HIP_SKIP_THIS_TEST")
catch_discover_tests("${_EXE_NAME}" DISCOVERY_MODE PRE_TEST PROPERTIES ADD_TAGS_AS_LABELS SKIP_REGULAR_EXPRESSION "HIP_SKIP_THIS_TEST")
file(GLOB CTEST_INC_FILES "${CMAKE_CURRENT_BINARY_DIR}/${_EXE_NAME}-*_include.cmake")
set_property(GLOBAL APPEND PROPERTY G_INSTALL_CTEST_INCLUDE_FILES ${CTEST_INC_FILES})

add_dependencies(${_EXE_NAME} hip_tests_config)

if(NOT _STANDALONE_FLAG EQUAL "1")
break()
endif()
endforeach()
endfunction()
endfunction()
72 changes: 45 additions & 27 deletions catch/cmake/relative_paths.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,13 @@

if len(sys.argv) > 1:
build_folder = sys.argv[1]
install_folder = sys.argv[2]
#print(f"In relative_paths.py [1]: build folder: {build_folder}")
#print(f"In relative_paths.py [2]: install folder: {install_folder}")
install_folder = sys.argv[2]
# print(f"In relative_paths.py [1]: build folder: {build_folder}")
# print(f"In relative_paths.py [2]: install folder: {install_folder}")
else:
print(f"Args not provided. Please provide arg1-build folder-catch_tests, arg2-install folder")
print(
f"Args not provided. Please provide arg1-build folder-catch_tests, arg2-install folder"
)
exit(1)

install_folder = os.path.abspath(install_folder)
Expand All @@ -24,7 +26,7 @@
get_filename_component(CTEST_CURRENT_DIR ${FULL_FILE_PATH} DIRECTORY)
get_filename_component(EXE_PATH ${CTEST_CURRENT_DIR}/.. REALPATH)
"""
if os.name == 'posix':
if os.name == "posix":
library_path_str = """
set(LIB_PATH ${EXE_PATH}/../../../lib)
file(GLOB HIP_LIBS "${LIB_PATH}/libamdhip64*")
Expand All @@ -34,11 +36,12 @@
set(LIB_PATH "/opt/rocm/lib")
endif()
"""
ctest_current_str = ctest_current_str + library_path_str
ctest_current_str += library_path_str

inc_cmake_pattern = "_include.cmake"
ctesttest_pattern = "CTestTestfile.cmake"


def make_test_files_portable(filenames):
"""
changes the absolute paths with relative paths
Expand All @@ -48,59 +51,74 @@ def make_test_files_portable(filenames):
try:
filename = os.path.abspath(filename)
# Read the entire content of the file
with open(filename, 'r') as file:
with open(filename, "r", encoding="utf-8") as file:
file_content = file.read()
#print(f"**Done reading now parsing", filename)
# print(f"**Done reading now parsing", filename)
# 1 replace abs path with filename. Make relative
old_text=os.path.dirname(os.path.abspath(filename))
abs_dir_path = os.path.join(old_text, '')
old_text = os.path.dirname(os.path.abspath(filename))
abs_dir_path = os.path.join(old_text, "")
old_text = abs_dir_path.replace("\\", "/")
if modified_msg not in file_content:
file_content = modified_msg + file_content
# Perform the replacement
modified_content = file_content.replace(old_text, '')
modified_content = file_content.replace(old_text, "")
if inc_cmake_pattern in filename:
# 2 get folder path relative to the current *_include.cmake
# 2 get folder path relative to the current *_include.cmake
if r"get_filename_component(FULL_FILE_PATH" not in modified_content:
modified_content = ctest_current_str + modified_content
modified_content = ctest_current_str + modified_content
# 3 CTEST_FILE to have parameterized relative file
test_cmake_pattern = r"\[==\[(\w+-[a-f0-9]+_tests.cmake)\]==\]"
replace_test_pattern = r"${CTEST_CURRENT_DIR}/\1"
modified_content = re.sub(test_cmake_pattern, replace_test_pattern, modified_content)
modified_content = re.sub(
test_cmake_pattern, replace_test_pattern, modified_content
)
# 4 script to use CatchAddTests.cmake from build rather than src
add_test_pattern = r'include\(".*CatchAddTests\.cmake"\)'
replace_add_test_pattern = 'include("${CTEST_CURRENT_DIR}/CatchAddTests.cmake")'
modified_content = re.sub(add_test_pattern, replace_add_test_pattern, modified_content)
replace_add_test_pattern = (
'include("${CTEST_CURRENT_DIR}/CatchAddTests.cmake")'
)
modified_content = re.sub(
add_test_pattern, replace_add_test_pattern, modified_content
)
# 5 use exe from previous folder
exe_pattern = r"TEST_EXECUTABLE\s+\[==\[(.*?)\]==\]"
replace_exe_pattern = r'TEST_EXECUTABLE ${EXE_PATH}/\1'
modified_content = re.sub(exe_pattern, replace_exe_pattern, modified_content)
replace_exe_pattern = r"TEST_EXECUTABLE ${EXE_PATH}/\1"
modified_content = re.sub(
exe_pattern, replace_exe_pattern, modified_content
)
# 6 include _ctest.cmake file with path
ctest_test_pattern = r'include\("(.*?_tests\.cmake)"\)'
replace_ctest_pattern = r'include("${CTEST_CURRENT_DIR}/\1")'
modified_content = re.sub(ctest_test_pattern, replace_ctest_pattern, modified_content)
modified_content = re.sub(
ctest_test_pattern, replace_ctest_pattern, modified_content
)
# 7 use script folder as cwd
cwd_pattern = r"TEST_WORKING_DIR\s+\[==\[(.*?)\]==\]"
replace_cwd_pattern = r'TEST_WORKING_DIR ${EXE_PATH}'
modified_content = re.sub(cwd_pattern, replace_cwd_pattern, modified_content)
replace_cwd_pattern = r"TEST_WORKING_DIR ${EXE_PATH}"
modified_content = re.sub(
cwd_pattern, replace_cwd_pattern, modified_content
)
# 8 modify ld_library_path
if os.name == 'posix':
if os.name == "posix":
lib_path_pattern = r"TEST_DL_PATHS\s+\[==\[(.*?)\]==\]"
replace_lib_path_pattern = r'TEST_DL_PATHS ${LIB_PATH}'
modified_content = re.sub(lib_path_pattern, replace_lib_path_pattern, modified_content)
replace_lib_path_pattern = r"TEST_DL_PATHS ${LIB_PATH}"
modified_content = re.sub(
lib_path_pattern, replace_lib_path_pattern, modified_content
)

filename = os.path.basename(filename)
install_path = os.path.join(install_script, filename)
# Write the modified content back to the file
with open(install_path, 'w') as file:
with open(install_path, "w", encoding="utf-8") as file:
file.write(modified_content)
#print(f"**Done parsing now writing into", install_path)
# print(f"**Done parsing now writing into", install_path)
except IOError as e:
print(f"Error: '{e}'")
sys.exit({e})
except Exception as e:
print(f"An error occurred: {e}")
sys.exit({e})

inccmake_files = glob.glob(build_folder + "/**/*"+inc_cmake_pattern, recursive=True)

inccmake_files = glob.glob(build_folder + "/**/*" + inc_cmake_pattern, recursive=True)
make_test_files_portable(inccmake_files)
Binary file removed catch/config/__pycache__/common.cpython-310.pyc
Binary file not shown.
Loading
Loading