Rewrite Point data structures as generic SoAs #74
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| self.ppbin, data, self.clust_data.weight, | ||
| self.kernel, self.clust_data.n_dim, block_size, | ||
| device_id) | ||
| cluster_id_is_seed = gpu_hip.mainRun(self.dc_, self.rhoc, self.dm, self.ppbin, |
Check warning
Code scanning / Prospector (reported by Codacy)
local variable 'cluster_id_is_seed' is assigned to but never used (F841) Warning
include/CLUEstering/CLUEstering.hpp
Outdated
| alpaka::memcpy( | ||
| queue_, | ||
| clue::make_host_view(h_points.clusterIndexes(), 2 * nPoints), | ||
| clue::make_device_view(device, d_points.view()->cluster_index, 2 * nPoints), |
Check warning
Code scanning / Flawfinder (reported by Codacy)
Does not check for buffer overflows when copying to destination (CWE-120). Make sure destination can always hold the source data. Warning
| PointsSoA<2> points(floatBuffer.data(), intBuffer.data(), PointShape<2>{n}); | ||
|
|
||
| SUBCASE("Check that the content of the SoA is the same as the buffer") { | ||
| CHECK(std::equal(floatBuffer.begin(), floatBuffer.end(), points.coords())); |
Check warning
Code scanning / Flawfinder (reported by Codacy)
Function does not check the second iterator for over-read conditions (CWE-126). This function is often discouraged by most C++ coding standards in favor of its safer alternatives provided since C++14. Consider using a form of this function that checks the second iterator before potentially overflowing it. Warning test
|
|
||
| SUBCASE("Check that the content of the SoA is the same as the buffer") { | ||
| CHECK(std::equal(floatBuffer.begin(), floatBuffer.end(), points.coords())); | ||
| CHECK(std::equal(intBuffer.begin(), intBuffer.end(), points.clusterIndexes())); |
Check warning
Code scanning / Flawfinder (reported by Codacy)
Function does not check the second iterator for over-read conditions (CWE-126). This function is often discouraged by most C++ coding standards in favor of its safer alternatives provided since C++14. Consider using a form of this function that checks the second iterator before potentially overflowing it. Warning test
| }); | ||
|
|
||
| // check content | ||
| CHECK(std::equal(floatBuffer.begin(), floatBuffer.end(), points.coords())); |
Check warning
Code scanning / Flawfinder (reported by Codacy)
Function does not check the second iterator for over-read conditions (CWE-126). This function is often discouraged by most C++ coding standards in favor of its safer alternatives provided since C++14. Consider using a form of this function that checks the second iterator before potentially overflowing it. Warning test
|
|
||
| // check content | ||
| CHECK(std::equal(floatBuffer.begin(), floatBuffer.end(), points.coords())); | ||
| CHECK(std::equal(floatBuffer.begin(), floatBuffer.end(), view->coords)); |
Check warning
Code scanning / Flawfinder (reported by Codacy)
Function does not check the second iterator for over-read conditions (CWE-126). This function is often discouraged by most C++ coding standards in favor of its safer alternatives provided since C++14. Consider using a form of this function that checks the second iterator before potentially overflowing it. Warning test
| // check content | ||
| CHECK(std::equal(floatBuffer.begin(), floatBuffer.end(), points.coords())); | ||
| CHECK(std::equal(floatBuffer.begin(), floatBuffer.end(), view->coords)); | ||
| CHECK(std::equal(intBuffer.begin(), intBuffer.end(), points.clusterIndexes())); |
Check warning
Code scanning / Flawfinder (reported by Codacy)
Function does not check the second iterator for over-read conditions (CWE-126). This function is often discouraged by most C++ coding standards in favor of its safer alternatives provided since C++14. Consider using a form of this function that checks the second iterator before potentially overflowing it. Warning test
| CHECK(std::equal(floatBuffer.begin(), floatBuffer.end(), points.coords())); | ||
| CHECK(std::equal(floatBuffer.begin(), floatBuffer.end(), view->coords)); | ||
| CHECK(std::equal(intBuffer.begin(), intBuffer.end(), points.clusterIndexes())); | ||
| CHECK(std::equal(intBuffer.begin(), intBuffer.end(), view->clusterIndexes)); |
Check warning
Code scanning / Flawfinder (reported by Codacy)
Function does not check the second iterator for over-read conditions (CWE-126). This function is often discouraged by most C++ coding standards in favor of its safer alternatives provided since C++14. Consider using a form of this function that checks the second iterator before potentially overflowing it. Warning test
There was a problem hiding this comment.
Cppcheck (reported by Codacy) found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
| results = np.vstack([np.zeros(npoints, dtype=np.int32), # cluster ids | ||
| np.zeros(npoints, dtype=np.int32)], # is_seed | ||
| dtype=np.int32) | ||
| coords = np.ascontiguousarray(coords, dtype=np.float32) # if already contiguous this is a no-op |
Check warning
Code scanning / Pylintpython3 (reported by Codacy)
Line too long (108/100) Warning
| results = np.vstack([np.zeros(npoints, dtype=np.int32), # cluster ids | ||
| np.zeros(npoints, dtype=np.int32)], # is_seed | ||
| dtype=np.int32) | ||
| coords = np.ascontiguousarray(coords, dtype=np.float32) # if already contiguous this is a no-op |
Check warning
Code scanning / Pylintpython3 (reported by Codacy)
Line too long (104/100) Warning
| results = np.vstack([np.zeros(npoints, dtype=np.int32), # cluster ids | ||
| np.zeros(npoints, dtype=np.int32)], # is_seed | ||
| dtype=np.int32) | ||
| coords = np.ascontiguousarray(coords, dtype=np.float32) # if already contiguous this is a no-op |
Check warning
Code scanning / Pylintpython3 (reported by Codacy)
Line too long (108/100) Warning
| ndim = len(input_data[:-1]) | ||
| coords = np.vstack([input_data[:-1], # coordinates SoA | ||
| input_data[-1]], # weights | ||
| dtype=np.float32) |
Check warning
Code scanning / Pylint (reported by Codacy)
Wrong continued indentation (remove 1 space). Warning
| dtype=np.float32) | ||
| results = np.vstack([np.zeros(npoints, dtype=np.int32), # cluster ids | ||
| np.zeros(npoints, dtype=np.int32)], # is_seed | ||
| dtype=np.int32) |
Check warning
Code scanning / Pylint (reported by Codacy)
Wrong continued indentation (remove 1 space). Warning
| results = np.vstack([np.zeros(npoints, dtype=np.int32), # cluster ids | ||
| np.zeros(npoints, dtype=np.int32)], # is_seed | ||
| dtype=np.int32) | ||
| coords = np.ascontiguousarray(coords, dtype=np.float32) # if already contiguous this is a no-op |
Check warning
Code scanning / Pylint (reported by Codacy)
Line too long (104/100) Warning
| ndim = input_data[0][0] | ||
| coords = np.vstack([input_data[:-1].T, # coordinates SoA | ||
| input_data[-1]], # weights | ||
| dtype=np.float32) |
Check warning
Code scanning / Pylint (reported by Codacy)
Wrong continued indentation (remove 1 space). Warning
| dtype=np.float32) | ||
| results = np.vstack([np.zeros(npoints, dtype=np.int32), # cluster ids | ||
| np.zeros(npoints, dtype=np.int32)], # is_seed | ||
| dtype=np.int32) |
Check warning
Code scanning / Pylint (reported by Codacy)
Wrong continued indentation (remove 1 space). Warning
| assert len(coords_x0[i]) == 1 | ||
| assert len(coords_x1[i]) == 1 | ||
| assert len(c.clust_data.coords[0]) == 999 | ||
| coordsoa_x0 = c._partial_dimension_dataset([0]) |
Check warning
Code scanning / Prospector (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class (protected-access) Warning test
| assert len(coords_x1[i]) == 1 | ||
| assert len(c.clust_data.coords[0]) == 999 | ||
| coordsoa_x0 = c._partial_dimension_dataset([0]) | ||
| coordsoa_x1 = c._partial_dimension_dataset([1]) |
Check warning
Code scanning / Prospector (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class (protected-access) Warning test
| assert len(coords_x1[i]) == 1 | ||
| assert len(coords_x2[i]) == 1 | ||
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0 = c._partial_dimension_dataset([0]) |
Check warning
Code scanning / Prospector (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class (protected-access) Warning test
| assert len(coords_x2[i]) == 1 | ||
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0 = c._partial_dimension_dataset([0]) | ||
| coordsoa_x1 = c._partial_dimension_dataset([1]) |
Check warning
Code scanning / Prospector (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class (protected-access) Warning test
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0 = c._partial_dimension_dataset([0]) | ||
| coordsoa_x1 = c._partial_dimension_dataset([1]) | ||
| coordsoa_x2 = c._partial_dimension_dataset([2]) |
Check warning
Code scanning / Prospector (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class (protected-access) Warning test
| assert len(coords_x0x2) == c.clust_data.n_points | ||
| assert len(coords_x1x2) == c.clust_data.n_points | ||
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0x1 = c._partial_dimension_dataset([0, 1]) |
Check warning
Code scanning / Prospector (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class (protected-access) Warning test
| assert len(coords_x1x2) == c.clust_data.n_points | ||
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0x1 = c._partial_dimension_dataset([0, 1]) | ||
| coordsoa_x0x2 = c._partial_dimension_dataset([0, 2]) |
Check warning
Code scanning / Prospector (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class (protected-access) Warning test
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0x1 = c._partial_dimension_dataset([0, 1]) | ||
| coordsoa_x0x2 = c._partial_dimension_dataset([0, 2]) | ||
| coordsoa_x1x2 = c._partial_dimension_dataset([1, 2]) |
Check warning
Code scanning / Prospector (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class (protected-access) Warning test
| c2 = clue.clusterer(1., 2., 1.6) | ||
| c2.read_data(box) | ||
| c2.run_clue(dimensions=[0, 1]) | ||
|
|
||
| # check that the result of clustering the 3D dataset using only | ||
| # two dimensions is the same as clustering the 2D dataset | ||
| assert c1.clust_prop == c2.clust_prop | ||
|
|
||
| if __name__ == "__main__": |
Check warning
Code scanning / Prospector (reported by Codacy)
expected 2 blank lines after class or function definition, found 1 (E305) Warning test
There was a problem hiding this comment.
Bandit (reported by Codacy) found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
| assert len(coords_x0[i]) == 1 | ||
| assert len(coords_x1[i]) == 1 | ||
| assert len(c.clust_data.coords[0]) == 999 | ||
| coordsoa_x0 = c._partial_dimension_dataset([0]) |
Check notice
Code scanning / Pylintpython3 (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class Note test
| assert len(coords_x1[i]) == 1 | ||
| assert len(c.clust_data.coords[0]) == 999 | ||
| coordsoa_x0 = c._partial_dimension_dataset([0]) | ||
| coordsoa_x1 = c._partial_dimension_dataset([1]) |
Check notice
Code scanning / Pylintpython3 (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class Note test
| assert len(coords_x1[i]) == 1 | ||
| assert len(coords_x2[i]) == 1 | ||
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0 = c._partial_dimension_dataset([0]) |
Check notice
Code scanning / Pylintpython3 (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class Note test
| assert len(coords_x2[i]) == 1 | ||
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0 = c._partial_dimension_dataset([0]) | ||
| coordsoa_x1 = c._partial_dimension_dataset([1]) |
Check notice
Code scanning / Pylintpython3 (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class Note test
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0 = c._partial_dimension_dataset([0]) | ||
| coordsoa_x1 = c._partial_dimension_dataset([1]) | ||
| coordsoa_x2 = c._partial_dimension_dataset([2]) |
Check notice
Code scanning / Pylintpython3 (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class Note test
| assert len(coords_x0x2) == c.clust_data.n_points | ||
| assert len(coords_x1x2) == c.clust_data.n_points | ||
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0x1 = c._partial_dimension_dataset([0, 1]) |
Check notice
Code scanning / Pylintpython3 (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class Note test
| assert len(coords_x1x2) == c.clust_data.n_points | ||
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0x1 = c._partial_dimension_dataset([0, 1]) | ||
| coordsoa_x0x2 = c._partial_dimension_dataset([0, 2]) |
Check notice
Code scanning / Pylintpython3 (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class Note test
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0x1 = c._partial_dimension_dataset([0, 1]) | ||
| coordsoa_x0x2 = c._partial_dimension_dataset([0, 2]) | ||
| coordsoa_x1x2 = c._partial_dimension_dataset([1, 2]) |
Check notice
Code scanning / Pylintpython3 (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class Note test
| @@ -131,11 +122,17 @@ | |||
| c1 = clue.clusterer(1., 2., 1.6) | |||
| c1.read_data(square) | |||
| c1.run_clue() | |||
|
|
|||
Check warning
Code scanning / Pylintpython3 (reported by Codacy)
Trailing whitespace Warning test
| n_dim : int | ||
| n_points : int | ||
|
|
||
| def __init__(self, coords = None, results = None, n_dim = None, n_points = None): |
Check warning
Code scanning / Pylint (reported by Codacy)
No space allowed around keyword argument assignment Warning
| assert len(coords_x0[i]) == 1 | ||
| assert len(coords_x1[i]) == 1 | ||
| assert len(c.clust_data.coords[0]) == 999 | ||
| coordsoa_x0 = c._partial_dimension_dataset([0]) |
Check notice
Code scanning / Pylint (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class Note test
| assert len(coords_x1[i]) == 1 | ||
| assert len(c.clust_data.coords[0]) == 999 | ||
| coordsoa_x0 = c._partial_dimension_dataset([0]) | ||
| coordsoa_x1 = c._partial_dimension_dataset([1]) |
Check notice
Code scanning / Pylint (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class Note test
| assert len(coords_x1[i]) == 1 | ||
| assert len(coords_x2[i]) == 1 | ||
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0 = c._partial_dimension_dataset([0]) |
Check notice
Code scanning / Pylint (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class Note test
| assert len(coords_x2[i]) == 1 | ||
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0 = c._partial_dimension_dataset([0]) | ||
| coordsoa_x1 = c._partial_dimension_dataset([1]) |
Check notice
Code scanning / Pylint (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class Note test
| assert len(coords_x0x2) == c.clust_data.n_points | ||
| assert len(coords_x1x2) == c.clust_data.n_points | ||
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0x1 = c._partial_dimension_dataset([0, 1]) |
Check notice
Code scanning / Pylint (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class Note test
| assert len(coords_x1x2) == c.clust_data.n_points | ||
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0x1 = c._partial_dimension_dataset([0, 1]) | ||
| coordsoa_x0x2 = c._partial_dimension_dataset([0, 2]) |
Check notice
Code scanning / Pylint (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class Note test
| assert len(c.clust_data.coords[0]) == 10000 | ||
| coordsoa_x0x1 = c._partial_dimension_dataset([0, 1]) | ||
| coordsoa_x0x2 = c._partial_dimension_dataset([0, 2]) | ||
| coordsoa_x1x2 = c._partial_dimension_dataset([1, 2]) |
Check notice
Code scanning / Pylint (reported by Codacy)
Access to a protected member _partial_dimension_dataset of a client class Note test
| @@ -131,11 +122,17 @@ | |||
| c1 = clue.clusterer(1., 2., 1.6) | |||
| c1.read_data(square) | |||
| c1.run_clue() | |||
|
|
|||
Check warning
Code scanning / Pylint (reported by Codacy)
Trailing whitespace Warning test
| c2 = clue.clusterer(1., 2., 1.6) | ||
| c2.read_data(box) | ||
| c2.run_clue(dimensions=[0, 1]) | ||
|
|
||
| # check that the result of clustering the 3D dataset using only | ||
| # two dimensions is the same as clustering the 2D dataset | ||
| assert c1.clust_prop == c2.clust_prop | ||
|
|
||
| if __name__ == "__main__": | ||
| c = clue.clusterer(1., 2., 1.6) |
Check warning
Code scanning / Pylint (reported by Codacy)
Constant name "c" doesn't conform to UPPER_CASE naming style Warning test
Update backend code Update binding modules Update python API Add tests for host-side point SoA Update run_clue for CUDA and HIP Update benchmarking scripts Move test folder Fix device memory access in memcpy
* Add `import_clusterer` method * Add test for new clusterer import * Add docstring * Fix "if main" of test file * Addition to gitignore
* Small fix in getGlobalBin * Update version * Formatting Fix after merge
Fix partial dimensional clustering Update package version
sbaldu
force-pushed
the
feature-numpy_coords
branch
from
00d9725 to
1f934a3
Compare
* Rework points as SoA Update backend code Update binding modules Update python API Add tests for host-side point SoA Update run_clue for CUDA and HIP Update benchmarking scripts Move test folder Fix device memory access in memcpy * Feature clusterer import method (cms-patatrack#76) * Add `import_clusterer` method * Add test for new clusterer import * Add docstring * Fix "if main" of test file * Addition to gitignore * Small fix in getGlobalBin (cms-patatrack#75) * Small fix in getGlobalBin * Update version * Formatting Fix after merge * Separate alpaka input and result buffers Fix partial dimensional clustering Update package version
No description provided.