Skip to content

Fix LineSegmentDetector segfault caused by C++/C# calling convention mismatch - #1877

Merged
shimat merged 1 commit into
mainfrom
claude/competent-shannon-5acdac
May 26, 2026
Merged

shimat merged 1 commit into
mainfrom
claude/competent-shannon-5acdac

Conversation

@shimat

@shimat shimat commented May 26, 2026 •

Copy link
Copy Markdown
Owner

Summary

Fixes #1876.

LineSegmentDetector.Create() returned an instance with a null RawPtr in 4.13.0.20260427, causing a segfault on Detect().

Root cause

PR #1846 updated the C# P/Invoke declarations for
imgproc_Ptr_LineSegmentDetector_get and imgproc_Ptr_LineSegmentDetector_delete to the new ExceptionStatus + out-parameter convention, but the corresponding C++ implementations in imgproc_LineSegmentDetector.h were left in the old
direct-return style. The calling-convention mismatch caused rawPtr to be populated with garbage/null, leading to the segfault.

Fix

Update the two C++ functions to match the convention already used everywhere else in the codebase (e.g. bgsegm.h, calib3d_StereoMatcher.h):

  • Return ExceptionStatus instead of void / raw pointer
  • Wrap the body with BEGIN_WRAP / END_WRAP
  • Pass the result via an output pointer parameter

Also adds LineSegmentDetectorTest covering the exact repro case from the issue.

Summary by CodeRabbit

  • Tests

    • Added comprehensive test suite for LineSegmentDetector functionality, covering object creation, line detection operations, and various output handling scenarios.
  • Refactor

    • Updated LineSegmentDetector API interfaces with enhanced error handling and resource management patterns for improved stability and consistency.

Review Change Stack

…eptionStatus pattern

The migration in #1846 updated the C# P/Invoke declarations for
imgproc_Ptr_LineSegmentDetector_get and imgproc_Ptr_LineSegmentDetector_delete
to use the ExceptionStatus + out-parameter convention, but the corresponding
C++ implementations were left in the old direct-return style, causing a
calling-convention mismatch that resulted in a null RawPtr and subsequent
segfault. Closes #1876.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@shimat shimat self-assigned this May 26, 2026
@shimat shimat added the bug Confirmed defect in OpenCvSharp label May 26, 2026
@coderabbitai

coderabbitai Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 985b3b67-d7eb-4907-b8db-8aa6c697424c

📥 Commits

Reviewing files that changed from the base of the PR and between d721631 and 75b4baa.

📒 Files selected for processing (2)
  • src/OpenCvSharpExtern/imgproc_LineSegmentDetector.h
  • test/OpenCvSharp.Tests/imgproc/LineSegmentDetectorTest.cs

📝 Walkthrough

Walkthrough

This PR updates the LineSegmentDetector C++ wrapper functions to return ExceptionStatus with proper exception wrapping instead of returning void or raw pointers directly. Two functional tests validate the detector creation and line detection across both detection method variants.

Changes

LineSegmentDetector wrapper and validation

Layer / File(s) Summary
LineSegmentDetector wrapper exception handling
src/OpenCvSharpExtern/imgproc_LineSegmentDetector.h
Wrapper functions imgproc_Ptr_LineSegmentDetector_delete and imgproc_Ptr_LineSegmentDetector_get now return ExceptionStatus and wrap operations in BEGIN_WRAP/END_WRAP. The get function takes an output pointer parameter (cv::LineSegmentDetector **returnValue) instead of returning the pointer directly.
LineSegmentDetector functional tests
test/OpenCvSharp.Tests/imgproc/LineSegmentDetectorTest.cs
New test class with CreateAndDetect verifying detector instantiation and line detection with return value, and DetectOutputArray testing the output parameter variant of detection. Both tests create synthetic images with drawn lines and validate non-empty results.

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: fixing a segfault in LineSegmentDetector caused by a C++/C# calling convention mismatch.
Linked Issues check ✅ Passed The changes address issue #1876 by fixing the C++ function signatures in imgproc_LineSegmentDetector.h to match the ExceptionStatus pattern and output parameter convention, restoring correct native-pointer handling.
Out of Scope Changes check ✅ Passed All changes are directly related to fixing the LineSegmentDetector segfault issue. Header modifications fix the calling convention mismatch, and test additions validate the exact scenario described in issue #1876.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/competent-shannon-5acdac

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@shimat
shimat merged commit 939d5fe into main May 26, 2026
1 check passed
@shimat
shimat deleted the claude/competent-shannon-5acdac branch May 26, 2026 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Confirmed defect in OpenCvSharp

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LineSegmentDetector segfaults in 4.13.0.20260427

1 participant