[Bug] Uncaught (in promise) dicomParser.ByteStream: parameter 'position' cannot be greater than or equal to 'byteArray' length

[Dengjun21]

I am using Cornerstone3D to display images, some images are displayed correctly and the tools are available correctly, but there are certain images that are not displayed correctly and this error is reported:

parameter ‘position’ cannot be greater than or equal to ‘byteArray’ length

Or:

attempt to read past end of buffer'.

I would like to know how to find out the cause of this error, because these images can be displayed normally in DICOM viewer, 3Dslicer can also be used, but some of them appear the above error when using Cornerstone3D, can you help to analyse it? Thank you very much!@yagni

[yagni]

Try running dciodvfy on it first to help you diagnose if there are any problems with the file. If you still have problems, post the (anonymized) file and I can help troubleshoot further.

[tokyobordel]

Hello. I have same error on this test images:
https://github.com/cornerstonejs/cornerstone3D/blob/main/packages/dicomImageLoader/testImages/CTImage.dcm_JPEG2000TransferSyntax_1.2.840.10008.1.2.4.91.dcm
https://github.com/cornerstonejs/cornerstone3D/blob/main/packages/dicomImageLoader/testImages/CTImage.dcm_JPEG2000LosslessOnlyTransferSyntax_1.2.840.10008.1.2.4.90.dcm

dicomParser.readFixedString: attempt to read past end of buffer

For some reason, by the time readFixedString is called, my byteArray is already empty (length == 0). I'm attaching the output of dciodvfy.

First image:

Second image:

I have a few more dicoms with this error and I noticed that these snapshots have transfer syntax 1.2.840.10008.1.2.4.90 and 1.2.840.10008.1.2.4.91. I have no problems with other dicoms (with another transfer syntax), they load and display normally in cornerstone3d.

[Dengjun21]

Very sorry for the late reply @yagni, these days I am making these attempts:

1. dciodvfy -new Checking for dcm file discrepancies

Here's the full results of the test in a picture that can be displayed properly

This is the result of the image check that could not be displayed properly, you can see that the 0018,0088 field is an illegal value -1, so I assumed that this was causing the problem, and uniformly abs(ds.SpacingBetweenSlices)ed the DICOM files that had a negative value for this attribute. But alas, the bug persists.

This is the output of deiodvfy -new after the unified process.

2. So I went to do the application to desensitise a few DICOM files and transmit them to you for some necessary communication. I don't know if a few desensitised DICOM files would be enough for you to help troubleshoot the problem, and if so, would it be OK to use an email address? Sorry again for the late reply.

[yagni]

@tokyobordel Can you paste the code showing how you're using dicomParser? Are you calling readFixedString() directly? I notice both of your files work in the DICOM dump with data dictionary example, so I am unable to reproduce.

@Dengjun21 None of those dciodvfy errors/warnings look like they would cause the dicomParser error you're getting. Does the data dump example successfully dump the file you're having problems with? If you want to send me an anonymized file, just one should be enough; you can attach it to this issue if it's anonymized.

[tokyobordel]

@yagni, no, this method is not called directly, but through cornerstone3D and the error occurs only when I use viewport with type VOLUME. If I use viewport with type STACK, then there is no problem, but I need MPR, so I need viewport with type VOLUME. Maybe I should contact cornerstone3D? Just in case, I'm attaching the code.

` const volumeId = "DICOM_VOLUME";
const viewportId = 'DICOM_VIEWPORT';
const viewportInput = {
viewportId,
type: ViewportType.ORTHOGRAPHIC,
element: input,
defaultOptions: {
orientation: projection,
},
};

        this.renderingEngine.enableElement(viewportInput);

        this.viewport =  this.renderingEngine.getViewport(viewportId) as Types.IVolumeViewport;

        const volumePromise = volumeLoader.createAndCacheVolume(volumeId, {imageIds}).then(volumeLoaded => {
            volumeLoaded.load()

            this.volume = volumeLoaded;
            
            this.viewport.setVolumes([
                { volumeId },
            ]);
        });

        this.viewport.render();`

[Dengjun21]

I uploaded normal and abnormal images to data dump example and I found that the two PixelData field results are different.

The normal one is this

The abnormal one is like this

So I used pydicom to read the file(abnormal one)

actual_length = len(ds.PixelData)
print(f"Actual length of PixelData: {actual_length}")
try:
    pixel_array = ds.pixel_array
    print("Successfully read pixel array, shape:", pixel_array.shape)
except Exception as e:
    print("Failed to read pixel array:", str(e))

The results of code:

Actual length of PixelData: 261638
Successfully read pixel array, shape: (512, 512)

So I tried to fix this problem(length=261646 -> 261638) during the desensitisation operation

actual_length = len(ds.PixelData)
ds[0x7fe0, 0x0010].value = ds.PixelData
ds[0x7fe0, 0x0010].length = actual_length

But it seems that dicomParser's logic won't read my corrected ds[0x7fe0, 0x0010].length information, and it still throws an error!

Do you know how to make the adjustment please?@yagni

[yagni]

@tokyobordel Yes, please make an issue in the cornerstone3D repo. The PixelData in your file looks good and since the data dump example reads it fine, I'm struggling to see a problem with dicomParser itself. Also it looks like those files are displaying just fine in this Cornerstone3D live example so it really seems like the files are fine.

@Dengjun21 Per the standard, the length of an encapsulated PixelData element is undefined (0xFFFFFFFF) so by setting it explicitly, you're corrupting the DICOM. Your dicomParser dump looks okay. I'm not sure the length discrepancy is a problem: PyDicom may just not be including the sequence delimiter in its count. Can you load your image into this Cornerstone3D live example?

[Dengjun21]

you're corrupting the DICOM.

Ok, got it, I'll stop this breaking DICOM standard behaviour thanks @yagni,
then I tried uploading it at show example and it displays fine. Weird...

So I'm guessing because it's streamLoaderVolume? The display here is an image, and the console print at the local demo url confirms it!

I made other attempts, although I could not directly locate and parse the dicomParser file error location, I tried to step-by-step Debug the code that leads to the error. The dicomParser error appears on the console after executing this line of code.

But when I Debug on down the line, it's reporting a different error than the previous Uncaught (in promise) dicomParser.ByteStream:, but this instead

And My Cornerstone3D version:

  "dependencies": {
    "@cornerstonejs/calculate-suv": "^1.1.0",
    "@cornerstonejs/core": "^1.65.0",
    "@cornerstonejs/dicom-image-loader": "^1.65.0",
    "@cornerstonejs/streaming-image-volume-loader": "^1.65.0",
    "@cornerstonejs/tools": "^1.65.0"}

[yagni]

@Dengjun21 Try opening an issue in Cornerstone3D. At this point it looks like it might be an error with how they’re handling encapsulated pixel data, but it’s hard for me to know. You can even point them back to this issue and I’ll offer assistance from the dicomParser side.

@tokyobordel Feel free to link this issue from your Cornerstone3D issue, too, and I can provide assistance there as needed. I’m just not familiar with the Cornerstone3D codebase.