page object compress_content_stream

[santhosh-sankaran]

Hi,

I am trying to use the compress_content_streams from page object in pypdf.

As my generated pdf after filling forms is around 300MB, trying a lossless compression using this.

I am not able to see any difference in the file size except few bytes from level = 0 to 9

I can see PyPDF2 v2.12.1 and lower version can do better compression. My output comes to around 70-80MB

But from PyPDF2 v3.0.0, the compression does not work as expected. This corrupts the PDF files generated. PDF size increases from 300MB to 320MB

pypdf compression does not corrupt files, but the compression is not much different from original file size. Size remains same around 300MB.

Can I know what I am doing wrong please.

#--------------------------------------------------------
#pypdf code snippet:

if(__COMPRESS__):
    writer = PdfWriter(clone_from = temp_outfile)
    for page in writer.pages:
        page.compress_content_streams(9) # Tried from 0 to 9
    writer.write(outfile)

#--------------------------------------------------------
#PyPDF2 v2.12.1 code snippet:

if(__COMPRESS__):
    reader = PdfReader(temp_outfile)
    writer = PdfWriter()
    writer.clone_document_from_reader(reader)
    for page in writer.pages:
        page.compress_content_streams()
    writer.write(outfile)


#--------------------------------------------------------

COMPRESS is a constant that holds boolean True or False

Can someone explain the behaviour change between PyPDF2 v2.12.1 and pypdf please

Thank you in advance.

[pubpub-zz]

Can you please provide a pdf file showing this issue. Without, no analyiss is possible

[santhosh-sankaran]

Hi @pubpub-zz

Thank you for a reply.

I have attached a dummy template with data that I use to fill in the forms.

I am unable to attach the generated files here as the limit for attachment is 25MB.

Attaching a screen shot of the file sizes here with.

Thank you.
Santhosh Sankaran

form_data.csv
form_template.pdf

[pubpub-zz]

I need the files for analysis
can you use a share file system such as this one : https://filetransfer.io/

[santhosh-sankaran]

Hi @pubpub-zz

Please find the Generated and compressed files in below link

I did pypdf compression with level 9 and it seems to have increased the size of compressed PDF.

https://filetransfer.io/data-package/MMMULIWU#link

Thank you.

Regards,
Santhosh Sankaran

[harshkg1234]

Hi @pubpub-zz

Please find the Generated and compressed files in below link

I did pypdf compression with level 9 and it seems to have increased the size of compressed PDF.

https://filetransfer.io/data-package/MMMULIWU#link

Thank you.

Regards, Santhosh Sankaran

Hi i am also facing same issues in newer pypdf version 4 compressing file giving errors
:-->AttributeError: 'ContentStream' object has no attribute 'indirect_reference'
in this code:

def compress_pdf(input_pdf_path, output_pdf_path):
reader = PdfReader(input_pdf_path)
writer = PdfWriter()

for page in reader.pages:
    # Compress the content streams of the page
    page.compress_content_streams(level=9)
    writer.add_page(page)

# Write the compressed PDF to the output file
with open(output_pdf_path, "wb") as f:
    writer.write(f)

#if i am using
page.compress_content_streams its working fine but sometimes giving compressed pdf more size than original

[pubpub-zz]

Hi @pubpub-zz
Please find the Generated and compressed files in below link
I did pypdf compression with level 9 and it seems to have increased the size of compressed PDF.
https://filetransfer.io/data-package/MMMULIWU#link
Thank you.
Regards, Santhosh Sankaran

Hi i am also facing same issues in newer pypdf version 4 compressing file giving errors :-->AttributeError: 'ContentStream' object has no attribute 'indirect_reference' in this code:

def compress_pdf(input_pdf_path, output_pdf_path): 
    reader = PdfReader(input_pdf_path) 
    writer = PdfWriter()


    for page in reader.pages:
        # Compress the content streams of the page
        page.compress_content_streams(level=9)
        writer.add_page(page)

    # Write the compressed PDF to the output file
    with open(output_pdf_path, "wb") as f:
        writer.write(f)

#if i am using page.compress_content_streams its working fine but sometimes giving compressed pdf more size than original

Apparently You are not dealing with the same issue: @santhosh-sankaran is indicating some issues with size whereas yours deals with an error.
Your code is incorrect: you need first to attach pages to the writer before applying compression. I recommend this writing:

def compress_pdf(input_pdf_path, output_pdf_path): 
    writer = PdfWriter(input_pdf_path)


    for page in writer.pages:
        # Compress the content streams of the page
        page.compress_content_streams(level=9)

    # Write the compressed PDF to the output file
    with open(output_pdf_path, "wb") as f:
        writer.write(f)

[pubpub-zz]

I may have found the discrepancy:
when you look at page 2,4,6 all are using the same content object:

this is not true in the other file (all different objects)

When you look at the original file you can not observe the object reuse

pypdf can not commonolise objects by it own whatever version is used (at least so far I'm aware of). can you confirm you are using the same input (I'm unable to rerun it on old version)

[santhosh-sankaran]

The file with name Generated-Wean*.pdf is my actual input to both PyPDF2-2.12.1 and pypdf-4.3.1

I have kept the same input file and tried compression on that, still not sure why it is producing different results.

As from your reply, I can see that PyPDF2 is able to understand that pages 2, 4 and 6 are all same and is able to group them to have single content replicated.

This is missing from pypdf-4.3.1 where it is not able to identify the pages as similar and produces a different content mapping.

[pubpub-zz]

There is no such capabilities in PyPDF2 nor pypdf to replace a content by a identical content. Can you please rerun only your code snippet on the input pdf and provide result

[santhosh-sankaran]

Hi @pubpub-zz

Please find below link from filetransfer.io

I have kept a screen recording (.webm) file for how I ran the scripts to compress the PDF file. Screen recording can be opened in edge browser.

My original input and outputs after compression are also attached in the filetransfer link.

I am using the same input file for PyPDF2 and pypdf. But still PyPDF2 is able to replace identical content with same content stream is my inference, while pypdf is unable to do that.

https://filetransfer.io/data-package/2AlcEuly

Thank you.
Warm Regards,
Santhosh Sankaran

[pubpub-zz]

I've reviewed the very old code and understand better now
there is a some code (I forgot or not well understood) executed when during the 'write to file/stream' process, that when it meets a indirect object not belonging to the writer object, it checks if an object has the same hash and if so uses this existing object. This code still exists but is no more executed as the objects are copied from the reader into the writer at the first cloning process.

This gives me some new ideas to implement some tools to reduce pdf size.

[pubpub-zz]

issue #2794 has been generated for tracking

[santhosh-sankaran]

@pubpub-zz
Thank you very much.