Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qzip cannot decompress gzip with QAT HW #90

Open
lianghouxu opened this issue Jul 14, 2023 · 3 comments
Open

qzip cannot decompress gzip with QAT HW #90

lianghouxu opened this issue Jul 14, 2023 · 3 comments

Comments

@lianghouxu
Copy link

lianghouxu commented Jul 14, 2023

use qzip to decompress created with 'qzip -O gzip' GZIP file, only the 1st round to use QAT_HW, then all fall back to QAT_SW.
[root@spr02 mysqlbackup]# qzip -d -k sbtest-0619.sql.gzip.gz
Reading input file sbtest-0619.sql.gzip.gz (536870912 Bytes)
Decompressing...
id1=31, id2=139, cm=8, flag=0
std GZIP
Reading input file sbtest-0619.sql.gzip.gz (536870912 Bytes)
Decompressing...
id1=119, id2=88, cm=5, flag=34
no QZ
Reading input file sbtest-0619.sql.gzip.gz (536870912 Bytes)
Decompressing...
id1=142, id2=115, cm=120, flag=89
no QZ
Reading input file sbtest-0619.sql.gzip.gz (536870912 Bytes)
Decompressing...
id1=112, id2=200, cm=129, flag=235
no QZ

@XingHongChenIntel
Copy link
Contributor

hello, lianghou, thanks for your question. could you please show the more detailed info, especailly for driver version and if you use the intree driver(using yum/rpm to install) or OOT driver(using source code to build). if you use the OOT driver, could you please share the driver config file. and it will be better if you can also share the test file "sbtest-0619.sql.gzip.gz" .

I have used the OOT driver which version is "QAT20_1.0.50", and test data "calgray" to test on "centos 8/kernel 5.18" platform.
and the test result is fine.

@lianghouxu
Copy link
Author

Hi, XingHong, I used OOT driver QAT20_1.0.50, too, and qzip v1.1.2, also built from source code.

The issue should be related to QATzip.

The steps to reproduce the issue:

  1. Get a 10G file, e.g. 10gfile.sql
  2. qzip -O gzip 10gfile.sql
  3. qzip -d 10gfile.sql.gz, then use pcm-accel -qat to monitor QAT performance.

I added some debug info in qatzip_gzip.c, we can see that only the first 512M data can be detected as std gzip correctly, other data all detected incorrectly.

[root@spr02 mysqlbackup]# qzip -d -k sbtest-10g.sql.gz
Reading input file sbtest-10g.sql.gz (536870912 Bytes)
Decompressing...
id1=31, id2=139, cm=8, flag=0
std GZIP
Reading input file sbtest-10g.sql.gz (536870912 Bytes)
Decompressing...
id1=176, id2=68, cm=133, flag=91
no QZ
Reading input file sbtest-10g.sql.gz (536870912 Bytes)
Decompressing...
id1=178, id2=16, cm=26, flag=48
no QZ
Reading input file sbtest-10g.sql.gz (536870912 Bytes)
Decompressing...
id1=233, id2=120, cm=67, flag=198

@XingHongChenIntel
Copy link
Contributor

yes, you are right, I have reproduce this issue, and create jira ticket https://jira.devtools.intel.com/browse/QATAPP-30505 to track this issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants