Skip to content

Print query detail when client uses -f or --execute with --debug option#12216

Closed
ebyhr wants to merge 1 commit intoprestodb:masterfrom
ebyhr:cli-debug
Closed

Print query detail when client uses -f or --execute with --debug option#12216
ebyhr wants to merge 1 commit intoprestodb:masterfrom
ebyhr:cli-debug

Conversation

@ebyhr
Copy link
Contributor

@ebyhr ebyhr commented Jan 12, 2019

Previously, presto cli with -f or --exceucte only shows result set even if --debug option is used.
By this commit, presto cli prints the details the same as interactive mode.

-f and --debug

$ cat test.sql
select * from tpch.sf10000.customer limit 1;

$ presto --debug -f test.sql
"750000001","Customer#750000001","LC4O0BHw2 wNroftaCEp0SDGZ0r1SQV4nm4Gt","11","21-456-231-7942","-765.35","HOUSEHOLD","s wake carefully outside the regular instructions. ironicall"

Query 20190112_063454_00067_x3xmi, FINISHED, 1 node
http://localhost:8080/ui/query.html?20190112_063454_00067_x3xmi
Splits: 21 total, 20 done (95.24%)
CPU Time: 0.1s total,  125K rows/s,     0B/s, 70% active
Per Node: 0.9 parallelism,  107K rows/s,     0B/s
Parallelism: 0.9
Peak Memory: 0B
0:00 [16.4K rows, 0B] [107K rows/s, 0B/s]

--execute and --debug

$ presto --debug --execute "select * from tpch.sf10000.customer limit 1;"
"1125000001","Customer#1125000001","Uj6zjz2geB9Y9mDZlmgiEGvr3","23","33-135-417-9700","-180.05","HOUSEHOLD","l instructions. carefully special"

Query 20190112_063519_00068_x3xmi, FINISHED, 1 node
http://localhost:8080/ui/query.html?20190112_063519_00068_x3xmi
Splits: 21 total, 21 done (100.00%)
CPU Time: 0.1s total,  156K rows/s,     0B/s, 88% active
Per Node: 1.0 parallelism,  162K rows/s,     0B/s
Parallelism: 1.0
Peak Memory: 0B
0:00 [21.9K rows, 0B] [162K rows/s, 0B/s]

No changes if --debug isn't used.
-f

$ presto -f test.sql
"1125000001","Customer#1125000001","Uj6zjz2geB9Y9mDZlmgiEGvr3","23","33-135-417-9700","-180.05","HOUSEHOLD","l instructions. carefully special"

--execute

$ presto --execute "select * from tpch.sf10000.customer limit 1;"
"375000001","Customer#375000001","A2qPnPHny3EfSYRt5HRK xbO25","14","24-835-946-7501","873.28","HOUSEHOLD"," dogged requests. packages are bold"

@nezihyigitbasi
Copy link
Contributor

With this change the query details will be printed to stdout, which is usually not a good idea. -f is usually used for batch runs/scripting and the output (which was previously only data) is usually redirected to a file. Now the output file will contain both data + stats, etc., which will break existing scripts and also will make difficult to pipe stuff together with this option (as the output will be csv/tsv + some other unstructured text, the query details).

Posix defines these streams as "standard output (for writing conventional output), and standard error (for writing diagnostic output)", so I think in -f mode we should write these details to stderr.

@ebyhr
Copy link
Contributor Author

ebyhr commented Jan 15, 2019

@nezihyigitbasi Thank you for your comment. Let me fix the output direction to stderr.

Previously, presto cli with -f or --exceucte only shows result set even if
--debug option is used. By this commit, presto cli prints the details the
same as interactive mode.
@ebyhr
Copy link
Contributor Author

ebyhr commented Jan 15, 2019

Changed output stream to stderr.

$ presto --debug -f test.sql 2>/dev/null
"7500001","Customer#007500001","o NIcrMHVcnYy3b6Or","24","34-544-477-2141","4996.81","BUILDING","ully alongside of the deposits. even, even f"
$
$ presto --debug -f test.sql
"11250001","Customer#011250001","RNFodMnae9vkA5Bswzb42rnp","14","24-397-921-1461","4930.66","HOUSEHOLD"," along the even packages nag according to the "

Query 20190115_132958_00012_vncji, FINISHED, 1 node
http://localhost:8080/ui/query.html?20190115_132958_00012_vncji
Splits: 21 total, 21 done (100.00%)
CPU Time: 0.1s total,  159K rows/s,     0B/s, 76% active
Per Node: 0.9 parallelism,  141K rows/s,     0B/s
Parallelism: 0.9
Peak Memory: 0B
0:00 [21.9K rows, 0B] [141K rows/s, 0B/s]
$ presto --debug --execute "select * from tpch.sf10000.customer limit 1;" 2>/dev/null
"1","Customer#000000001","IVhzIApeRb ot,c,E","15","25-989-741-2988","711.56","BUILDING","to the even, regular platelets. regular, ironic epitaphs nag e"```
$
$ presto --debug --execute "select * from tpch.sf10000.customer limit 1;"
"750000001","Customer#750000001","LC4O0BHw2 wNroftaCEp0SDGZ0r1SQV4nm4Gt","11","21-456-231-7942","-765.35","HOUSEHOLD","s wake carefully outside the regular instructions. ironicall"

Query 20190115_133029_00013_vncji, FINISHED, 1 node
http://localhost:8080/ui/query.html?20190115_133029_00013_vncji
Splits: 21 total, 21 done (100.00%)
CPU Time: 0.1s total,  153K rows/s,     0B/s, 67% active
Per Node: 0.9 parallelism,  145K rows/s,     0B/s
Parallelism: 0.9
Peak Memory: 0B
0:00 [21.9K rows, 0B] [145K rows/s, 0B/s]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants