redirect log to python console #3090

guolinke · 2020-05-16T03:59:03Z

for #1493

python-package/lightgbm/basic.py

include/LightGBM/utils/log.h

guolinke · 2020-05-16T04:59:01Z

@jameslamb it seems this redir method could be used in R. should we unify them?

jameslamb · 2020-05-16T05:26:41Z

@jameslamb it seems this redir method could be used in R. should we unify them?

@guolinke I believe Rprintf and Rf_error that we use from the R headers already redirect logs to the R console:

LightGBM/include/LightGBM/utils/log.h

Lines 137 to 140 in 0341906

    
           #else 
        
             Rprintf("[LightGBM] [%s] ", level_str); 
        
             Rvprintf(format, val); 
        
             Rprintf("\n");

There is a lot of magic in those functions so I'd rather leave them alone than have us maintain code that needs to understand how redirecting output to R works.

guolinke · 2020-05-16T08:33:51Z

python-package/lightgbm/basic.py

@@ -21,13 +21,21 @@
 from .libpath import find_lib_path


+def _log_callback(msg):
+    """Redirect logs from native library into Python console"""
+    print("{0:s}".format(decode_string(msg)), end='')


it seems python 2 don't support end=''. @StrikerRUS @henry0312 any better solution? or we drop the support of python 2?

Just add print_function to the 3rd line

from __future__ import absolute_import, print_function

jameslamb · 2020-05-16T21:41:21Z

I missed an opportunity in #3090 (comment) to say "one of the few places where the R package is ahead of the Python package" 😛

jameslamb · 2020-05-17T01:30:58Z

We recently merged some fixes to CI (#3092). Sorry for the inconvenience, but if you merge the most recent master into this branch it should fix the issues with the R package builds on Travis.

StrikerRUS

@guolinke I'm so happy to see this enhancement! Thank you very very much!

I left some comments below.

include/LightGBM/c_api.h

python-package/lightgbm/basic.py

Co-authored-by: Nikita Titov <[email protected]>

python-package/lightgbm/basic.py

include/LightGBM/utils/log.h

StrikerRUS · 2020-05-18T15:39:00Z

Just rebased to master with the aim to pass all CI tests and removed one word from param description.

StrikerRUS

Amazing! I have been waiting for this so long.

Thanks a lot!

* redir log to python console * fix pylint * Apply suggestions from code review * Update basic.py * Apply suggestions from code review Co-authored-by: Nikita Titov <[email protected]> * Update c_api.h * Apply suggestions from code review * Apply suggestions from code review * super-minor: better wording Co-authored-by: Nikita Titov <[email protected]> Co-authored-by: StrikerRUS <[email protected]>

* Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * Initial CUDA work * redirect log to python console (#3090) * redir log to python console * fix pylint * Apply suggestions from code review * Update basic.py * Apply suggestions from code review Co-authored-by: Nikita Titov <[email protected]> * Update c_api.h * Apply suggestions from code review * Apply suggestions from code review * super-minor: better wording Co-authored-by: Nikita Titov <[email protected]> Co-authored-by: StrikerRUS <[email protected]> * re-order includes (fixes #3132) (#3133) * Revert "re-order includes (fixes #3132) (#3133)" (#3153) This reverts commit 656d267. * Missing change from previous rebase * Minor cleanup and removal of development scripts. * Only set gpu_use_dp on by default for CUDA. Other minor change. * Fix python lint indentation problem. * More python lint issues. * Big lint cleanup - more to come. * Another large lint cleanup - more to come. * Even more lint cleanup. * Minor cleanup so less differences in code. * Revert is_use_subset changes * Another rebase from master to fix recent conflicts. * More lint. * Simple code cleanup - add & remove blank lines, revert unneccessary format changes, remove added dead code. * Removed parameters added for CUDA and various bug fix. * Yet more lint and unneccessary changes. * Revert another change. * Removal of unneccessary code. * temporary appveyor.yml for building and testing * Remove return value in ReSize * Removal of unused variables. * Code cleanup from reviewers suggestions. * Removal of FIXME comments and unused defines. * More reviewers comments cleanup. * More reviewers comments cleanup. * More reviewers comments cleanup. * Fix config variables. * Attempt to fix check-docs failure * Update Paramster.rst for num_gpu * Removing test appveyor.yml * Add �CUDA_RESOLVE_DEVICE_SYMBOLS to libraries to fix linking issue. * Fixed handling of data elements less than 2K. * More reviewers comments cleanup. * Removal of TODO and fix printing of int64_t * Add cuda change for CI testing and remove cuda from device_type in python. * Missed one change form previous check-in * Removal AdditionConfig and fix settings. * Limit number of GPUs to one for now in CUDA. * Update Parameters.rst for previous check-in * Whitespace removal. * Cleanup unused code. * Changed uint/ushort/ulong to unsigned int/short/long to help Windows based CUDA compiler work. * Lint change from previous check-in. * Changes based on reviewers comments. * More reviewer comment changes. * Adding warning for is_sparse. Revert tmp_subset code. Only return FeatureGroupData if not is_multi_val_ * Fix so that CUDA code will compile even if you enable the SCORE_T_USE_DOUBLE define. * Reviewer comment cleanup. * Replace warning with Log message. Removal of some of the USE_CUDA. Fix typo and removal of pragma once. * Remove PRINT debug for CUDA code. * Allow to use of multiple GPUs for CUDA. * More multi-GPUs enablement for CUDA. * More code cleanup based on reviews comments. * Update docs with latest config changes. Co-authored-by: Gordon Fossum <[email protected]> Co-authored-by: ChipKerchner <[email protected]> Co-authored-by: Guolin Ke <[email protected]> Co-authored-by: Nikita Titov <[email protected]> Co-authored-by: StrikerRUS <[email protected]> Co-authored-by: James Lamb <[email protected]>

github-actions · 2023-08-24T13:07:14Z

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.

redir log to python console

a8cc932

guolinke requested review from btrotta, chivee, henry0312, StrikerRUS and wxchan as code owners May 16, 2020 03:59

fix pylint

c2ddd12

guolinke commented May 16, 2020

View reviewed changes

python-package/lightgbm/basic.py Outdated Show resolved Hide resolved

guolinke commented May 16, 2020

View reviewed changes

include/LightGBM/utils/log.h Show resolved Hide resolved

Apply suggestions from code review

1f3080e

guolinke commented May 16, 2020

View reviewed changes

Update basic.py

207c91a

jameslamb added the feature label May 16, 2020

StrikerRUS reviewed May 17, 2020

View reviewed changes

include/LightGBM/c_api.h Show resolved Hide resolved

python-package/lightgbm/basic.py Outdated Show resolved Hide resolved

python-package/lightgbm/basic.py Outdated Show resolved Hide resolved

python-package/lightgbm/basic.py Outdated Show resolved Hide resolved

guolinke commented May 17, 2020

View reviewed changes

python-package/lightgbm/basic.py Outdated Show resolved Hide resolved

guolinke commented May 17, 2020

View reviewed changes

python-package/lightgbm/basic.py Outdated Show resolved Hide resolved

guolinke and others added 2 commits May 17, 2020 22:28

Apply suggestions from code review

05d5fd9

Co-authored-by: Nikita Titov <[email protected]>

Update c_api.h

c013a48

guolinke changed the title ~~redir log to python console~~ redirect log to python console May 17, 2020

guolinke commented May 18, 2020

View reviewed changes

python-package/lightgbm/basic.py Show resolved Hide resolved

guolinke commented May 18, 2020

View reviewed changes

python-package/lightgbm/basic.py Outdated Show resolved Hide resolved

Apply suggestions from code review

deb316a

guolinke commented May 18, 2020

View reviewed changes

include/LightGBM/utils/log.h Outdated Show resolved Hide resolved

guolinke and others added 3 commits May 18, 2020 19:53

Apply suggestions from code review

ddbd5b8

Merge branch 'master' into log-redir

9d01e90

super-minor: better wording

7501476

StrikerRUS approved these changes May 18, 2020

View reviewed changes

StrikerRUS merged commit dea2391 into master May 20, 2020

StrikerRUS deleted the log-redir branch May 20, 2020 18:51

Dronablo mentioned this pull request Jun 10, 2020

Jupyter notebook output is flooded by LGBM output #3156

Closed

jamescolless mentioned this pull request Jan 21, 2021

Logging to Python Log #3805

Closed

github-actions bot locked as resolved and limited conversation to collaborators Aug 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

redirect log to python console #3090

redirect log to python console #3090

guolinke commented May 16, 2020

guolinke commented May 16, 2020

jameslamb commented May 16, 2020

guolinke May 16, 2020

StrikerRUS May 16, 2020

jameslamb commented May 16, 2020

jameslamb commented May 17, 2020

StrikerRUS left a comment

StrikerRUS commented May 18, 2020

StrikerRUS left a comment

github-actions bot commented Aug 24, 2023

redirect log to python console #3090

redirect log to python console #3090

Conversation

guolinke commented May 16, 2020

guolinke commented May 16, 2020

jameslamb commented May 16, 2020

guolinke May 16, 2020

Choose a reason for hiding this comment

StrikerRUS May 16, 2020

Choose a reason for hiding this comment

jameslamb commented May 16, 2020

jameslamb commented May 17, 2020

StrikerRUS left a comment

Choose a reason for hiding this comment

StrikerRUS commented May 18, 2020

StrikerRUS left a comment

Choose a reason for hiding this comment

github-actions bot commented Aug 24, 2023