Boost log to file #172

stephengtuggy · 2020-07-09T16:14:44Z

Thank you for submitting a pull request and becoming a contributor to the Vega Strike Core Engine.

Please answer the following:

Code Changes:

Have the PR Validation Tests been run? See https://github.com/vegastrike/Vega-Strike-Engine-Source/wiki/Pull-Request-Validation
This is a documentation change only

Issues:

Please list any related issues
Further implements Proposal: Use boost::logs to capture log data #95

Purpose:

What is this pull request trying to do? Have Boost logging output go to a file; convert more stdout, stderr, and logging messages to go through Boost.Log; log only fatal errors to the console. (I was going to try to finish converting the entire project, but that appears impractical to do within this PR.)
What release is this for? 0.7.0?
Is there a project or milestone we should apply this to? 0.7.x?

with auto rotation, free space checking, and no auto flush (because it's probably too performance-intensive)

BenjamenMeyer · 2020-07-23T03:16:55Z

engine/src/cmd/basecomputer.cpp

            }
        }

-        fflush(stderr);
+        // TODO: Find a way to flush the Boost logs? Console log specifically?


part of the boost log configuration IIRC

@BenjamenMeyer yes, you can set the log sinks to auto-flush after every line of output. I have done that with the console logger, but my thinking is that that will be too much overhead with the file logger; it will slow down the game too much. So we will need a way to manually flush the file log sink when needed.

Luckily, I now have a way to do that:

VSFileSystem::pFileLogSink->flush();

I just haven't updated this file yet to use it.

Thanks for the review, by the way.

This way, I don't have to worry so much about whether a function/method is called before or after initLogging during the startup sequence.

stephengtuggy · 2020-07-27T03:53:59Z

OK, I think this one is ready for review. I was going to try to finish converting the entire project. But other people are working on the same files, of course, and so much is shifting around that that goal seems pretty impractical at this point.

I did get the Boost logging to go to a file by default -- once it has reached the stage of game startup where it is able to. And I believe I converted quite a few of the most common and annoying messages, so they won't show up on the console anymore.

What do you guys think of the log location I chose? $HOME/.vegastrike/logs/. Since this isn't a system program, that seemed like a good location. But maybe I'm wrong.

The file logs include a timestamp in the filename. They rotate at 10 MiB, or at midnight. Logging will stop if free space on the volume drops to 1 GiB.

Any other questions?

nabaco

Hi @stephengtuggy, good job. I think we can merge it, and add some documentation on the week how to use this infrastructure (maybe as art of our coding style) so future developers will know how to use it.

evertvorster · 2020-07-27T05:05:00Z

@stephengtuggy Thanks for the hard work that you put in.
If you don't mind, I'll also use the .vegastrike/logs directory to keep console logs of the stuff that is not in boost. Currently I am using files in .vegastrike. As you mentioned, this is not system software, so /var/log seems a bit far away.

I am a little concerned about the size of the logs. If you have a rather large /home, this would mean that the files will never be deleted. Is this something you turn on, or is the boost logging on by default?

For the console logs, a backup of the last log is made, and then the new log is written in it's place. This gives us a log rotation of 2, keeping the size of the thing under control. How far back in history do you want to go? It may be worth considering a log rotation scheme of 10, and compressing the backups.

stephengtuggy · 2020-07-27T15:24:19Z

Sorry, one more commit is on the way. Need to make sure to flush the logs before exit, wherever possible.

stephengtuggy · 2020-07-27T16:04:54Z

@stephengtuggy Thanks for the hard work that you put in.

You're welcome. :-)

If you don't mind, I'll also use the .vegastrike/logs directory to keep console logs of the stuff that is not in boost. Currently I am using files in .vegastrike. As you mentioned, this is not system software, so /var/log seems a bit far away.

Sure, that's fine.

I am a little concerned about the size of the logs. If you have a rather large /home, this would mean that the files will never be deleted. Is this something you turn on, or is the boost logging on by default?

The boost logging is on by default, but by default, it only logs warning, error, and fatal messages. This means that during some runs of the game, no log file will be created at all. In others, only one or two messages will be logged.

For the console logs, a backup of the last log is made, and then the new log is written in it's place. This gives us a log rotation of 2, keeping the size of the thing under control. How far back in history do you want to go? It may be worth considering a log rotation scheme of 10, and compressing the backups.

That is worth considering. However, I don't know how to do that with Boost::Log. I'm not actually sure if their system supports that without some customization. Maybe a future PR can deal with that.

stephengtuggy · 2020-07-27T17:26:21Z

There. NOW it's ready.

stephengtuggy · 2020-07-27T17:31:23Z

Hi @stephengtuggy, good job. I think we can merge it, and add some documentation on the week how to use this infrastructure (maybe as art of our coding style) so future developers will know how to use it.

Thanks.

Usage instructions remain the same as before. There is at least a preliminary usage guide on the wiki here.

BenjamenMeyer

we can merge once a play test has been confirmed

stephengtuggy · 2020-07-28T14:48:54Z

we can merge once a play test has been confirmed

I have play tested it. Do you mean you want someone else to as well?

BenjamenMeyer · 2020-07-28T14:51:39Z

@stephengtuggy it would be nice if we could have a secondary person do play tests...but that's a lot of extra work so as long as we have someone willing to take ownership of the play test and affirm it I'm good.

evertvorster · 2020-07-28T15:02:40Z

I'll be playing it now, just rebuilding my game engine. :)

stephengtuggy added 6 commits July 7, 2020 15:10

main.cpp: Switch Boost Logging to go to a file

64d8ced

with auto rotation, free space checking, and no auto flush (because it's probably too performance-intensive)

basecomputer.cpp: remove last references to vs_dprintf, stderr, etc.

6122e87

missile_generic.cpp: Remove last vestiges of vs_dprintf, fprintf, etc.

2acbbd8

stardate.cpp: finish replacing vs_dprintf with BOOST_LOG_TRIVIAL

904f417

savegame.cpp: Replaced vs_dprintf with BOOST_LOG_TRIVIAL, but more to go

023805d

main.cpp: log to $HOME/.vegastrike/logs; and to console, just fatal errs

d0a5e35

stephengtuggy self-assigned this Jul 9, 2020

Merge branch 'master' into boost_log_to_file

5a47416

stephengtuggy mentioned this pull request Jul 14, 2020

Updates the vs script to accept 1 argument, to load missions vegastrike/Assets-Production#16

Merged

stephengtuggy added 3 commits July 14, 2020 16:35

Merge branch 'master' into boost_log_to_file

d4a8140

Make pFileLogSink a global variable; flush it when hard exiting

5f0277f

Finish converting main.cpp to use Boost logging

6854bd4

BenjamenMeyer reviewed Jul 23, 2020

View reviewed changes

stephengtuggy added 10 commits July 24, 2020 20:02

Merge remote-tracking branch 'vegastrike/master' into boost_log_to_file

e48ce29

minor: whitespace

1d65680

Split initLogging into 2 parts; vsfilesystem.cpp Boost logging

2a57573

This way, I don't have to worry so much about whether a function/method is called before or after initLogging during the startup sequence.

basecomputer.cpp: Fix a couple of boost::format strings; flush log(s)

860a22b

Merge remote-tracking branch 'vegastrike/master' into boost_log_to_file

0759614

Merge remote-tracking branch 'vegastrike/master' into boost_log_to_file

e60810a

Shift responsibility for flushing logs, incl. console, to fn flushLogs()

6e5a177

minor: whitespace

1450e46

Merge remote-tracking branch 'vegastrike/master' into boost_log_to_file

459d20c

Oops! Accidentally made flushLogs() call itself (infinite recursion)

79177e5

stephengtuggy added the enhancement label Jul 27, 2020

stephengtuggy linked an issue Jul 27, 2020 that may be closed by this pull request

Proposal: Use boost::logs to capture log data #95

Closed

cmd/ai/script.cpp,cmd/script/director.cpp: Boost-log more msgs

3f1abf5

stephengtuggy marked this pull request as ready for review July 27, 2020 03:47

stephengtuggy requested review from BenjamenMeyer, royfalk and nabaco July 27, 2020 03:48

nabaco changed the title ~~DRAFT: Boost log to file~~ Boost log to file Jul 27, 2020

nabaco approved these changes Jul 27, 2020

View reviewed changes

Flush logs before exit()-ing, wherever possible

7039134

stephengtuggy mentioned this pull request Jul 27, 2020

Limit the number of Boost log files to 10, and compress the backups #201

Open

More logging and flushing

c8abcb8

stephengtuggy requested a review from nabaco July 27, 2020 17:29

stephengtuggy requested a review from evertvorster July 27, 2020 20:32

nabaco approved these changes Jul 28, 2020

View reviewed changes

BenjamenMeyer approved these changes Jul 28, 2020

View reviewed changes

BenjamenMeyer added this to the 0.7.x milestone Jul 28, 2020

BenjamenMeyer merged commit 3555faa into vegastrike:master Jul 28, 2020

stephengtuggy deleted the boost_log_to_file branch July 28, 2020 16:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Boost log to file #172

Boost log to file #172

stephengtuggy commented Jul 9, 2020 •

edited

Loading

BenjamenMeyer Jul 23, 2020

stephengtuggy Jul 23, 2020

stephengtuggy Jul 23, 2020

stephengtuggy commented Jul 27, 2020

nabaco left a comment

evertvorster commented Jul 27, 2020

stephengtuggy commented Jul 27, 2020

stephengtuggy commented Jul 27, 2020

stephengtuggy commented Jul 27, 2020

stephengtuggy commented Jul 27, 2020

BenjamenMeyer left a comment

stephengtuggy commented Jul 28, 2020

BenjamenMeyer commented Jul 28, 2020

evertvorster commented Jul 28, 2020

Boost log to file #172

Boost log to file #172

Conversation

stephengtuggy commented Jul 9, 2020 • edited Loading

BenjamenMeyer Jul 23, 2020

Choose a reason for hiding this comment

stephengtuggy Jul 23, 2020

Choose a reason for hiding this comment

stephengtuggy Jul 23, 2020

Choose a reason for hiding this comment

stephengtuggy commented Jul 27, 2020

nabaco left a comment

Choose a reason for hiding this comment

evertvorster commented Jul 27, 2020

stephengtuggy commented Jul 27, 2020

stephengtuggy commented Jul 27, 2020

stephengtuggy commented Jul 27, 2020

stephengtuggy commented Jul 27, 2020

BenjamenMeyer left a comment

Choose a reason for hiding this comment

stephengtuggy commented Jul 28, 2020

BenjamenMeyer commented Jul 28, 2020

evertvorster commented Jul 28, 2020

stephengtuggy commented Jul 9, 2020 •

edited

Loading