Add new file logged on the fly

[jb-alvarado]

Hello,
I'm testing your crate at the moment and I would like to know if it is possible to add/remove file loggers on the fly.

I need a option where I can add/and remove file logger dynamically and use then like:

info!(target: "{file1}", "logging message");

info!(target: "{file2}", "logging message");

// ...
// or:

info!(target: "{file}", channel = 1; "logging message");

info!(target: "{file}", channel = 2; "logging message");

// etc.

Do you have any suggestions?

[emabee]

Without trying it out, I'd say: with Logger::log_to_writer you can provide a writer that realises your logic, using itself instances of FileLogWriter.

Let me know if something is missing, or if you think it would make sense to add (parts of) your code to flexi_logger.

[jb-alvarado]

Without trying it out, I'd say: with Logger::log_to_writer you can provide a writer that realises your logic, using itself instances of FileLogWriter.

You mean I should implement my own writer? I was hoping that I don't need to do this 🙂. Because of rotation etc.

Let me know if something is missing, or if you think it would make sense to add (parts of) your code to flexi_logger.

If you are willing to integrate the Paris crate, that would be awesome!

For my project I will also implement a mail logging function (with a queue, so not every message alone will send but all two minutes a bunch), but I think that is to specific to integrate in a logging crate.

[emabee]

The rotation etc is implemented within the FileLogWriter, no worries 😀. You just would add the logic to "add/and remove file logger".

After a first glance, I wouldn't know how a decent integration could look like. Using Paris as an alternative backend (rather than writing to stdout directly) would allow using the colouring style of Paris, but it would only work with stdout going to a terminal -- if the programs output is redirected to a file, then you would get color control bytes in there, right?

[jb-alvarado]

After a first glance, I wouldn't know how a decent integration could look like. Using Paris as an alternative backend (rather than writing to stdout directly) would allow using the colouring style of Paris, but it would only work with stdout going to a terminal -- if the programs output is redirected to a file, then you would get color control bytes in there, right?

Yes, by default the color controls would go to the file. It can be useful for someone who uses less, but opening a logfile in VS Code would be ugly. In my program I open the log files in a web frontend and there I replace the controls with html elements and color the strings.
I could imagine several options:

• color only terminal output
• format tags with paris also in logfiles
• remove every color tag
• don't format the color tags with paris and leave everything untouched so the tags end up in the file.

[jb-alvarado]

Without trying it out, I'd say: with Logger::log_to_writer you can provide a writer that realises your logic, using itself instances of FileLogWriter.

Sorry, I have no idea how this could work. Can you give me some more hints? When I use log_to_writer with a FileLogWriter, then this would be configured static.

[emabee]

You would provide your own class that implements LogWriter and manages FileLogWriter instances and dispatches the incoming calls to them according to your needs.

[jb-alvarado]

Ok, with the help of chatGPT I got this result:

use flexi_logger::writers::{FileLogWriter, LogWriter};
use flexi_logger::{Age, Cleanup, Criterion, DeferredNow, FileSpec, Naming, Record};
use log::{debug, error, info, trace, warn};
use std::collections::HashMap;
use std::io;
use std::sync::{Arc, Mutex};

struct MultiFileLogger {
    writers: Arc<Mutex<HashMap<String, Arc<Mutex<FileLogWriter>>>>>,
}

impl MultiFileLogger {
    pub fn new() -> Self {
        MultiFileLogger {
            writers: Arc::new(Mutex::new(HashMap::new())),
        }
    }

    fn get_writer(&self, target: &str) -> io::Result<Arc<Mutex<FileLogWriter>>> {
        let mut writers = self.writers.lock().unwrap();
        if !writers.contains_key(target) {
            let writer = FileLogWriter::builder(
                FileSpec::default()
                    .suppress_timestamp()
                    .basename("ffplayout")
                    .discriminant(target),
            )
            .append()
            .rotate(
                Criterion::Age(Age::Day),
                Naming::Timestamps,
                Cleanup::KeepLogFiles(7),
            )
            .print_message()
            .try_build()
            .map_err(|e| io::Error::new(io::ErrorKind::Other, e.to_string()))?;
            writers.insert(target.to_string(), Arc::new(Mutex::new(writer)));
        }
        Ok(writers.get(target).unwrap().clone())
    }
}

impl LogWriter for MultiFileLogger {
    fn write(&self, now: &mut DeferredNow, record: &Record) -> io::Result<()> {
        let target = record.target();
        let writer = self.get_writer(target);
        let w = writer?.lock().unwrap().write(now, record);

        w
    }

    fn flush(&self) -> io::Result<()> {
        let writers = self.writers.lock().unwrap();
        for writer in writers.values() {
            writer.lock().unwrap().flush()?;
        }
        Ok(())
    }
}

fn main() {
    let logger = MultiFileLogger::new();

    flexi_logger::Logger::try_with_str("trace")
        .expect("LogSpecification String has errors")
        .print_message()
        .log_to_writer(Box::new(logger))
        .start()
        .unwrap();

    trace!(target: "channel1", "This is a trace message for file1");
    debug!(target: "channel1", "This is a debug message for file1");
    info!(target: "channel2", "This is an info message for file2");
    warn!(target: "channel1", "This is a warning for file1");
    error!(target: "channel2", "This is an error message for file2");
}

Was your idea in the same direction? I have not made any longer runs, but just writing to multiple files works. Hopefully also the rotation :-).

Is there also a way log name the files without _rCURRENT and the rotation only with date (YYYY-mm-dd)? The simpler date format would make it more simple to read the log files in other programs.

[emabee]

Yes, that's a pretty starter for what I had in mind. LogWriter has some provided methods, which you might want to look at and maybe implement explicitly.

Is there also a way log name the files without _rCURRENT?

Yes, with Naming::TimestampsDirect instead of Naming::Timestamps.

and the rotation only with date (YYYY-mm-dd)?

That does not yet exist, but could be offered as an option. Would you want to ask chatGPT to provide a PR for that? 🤩