Experimental: switch logging to loguru

[antirotor]

Changelog Description

Loguru is a library which aims to bring enjoyable logging in Python. AYON was already customizing logging to handle some colorization and contextualization but using loguru will allow to do it with more consistency and in standard way. It also simplify connecting logging facility to some telemetry/log aggregation system.

TODOs:

• fine-tune highlighting
• implement changes in logger from ynput/ayon-review-desktop
• add api to expose sink to addons

Note

For more information see PR in ayon-launcher

[BigRoy]

Aside of the API changes - I'm not convinced directly that we need loguru so this may need some more discussion.

What would you say for you is the selling point to use it?

I saw this on their repo:

Although logging impact on performances is in most cases negligible, a zero-cost logger would allow to use it anywhere without much concern. In an upcoming release, Loguru's critical functions will be implemented in C for maximum speed.

I wonder if that means we may at some point start having issues with cross-OS binaries and having this in dependency package out of the box, or?

[BigRoy]

Should we also have a process session uuid - so that if you wanted to filter to all logs from a particular session, then this is the go to way.

Then additionally, we should perhaps pass process_session_id to the child processes so they can store parent_process_session_id so that from one session, one could move upstream to the parent sessions?

[antirotor]

This is something that was already there - I just copied it there to maintain compatibility. Something like this would have to be part of the logs definitely, but added to the sink/handler differently.

[BigRoy]

Ah I see - it's taken from here.

[antirotor]

What would you say for you is the selling point to use it?

It is solving issues we have with colorizing, meaningful formatting and logging of exceptions, it is super easy to setup and customize, it provides a lot more information from the start than standard python logging. It is thread-safe by default, lazy evaluating. Better to use this than reinventing the wheel.

You could argue about the speed, sure. I saw some benchmark somewhere and the speed differences between logger were totally marginal for our use case ranging betwee 0.013msec to 0.031msec per log making loguru slower then rest of it...

I would argue that developer friendliness is much more important than speed in case of pipeline code. I would think differently deploying that with the code of some massive scalable cloud app.

You might consider something like structlog that is compatible with standard python logger, for sure. It is probaly feature-wise similar to loguru, faster, somewhat compatible with standard python logging.

[BigRoy]

It is solving issues we have with colorizing, meaningful formatting and logging of exceptions, it is super easy to setup and customize, it provides a lot more information from the start than standard python logging. It is thread-safe by default, lazy evaluating. Better to use this than reinventing the wheel.

• Doesn't this just move the colorizing issue into whomever is writing the log instead of 'automatically coloring things'? The fact that we're colorizing now based on the log message content as opposed to just the logging level and standardized coloring in our default handler I think is where the real issue is?
• Python's built-in logging loggers are also thread-safe.

You could argue about the speed, sure. I saw some benchmark somewhere and the speed differences between logger were totally marginal for our use case ranging betwee 0.013msec to 0.031msec per log making loguru slower then rest of it...

I would argue that developer friendliness is much more important than speed in case of pipeline code. I would think differently deploying that with the code of some massive scalable cloud app.

I've never really been held back by writing log output. Setting up a handler, maybe? But that's something we do once for AYON and it'll live for a long while. I have however had to suffer through slow logs already (usually related to the UIs, not logs themselves). The fact that we do not use logging's lazy evaluation (where it only formats if a handler is attached) could potentially improve performance, but admittedly the majority of cases (e.g. during publishing) we're always handling all log levels anyway, so lazy formatting isn't really a thing because the contents will always need to evaluate.

Anyway, not saying I'm against it. Just don't really see yet why we need the dependency that may affect other things - e.g. since this doesn't go through standard logging it may affect pytest testing as well, etc. So I'd rather have a strong selling point that says "we need it."

[antirotor]

Anyway, not saying I'm against it. Just don't really see yet why we need the dependency that may affect other things - e.g. since this doesn't go through standard logging it may affect pytest testing as well, etc. So I'd rather have a strong selling point that says "we need it."

I don't believe that speed or tests are the issue. Main selling point is support for observability - to have that, you need to output structured logs along "normal" ones, essentially as json. You can write your own handlers, and formatters, handle colorization and highlighting patterns, or you can drop all of that for dependency that is not maintained by us but much bigger community and industry. Think of it as not adding dependency, but more of replacing existing (blessed, colorama, and our own code) with something production proven, providing all the functionality we currently have plus almost out of the box readiness for modern observability pipelines.

Maybe the question is not if to replace it, but with what - loguru or structlog - or spending some time to implement something like that ourselves.