Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON output fetching fixes #1462

Merged
merged 15 commits into from
Dec 10, 2024
Merged

JSON output fetching fixes #1462

merged 15 commits into from
Dec 10, 2024

Conversation

denis256
Copy link
Member

@denis256 denis256 commented Nov 5, 2024

Description

Included changes:

  • updated Terragrunt version to latest to detect output parsing errors
  • updated Terragrunt log level flags to redirect output
  • added cleanup and formatting of json output
  • added tests to track output fetching with Terragrunt
  • added test to track unicode characters handling

Fixes #1453.
Fixes #1461.

TODOs

Read the Gruntwork contribution guidelines.

  • Update the docs.
  • Run the relevant tests successfully, including pre-commit checks.
  • Ensure any 3rd party code adheres with our license policy or delete this line if its not applicable.
  • Include release notes. If this PR is backward incompatible, include a migration guide.

Release Notes (draft)

Added / Removed / Updated [X].

Updated output parsing from Terragrunt

Migration Guide

@denis256 denis256 requested a review from ryehowell November 8, 2024 16:21
@denis256
Copy link
Member Author

denis256 commented Nov 8, 2024

terraform-aws-ci tests are passing too

image

Copy link

@levkohimins levkohimins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know if my suggestions are not relevant, I will approve it.

}
_, tgLogSet := options.EnvVars["TERRAGRUNT_DISABLE_LOG_FORMATTING"]
if !tgLogSet {
options.EnvVars["TERRAGRUNT_DISABLE_LOG_FORMATTING"] = "true"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This env var is deprecated, please use TERRAGRUNT_LOG_FORMAT=key-value

*But only if you use the latest version that supports log format.


var (
ansiLineRegex = regexp.MustCompile(`(?m)^\x1b\[[0-9;]*m.*`)
tgLogLevel = regexp.MustCompile(`.*time=\S+ level=\S+ prefix=\S+ binary=\S+ msg=.*`)
Copy link

@levkohimins levkohimins Nov 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use TERRAGRUNT_LOG_CUSTOM_FORMAT="%msg(color=disable)" to get rid of cleanJson function at all.

*But only if you use the latest version that supports custom log format.

Copy link
Member Author

@denis256 denis256 Dec 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Output still has lines like:

time=2024-12-05UTC17:32:10Z level=stdout tf-path=terraform msg=Initializing the backend...

which requires cleaning when parsing output

james03160927
james03160927 previously approved these changes Dec 9, 2024
Copy link
Contributor

@james03160927 james03160927 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nit comments but overall LGTM.

}
_, tgLogSet := options.EnvVars["TERRAGRUNT_LOG_FORMAT"]
if !tgLogSet {
options.EnvVars["TERRAGRUNT_LOG_FORMAT"] = "key-value"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain what this does? It could be great to add some comments explanining the default behaviour here and below.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added description

const skipJsonLogLine = " msg="

var (
ansiLineRegex = regexp.MustCompile(`(?m)^\x1b\[[0-9;]*m.*`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we add comments on what these regex mean in plain English?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

Copy link
Contributor

@james03160927 james03160927 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@denis256 denis256 merged commit a70a955 into main Dec 10, 2024
2 of 3 checks passed
@denis256 denis256 deleted the bug/tg-output-handling-1453 branch December 10, 2024 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants