Skip to content

Data Pull Script Script (LG-9705)#8345

Merged
zachmargolis merged 12 commits intomainfrom
margolis-data-pull-script
May 8, 2023
Merged

Data Pull Script Script (LG-9705)#8345
zachmargolis merged 12 commits intomainfrom
margolis-data-pull-script

Conversation

@zachmargolis
Copy link
Contributor

@zachmargolis zachmargolis commented May 5, 2023

🎫 Ticket

See RFC for Data Pull Script

🛠 Summary of changes

Adds a new ./bin/data-pull executable with a few subtasks, this is intended for batch-compatible, human- and machine-friendly output.

I know that the RFC is still being reviewed, but I had some time and energy, so I just went for it.

📝

Still left to do:

  • Actually implement profile-lookup (or remove entirely and implement in a future PR)
  • Finish writing specs for each subtask
  • Finalize audit logging, maybe add --reason flag
    • Talked with Paul, we're going to add audit logging in the wrapper outside this script, so this script focuses on the task at hand

@zachmargolis zachmargolis requested review from a team May 5, 2023 00:20
gem 'rspec-retry'
gem 'rspec_junit_formatter'
gem 'shoulda-matchers', '~> 4.0', require: false
gem 'tableparser', require: false
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the specs, I wanted to be able to parse the Terminal::Table output, and as far as I could tell, that gem doesn't have any parsing.

I happened to have my own smol gem (https://github.com/zachmargolis/tableparser) that does this, so I opted to use it here. Open to removing if anybody feels strongly!

It's a pretty limited gem, I YARD doc'ed it as best I could

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of using something one of us "owns" rather than introducing a 3rd-party dependency... no objection here.

Comment on lines +87 to +98
# @api private
# A subtask is a class that has a run method, the type signature should look like:
# +#run(args: Array<String>, include_missing: Boolean) -> Result+
# @return [Class,nil]
def subtask(name)
{
'uuid-lookup' => UuidLookup,
'uuid-convert' => UuidConvert,
'email-lookup' => EmailLookup,
'profile-status' => ProfileStatus,
}[name]
end
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like having these all in the same file is fine for now? Open to feedback if others feel differently

@zachmargolis zachmargolis requested a review from pauldoomgov May 5, 2023 00:26
@zachmargolis zachmargolis changed the title Prototype Data Pull Script Script Data Pull Script Script (LG-9705) May 8, 2023
@zachmargolis zachmargolis marked this pull request as ready for review May 8, 2023 18:35
@zachmargolis
Copy link
Contributor Author

This is ready for review now!

gem 'rspec-retry'
gem 'rspec_junit_formatter'
gem 'shoulda-matchers', '~> 4.0', require: false
gem 'tableparser', require: false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of using something one of us "owns" rather than introducing a 3rd-party dependency... no objection here.

@zachmargolis zachmargolis merged commit 0ba016f into main May 8, 2023
@zachmargolis zachmargolis deleted the margolis-data-pull-script branch May 8, 2023 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants