Skip to content

Adds make tidy#8439

Merged
n1zyy merged 3 commits intomainfrom
mattw/make_clobber
May 24, 2023
Merged

Adds make tidy#8439
n1zyy merged 3 commits intomainfrom
mattw/make_clobber

Conversation

@n1zyy
Copy link
Copy Markdown
Contributor

@n1zyy n1zyy commented May 19, 2023

Do people like this? This is a little bit of a ragecoded thing.

I realized my identity-idp clone was using over 4GB of disk space. The bulk of the ridiculous stuff was:

  • More than 1GB of test Attempts API log files, a unique one-off I cleaned up
  • 1.6 GB of vendor/bundle/ stuff
  • 700MB of logs

I wanted something like make tidy to exist, but it was somewhat duplicative of what make setup does. I don't like running make setup often because it's kind of heavy-handed and I don't want to mess with my DB. So I moved the relevant parts into individual Makefile targets.

This is purely a "I want this to exist" thing, so if it's contentious I'm fine withdrawing it.

@n1zyy
Copy link
Copy Markdown
Contributor Author

n1zyy commented May 19, 2023

Just for giggles, my first run of bundle clean yielded:

                                                                        ST 23   mattw/make_clobber
Removing actioncable (7.0.4.1)
Removing actioncable (7.0.4.2)
Removing actionmailbox (7.0.4.1)
Removing actionmailbox (7.0.4.2)
Removing actionmailer (7.0.4.1)
Removing actionmailer (7.0.4.2)
Removing actionpack (7.0.4.1)
Removing actionpack (7.0.4.2)
Removing actiontext (7.0.4.1)
Removing actiontext (7.0.4.2)
Removing actionview (7.0.4.1)
Removing actionview (7.0.4.2)
Removing activejob (7.0.4.1)
Removing activejob (7.0.4.2)
Removing activemodel (7.0.4.1)
Removing activemodel (7.0.4.2)
Removing activerecord (7.0.4.1)
Removing activerecord (7.0.4.2)
Removing activestorage (7.0.4.1)
Removing activestorage (7.0.4.2)
Removing activesupport (7.0.4.1)
Removing activesupport (7.0.4.2)
Removing aws-partitions (1.719.0)
Removing aws-partitions (1.725.0)
Removing aws-sdk-cloudwatchlogs (1.62.0)
Removing aws-sdk-core (3.170.0)
Removing aws-sdk-kms (1.63.0)
Removing aws-sdk-pinpoint (1.70.0)
Removing aws-sdk-pinpointsmsvoice (1.34.0)
Removing aws-sdk-s3 (1.119.1)
Removing aws-sdk-ses (1.49.0)
Removing aws-sdk-sns (1.60.0)
Removing axe-core-api (4.6.0)
Removing axe-core-rspec (4.6.0)
Removing bcrypt (3.1.18)
Removing benchmark-ips (2.11.0)
Removing benchmark-ips (2.12.0)
Removing bindata (2.4.15)
Removing bootsnap (1.16.0)
Removing brakeman (5.4.0)
Removing bundler-audit (0.9.1)
Removing concurrent-ruby (1.2.0)
Removing connection_pool (2.2.5)
Removing connection_pool (2.3.0)
Removing cssbundling-rails (1.1.2)
Removing device_detector (1.1.0)
Removing devise (4.9.0)
Removing dotiw (5.3.3)
Removing email_spec (2.2.1)
Removing errbase (0.2.2)
Removing faker (3.1.1)
Removing faraday (2.6.0)
Removing faraday-net_http (3.0.1)
Removing faraday-retry (2.1.0)
Removing ffi (1.15.5)
Removing formatador (1.1.0)
Removing geocoder (1.8.1)
Removing good_job (3.12.7)
Removing good_job (3.14.0)
Removing google-protobuf-3.22.0-x86_64 (darwin)
Removing google-protobuf-3.22.2-x86_64 (darwin)
Removing guard (2.18.0)
Removing i18n (1.12.0)
Removing irb (1.6.3)
Removing jsbundling-rails (1.0.3)
Removing jwt (2.7.0)
Removing launchy (2.5.2)
Removing listen (3.8.0)
Removing lograge (0.12.0)
Removing loofah (2.19.1)
Removing mail (2.8.0.1)
Removing minitest (5.17.0)
Removing msgpack (1.6.1)
Removing net-sftp (4.0.0)
Removing net-ssh (7.0.1)
Removing net-ssh (7.1.0)
Removing newrelic_rpm (8.16.0)
Removing nokogiri-1.14.2-x86_64 (darwin)
Removing octokit (6.0.1)
Removing octokit (6.1.0)
Removing openssl-signature_algorithm (1.3.0)
Removing parallel_tests (3.8.1)
Removing parser (3.2.0.0)
Removing parser (3.2.1.0)
Removing parser (3.2.1.1)
Removing pg (1.4.6)
Removing pg_query (4.2.0)
Removing phonelib (0.7.7)
Removing premailer (1.15.0)
Removing premailer (1.19.0)
Removing premailer (1.20.0)
Removing premailer-rails (1.11.1)
Removing propshaft (0.7.0)
Removing pry (0.14.2)
Removing psych (5.1.0)
Removing puma (6.1.1)
Removing rack (2.2.6.2)
Removing rack (2.2.6.3)
Removing rack (2.2.6.4)
Removing rack-attack (6.6.1)
Removing rack-cors (2.0.0)
Removing rack-mini-profiler (3.0.0)
Removing rack-proxy (0.7.6)
Removing rack-test (2.0.2)
Removing rack-timeout (0.6.3)
Removing rails (7.0.4.1)
Removing rails (7.0.4.2)
Removing rails-erd (1.7.2)
Removing railties (7.0.4.1)
Removing railties (7.0.4.2)
Removing rb-fsevent (0.11.2)
Removing redis (5.0.5)
Removing redis-client (0.12.0)
Removing redis-client (0.13.0)
Removing redis-client (0.14.0)
Removing redis-namespace (1.10.0)
Removing redis-namespace (1.8.1)
Removing redis-session-store (0.11.5)
Removing regexp_parser (2.7.0)
Removing request_store (1.5.1)
Removing responders (3.1.0)
Removing rotp (6.2.2)
Removing rqrcode (2.1.2)
Removing rspec-core (3.12.1)
Removing rspec-mocks (3.12.3)
Removing rspec-mocks (3.12.4)
Removing rubocop-ast (1.27.0)
Removing rubocop-rails (2.18.0)
Removing ruby-progressbar (1.12.0)
Removing ruby-progressbar (1.13.0)
Removing ruby-saml (1.15.0)
Removing ruby-statistics (3.0.1)
Removing selenium-webdriver (4.8.1)
Removing simple_form (5.2.0)
Removing simplecov-cobertura (2.1.0)
Removing simplecov_json_formatter (0.1.4)
Removing sprockets (4.2.0)
Removing stringio (3.0.5)
Removing strong_migrations (1.4.3)
Removing strong_migrations (1.4.4)
Removing subprocess (1.5.6)
Removing unf_ext (0.0.8.2)
Removing webmock (3.18.1)
Removing xmldsig (0.7.0)
Removing redis-session-store (253aa4f466cf)
Removing saml_idp (d8e7deb7da3a)

vendor/bundle/ is still 1.3 GB though.

Copy link
Copy Markdown
Contributor

@zachmargolis zachmargolis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM small formatting comments

Makefile Outdated
download_acuant_sdk: ## Downloads the most recent Acuant SDK release from Github
@scripts/download_acuant_sdk.sh

clobber_db: ## resets the database for make setup
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra space

Suggested change
clobber_db: ## resets the database for make setup
clobber_db: ## resets the database for make setup

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

I think this explanation makes it worse, but: I had done this deliberately, because it looked like that was the pattern used elsewhere in the file. But looking at it again, nothing else uses 2 spaces. Maybe I saw the two pound signs and wires got crossed in my mind...? Will fix these.

Makefile Outdated
bin/rake db:environment:set
bin/rake dev:prime

clobber_assets: ## removes assets
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra space

Suggested change
clobber_assets: ## removes assets
clobber_assets: ## removes assets

Makefile Outdated
bin/rake assets:clobber
RAILS_ENV=test bin/rake assets:clobber

clobber_logs: ## purges logs
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra space ```suggestion
clobber_logs: ## purges logs

Makefile Outdated
Comment on lines +294 to +295
## Remove assets and logs, and unused gems, but leave DB alone
tidy: clobber_assets clobber_logs
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

putting the comment at the end lets make help read it correctly

Suggested change
## Remove assets and logs, and unused gems, but leave DB alone
tidy: clobber_assets clobber_logs
tidy: clobber_assets clobber_logs ## Remove assets and logs, and unused gems, but leave DB alone

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the "It would look better to say nothing" front... Until I saw this comment, I was unaware of make help! I thought we just had a pattern of commenting everything. 😇

Thanks for these comments; I'm going to clean these up and also make the comments read slightly better now that I have this context.

[skip changelog]
Copy link
Copy Markdown
Contributor

@aduth aduth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment on lines +283 to +285
clobber_assets: ## Removes (clobbers) assets
bin/rake assets:clobber
RAILS_ENV=test bin/rake assets:clobber
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd personally find some value in this one, curious if it is aware of all of the asset outputs, i.e. public/assets (Sprockets), public/packs (Webpack/jsbundling-rails), app/assets/builds (Sass/cssbundling-rails).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://www.rubydoc.info/gems/sprockets-rails/2.3.3 suggests it isn't natively aware of some of those. public/packs is still there (15MB), for example. It looks like we can add them in an initializer, but I'd like to propose that be a separate PR if warranted.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, to be fair, that seems like something the jsbundling-rails gem should be handling, since it's already doing things with other Rake tasks like rake assets:precompile.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@n1zyy Based on what I see here, it can be achieved with javascript:clobber task:

rails/jsbundling-rails#87 (comment)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aduth I like the idea, but I can't seem to verify what it's actually removing.

https://github.com/rails/jsbundling-rails/blob/main/lib/tasks/jsbundling/clobber.rake suggests that it may automatically extend assets:clobber.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(More concretely: I can see that it runs rm_rf Dir["app/assets/builds/**/[^.]*.{js,js.map}"], verbose: false, but I don't actually have those files when running the app locally and don't follow why.)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh! I forgot that we do things a little custom such that we don't output JavaScript to app/assets/builds, which probably explains why you're not seeing them and not seeing the task have any meaningful impact. We'd probably also need something custom to delete public/packs, then.

Co-authored-by: Andrew Duthie <andrew.duthie@gsa.gov>
@n1zyy n1zyy merged commit 47a3926 into main May 24, 2023
@n1zyy n1zyy deleted the mattw/make_clobber branch May 24, 2023 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants