Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel checks #47

Open
denisdefreyne opened this issue Aug 26, 2018 · 1 comment
Open

Parallel checks #47

denisdefreyne opened this issue Aug 26, 2018 · 1 comment

Comments

@denisdefreyne
Copy link
Member

@Fjan said:

My rather large site has a bunch of checks before being deployed: spell checker, link checker, assets checker, HTML validator, and several more which makes "nanoc check" a pretty long step in my workflow, much longer than the compile step most of the time.

I'd like to submit a feature request to run the checks in parallel, because they seem like a perfect candidate for it. I took a quick look at the code and I think it should be relatively easy since none of the checks changes any state. It would just need some work to use a thread-safe object for the error collection and the output of the check command would need rethinking to prevent it being interleaved. For example, checks could print a single dot to show they're doing something, (or if we want to do it really nice we could make each thread report a progress percentage to the parent).

@Fjan
Copy link

Fjan commented Apr 19, 2024

I'm using this patch in Checks.rb that does little more than adjust the loop to run each check in a separate thread, and a few formatting fixes. It works great.

class Nanoc::Checking::Runner
  ClassID = Struct.new(:identifier)
  def run_checks(classes)
    Nanoc::Core::Compiler.new_for(@site).run_until_reps_built
    puts "Starting checks…"
    length = classes.map { |c| c.identifier.to_s.length }.max + 10
    Parallel.map(classes) do |klass|
      check = klass.create(@site)
      check.run
      puts format("  %-#{length}s%s", "Check #{klass.identifier}: ",check.issues.empty? ? 'ok'.green : 'error'.red)
      check.issues.map!{Nanoc::Checking::Issue.new(_1.description,_1.subject,ClassID.new(klass.identifier))}
      check.issues
    end.reduce(:union)
  end
end

(edited to use parallel gem, because some parts of the check runner are not thread safe. This required me to replace class in the Issue with a Struct to allow it to serialize)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants