Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions updater/lib/dependabot/dependency_snapshot.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,32 @@ def self.create_from_job_definition(job:, job_definition:)

attr_reader :base_commit_sha, :dependency_files, :dependencies

# Returns the subset of all project dependencies which are permitted
# by the project configuration.
def allowed_dependencies
@allowed_dependencies ||= dependencies.select { |d| job.allowed_update?(d) }
end

# Returns the subset of all project dependencies which are specifically
# requested to be updated by the job definition.
def job_dependencies
return [] unless job.dependencies&.any?
return @job_dependencies if defined? @job_dependencies
Comment on lines +39 to +40
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💅 Not significant!

Are there cases where job.dependencies would change? It seems 🤏 surprising to have a guard clause that checks state before the return @ivar if defined? @ivar clause. I would typically expect that to be a part of the memoization.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we could just remove that line and let it return an implicit empty set it memoizes, I kind of just wanted to call out that we make no attempt to derive the job dependencies if the source is empty - and it shouldn't ever change as all job data is effectively immutable ( something we could maybe enforce later? )


# Gradle, Maven and Nuget dependency names can be case-insensitive and
# the dependency name in the security advisory often doesn't match what
# users have specified in their manifest.
#
# It's technically possibly to publish case-sensitive npm packages to a
# private registry but shouldn't cause problems here as job.dependencies
# is set either from an existing PR rebase/recreate or a security
# advisory.
job_dependency_names = job.dependencies.map(&:downcase)
@job_dependencies = dependencies.select do |dep|
job_dependency_names.include?(dep.name.downcase)
end
end

private

def initialize(job:, base_commit_sha:, dependency_files:)
Expand Down
4 changes: 4 additions & 0 deletions updater/lib/dependabot/environment.rb
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ def self.github_actions?
@github_actions ||= environment_variable("GITHUB_ACTIONS", false)
end

def self.deterministic_updates?
@deterministic_updates ||= environment_variable("UPDATER_DETERMINISTIC", false)
end

def self.job_definition
@job_definition ||= JSON.parse(File.read(job_path))
end
Expand Down
8 changes: 8 additions & 0 deletions updater/lib/dependabot/job.rb
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,14 @@ def reject_external_code?
@reject_external_code
end

# TODO: Remove vulnerability checking
#
# This method does too much, let's make it focused on _just_ determining
# if the given dependency is within the configurations allowed_updates.
#
# The calling operation should be responsible for checking vulnerability
# separately, if required.
Comment on lines +139 to +143
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

#
# rubocop:disable Metrics/PerceivedComplexity
def allowed_update?(dependency)
allowed_updates.any? do |update|
Expand Down
30 changes: 8 additions & 22 deletions updater/lib/dependabot/updater.rb
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ def run
error_handler: error_handler
).perform
rescue *ErrorHandler::RUN_HALTING_ERRORS.keys => e
# TODO: Drop this into Security-specific operations
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤤

if e.is_a?(Dependabot::AllVersionsIgnored) && !job.security_updates_only?
error = StandardError.new(
"Dependabot::AllVersionsIgnored was unexpectedly raised for a non-security update job"
Expand Down Expand Up @@ -581,34 +582,19 @@ def existing_pull_request(updated_dependencies)
end

def dependencies
all_deps = dependency_snapshot.dependencies

# Rebases and security updates have dependencies, version updates don't
if job.dependencies
# Gradle, Maven and Nuget dependency names can be case-insensitive and
# the dependency name in the security advisory often doesn't match what
# users have specified in their manifest.
#
# It's technically possibly to publish case-sensitive npm packages to a
# private registry but shouldn't cause problems here as job.dependencies
# is set either from an existing PR rebase/recreate or a security
# advisory.
job_dependencies = job.dependencies.map(&:downcase)
return all_deps.select do |dep|
job_dependencies.include?(dep.name.downcase)
end
return dependency_snapshot.job_dependencies if job.dependencies

if dependency_snapshot.dependencies.any? && dependency_snapshot.allowed_dependencies.none?
Dependabot.logger.info("Found no dependencies to update after filtering allowed updates")
return []
end

allowed_deps = all_deps.select { |d| job.allowed_update?(d) }
allowed_deps = dependency_snapshot.allowed_dependencies
Comment on lines -602 to +593
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm loving the ergonomics of having dependency_snapshot! 👏

# Return dependencies in a random order, with top-level dependencies
# considered first so that dependency runs which time out don't always hit
# the same dependencies
allowed_deps = allowed_deps.shuffle unless ENV["UPDATER_DETERMINISTIC"]

if all_deps.any? && allowed_deps.none?
Dependabot.logger.info("Found no dependencies to update after filtering allowed " \
"updates")
end
allowed_deps = allowed_deps.shuffle unless Environment.deterministic_updates?
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dig the naming. It adds helpful context to a variable name that I had to think about 🙂


# Consider updating vulnerable deps first. Only consider the first 10,
# though, to ensure they don't take up the entire update run
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,19 +68,12 @@ def perform
:error_handler

def dependencies
all_deps = dependency_snapshot.dependencies

allowed_deps = all_deps.select { |d| job.allowed_update?(d) }
# Return dependencies in a random order, with top-level dependencies
# considered first so that dependency runs which time out don't always hit
# the same dependencies
allowed_deps = allowed_deps.shuffle unless ENV["UPDATER_DETERMINISTIC"]

if all_deps.any? && allowed_deps.none?
if dependency_snapshot.dependencies.any? && dependency_snapshot.allowed_dependencies.none?
Dependabot.logger.info("Found no dependencies to update after filtering allowed updates")
return []
end

allowed_deps
dependency_snapshot.allowed_dependencies
end

# Returns a Dependabot::DependencyChange object that encapsulates the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,18 +44,7 @@ def perform
:created_pull_requests

def dependencies
# Gradle, Maven and Nuget dependency names can be case-insensitive and
# the dependency name in the security advisory often doesn't match what
# users have specified in their manifest.
#
# It's technically possibly to publish case-sensitive npm packages to a
# private registry but shouldn't cause problems here as job.dependencies
# is set either from an existing PR rebase/recreate or a security
# advisory.
job_dependencies = job.dependencies.map(&:downcase)
dependency_snapshot.dependencies.select do |dep|
job_dependencies.include?(dep.name.downcase)
end
dependency_snapshot.job_dependencies
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💥

end

# rubocop:disable Metrics/AbcSize
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,19 +39,16 @@ def perform
:created_pull_requests

def dependencies
all_deps = dependency_snapshot.dependencies

allowed_deps = all_deps.select { |d| job.allowed_update?(d) }
# Return dependencies in a random order, with top-level dependencies
# considered first so that dependency runs which time out don't always hit
# the same dependencies
allowed_deps = allowed_deps.shuffle unless ENV["UPDATER_DETERMINISTIC"]

if all_deps.any? && allowed_deps.none?
if dependency_snapshot.dependencies.any? && dependency_snapshot.allowed_dependencies.none?
Dependabot.logger.info("Found no dependencies to update after filtering allowed updates")
return []
end

allowed_deps
if Environment.deterministic_updates?
dependency_snapshot.allowed_dependencies
else
dependency_snapshot.allowed_dependencies.shuffle
end
end

def check_and_create_pr_with_error_handling(dependency)
Expand Down