Add --network-trace to dry-run

[brrygrdn]

We are currently investigating some slow running jobs and one thing that is noticeable is that some ecosystem implementations tend to make excessive network calls within core's code.

This change does two things:

• Adds ActiveSupport::Notification as a generic instrumentor for Excon, per the docs
• Adds a --network-trace argument to the dry-run script that optionally prints out every request made during the run along with a total count at the end of the job

Example:

$ bin/dry-run.rb maven "apache/avro" --dir="/lang/java" --cache=files --dep "org.eclipse.jetty:jetty-server" --network-tracing
warning: parser/current is loading parser/ruby27, which recognizes2.7.6-compliant syntax, but you are running 2.7.5.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
=> reading dependency files from cache manifest: ./dry-run/apache/avro/lang/java/cache-manifest-maven.json
=> parsing dependency files
=> updating 1 dependencies: org.eclipse.jetty:jetty-server

=== org.eclipse.jetty:jetty-server (9.4.44.v20210927)
 => checking for updates 1/1
🌍 https//repo.maven.apache.org:443/maven2/org/apache/apache/23/apache-23.pom
🌍 https//repository.apache.org:443/snapshots/org/eclipse/jetty/jetty-server/maven-metadata.xml
🌍 https//repo.maven.apache.org:443/maven2/org/eclipse/jetty/jetty-server/maven-metadata.xml
🌍 https//repository.apache.org:443/snapshots/org/eclipse/jetty/jetty-server/11.0.9/jetty-server-11.0.9.jar
🌍 https//repository.apache.org:443/snapshots/org/eclipse/jetty/jetty-server/11.0.9/jetty-server-11.0.9.jar
🌍 https//repo.maven.apache.org:443/maven2/org/eclipse/jetty/jetty-server/11.0.9/jetty-server-11.0.9.jar
 => latest available version is 11.0.9
 => latest allowed version is 9.4.44.v20210927
🌍 https//repo.maven.apache.org:443/maven2/org/apache/apache/23/apache-23.pom
🌍 https//repository.apache.org:443/snapshots/org/eclipse/jetty/jetty-server/maven-metadata.xml
🌍 https//repo.maven.apache.org:443/maven2/org/eclipse/jetty/jetty-server/maven-metadata.xml
🌍 https//repo.maven.apache.org:443/maven2/org/apache/apache/23/apache-23.pom
🌍 https//repository.apache.org:443/snapshots/org/eclipse/jetty/jetty-servlet/maven-metadata.xml
🌍 https//repo.maven.apache.org:443/maven2/org/eclipse/jetty/jetty-servlet/maven-metadata.xml
🌍 https//repo.maven.apache.org:443/maven2/org/apache/apache/23/apache-23.pom
🌍 https//repository.apache.org:443/snapshots/org/eclipse/jetty/jetty-util/maven-metadata.xml
🌍 https//repo.maven.apache.org:443/maven2/org/eclipse/jetty/jetty-util/maven-metadata.xml
 => requirements to unlock: all
 => requirements update strategy: 
🌍 https//repo.maven.apache.org:443/maven2/org/apache/apache/23/apache-23.pom
🌍 https//repository.apache.org:443/snapshots/org/eclipse/jetty/jetty-servlet/maven-metadata.xml
🌍 https//repo.maven.apache.org:443/maven2/org/eclipse/jetty/jetty-servlet/maven-metadata.xml
🌍 https//repository.apache.org:443/snapshots/org/eclipse/jetty/jetty-servlet/11.0.9/jetty-servlet-11.0.9.jar
🌍 https//repository.apache.org:443/snapshots/org/eclipse/jetty/jetty-servlet/11.0.9/jetty-servlet-11.0.9.jar
🌍 https//repo.maven.apache.org:443/maven2/org/eclipse/jetty/jetty-servlet/11.0.9/jetty-servlet-11.0.9.jar
🌍 https//repo.maven.apache.org:443/maven2/org/apache/apache/23/apache-23.pom
🌍 https//repository.apache.org:443/snapshots/org/eclipse/jetty/jetty-util/maven-metadata.xml
🌍 https//repo.maven.apache.org:443/maven2/org/eclipse/jetty/jetty-util/maven-metadata.xml
🌍 https//repository.apache.org:443/snapshots/org/eclipse/jetty/jetty-util/11.0.9/jetty-util-11.0.9.jar
🌍 https//repository.apache.org:443/snapshots/org/eclipse/jetty/jetty-util/11.0.9/jetty-util-11.0.9.jar
🌍 https//repo.maven.apache.org:443/maven2/org/eclipse/jetty/jetty-util/11.0.9/jetty-util-11.0.9.jar
 => updating org.eclipse.jetty:jetty-server, org.eclipse.jetty:jetty-servlet, org.eclipse.jetty:jetty-util

    ± pom.xml
    ~~~
    43c43
    <     <jetty.version>9.4.44.v20210927</jetty.version>
    ---
    >     <jetty.version>11.0.9</jetty.version>
    ~~~
🌍 Total requests made: '27'

Potential follow-ups

The use of ActiveSupport notification may pave the way for attaching a metric service to core in future.

[jakecoffman]

I like that this makes dry-run output closer to what we see in the job logs. Should we maybe have it on by default?

[brrygrdn]

@jakecoffman Yeah, that's a good point, I don't see the harm in turning it on by default as I think since network calls are our biggest expense keeping them visible is no bad thing

[landongrindheim]

This is awesome 🙌