-
Notifications
You must be signed in to change notification settings - Fork 174
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Breadcrumbs/Mongo integration: Handle 'any_of' cases #522
Conversation
lib/bugsnag/integrations/mongo.rb
Outdated
def sanitize_filter(filter_hash) | ||
filter_hash.map do |key, value| | ||
if value && value.is_a?(Array) | ||
filtered_or = value.map { |filter| sanitize_filter(filter) } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This variable could be slightly misleading because an Array value doesn't necessarily mean it was an $or
(it could have been $and
or something else).
lib/bugsnag/integrations/mongo.rb
Outdated
# @param filter_hash [Hash] the filter hash for the mongo transaction | ||
# | ||
# @return [Hash] the filtered hash | ||
def sanitize_filter(filter_hash) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This method fails on queries such as SomeModel.where(tags: ['a', 'b'])
:
irb(main):001:0> def sanitize_filter(filter_hash)
irb(main):002:1> filter_hash.map do |key, value|
irb(main):003:2* if value && value.is_a?(Array)
irb(main):004:3> filtered_or = value.map { |filter| sanitize_filter(filter) }
irb(main):005:3> [key, filtered_or]
irb(main):006:3> else
irb(main):007:3* [key, '?']
irb(main):008:3> end
irb(main):009:2> end.to_h
irb(main):010:1> end
=> :sanitize_filter
irb(main):011:0> sanitize_filter({'tags' => [1, 2]})
NoMethodError: undefined method `map' for 1:Integer
Did you mean? tap
from (irb):2:in `sanitize_filter'
from (irb):4:in `block (2 levels) in sanitize_filter'
from (irb):4:in `map'
from (irb):4:in `block in sanitize_filter'
from (irb):2:in `each'
from (irb):2:in `map'
from (irb):2:in `sanitize_filter'
from (irb):11
I would change the method/algorithm to something like:
def sanitize_filter_element(element)
if element.is_a?(Array)
element.map { |e| sanitize_filter_element(e) }
elsif element.is_a?(Hash)
element.map { |k, v| [k, sanitize_filter_element(v)] }.to_h
else
'?'
end
end
The only concern I have about the above is that there could be giant arrays for $in
, but I'll pull the old college textbook approach of writing "This is left as an exercise for the reader".
@@ -2,4 +2,5 @@ class MongoModel | |||
include Mongoid::Document | |||
|
|||
field :string_field, type: String | |||
field :numeric_field, type: Numeric |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A Numeric
type isn't in the valid types of https://docs.mongodb.com/mongoid/current/tutorials/mongoid-documents/#fields so I would just use Integer
here instead.
lib/bugsnag/integrations/mongo.rb
Outdated
# Removes values from filter hashes, replacing them with '?' | ||
# | ||
# @param filter_hash [Hash] the filter hash for the mongo transaction | ||
# @param depth [Numeric] the current filter depth |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[Integer]
(as the depth isn't going to be something that isn't a whole number)
lib/bugsnag/integrations/mongo.rb
Outdated
# | ||
# @return [Hash] the filtered hash | ||
def sanitize_filter_hash(filter_hash, depth = 0) | ||
filter_hash.each_with_object({}) do |args, output| |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would change the block params to |(key, value), output|
so you don't need the next line
spec/integrations/mongo_spec.rb
Outdated
module ::Mongo | ||
module Monitoring | ||
COMMAND = 'command' | ||
class Global |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be module
instead of class
to match the mongo gem
spec/integrations/mongo_spec.rb
Outdated
expect(subscriber.respond_to?(:succeeded)).to be true | ||
expect(subscriber.respond_to?(:failed)).to be true | ||
end | ||
require './lib/bugsnag/integrations/mongo' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should probably be load
(and you might need to end the argument with ".rb") instead of require
. The reason is that if the file was required before from some other test, then the require
won't load the file.
spec/integrations/mongo_spec.rb
Outdated
end | ||
|
||
describe "#succeeded" do | ||
it "calls #leave_mongo_beradcrumb with the event_name and event" do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
leave_mongo_breadcrumb (spelling)
spec/integrations/mongo_spec.rb
Outdated
subscriber.send(:leave_mongo_breadcrumb, event_name, event) | ||
end | ||
|
||
it "adds the correct colleciton name for 'getMore' commands" do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
collection (spelling)
spec/integrations/mongo_spec.rb
Outdated
:request_id => "123456", | ||
:duration => "123.456", | ||
:collection => "collection_name_command", | ||
:filter => "{\"a\":\"?\",\"b\":\"?\",\"$or\":[{\"c\":\"?\"},{\"d\":\"?\"}]}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would use single quotes for the outer quotes here so that the blackslashes aren't needed and it is easier to read this string literal
spec/integrations/mongo_spec.rb
Outdated
|
||
describe "#sanitize_filter_hash" do | ||
it "calls into #sanitize_filter_value with the value from each {k,v} pair" do | ||
expect(subscriber).to receive(:sanitize_filter_value).with(1, 0).ordered.and_return('?') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would omit all your sanitize_filter_value
method/message expectations since your equality expectations should be good enough. Method/message expectations tend to be for things that have side effects that you don't want run (e.g. remote/network calls).
spec/integrations/mongo_spec.rb
Outdated
it "removes the command from the request_data" do | ||
subscriber.send(:pop_command, request_id) | ||
command_hash = Bugsnag.configuration.request_data[Bugsnag::MongoBreadcrumbSubscriber::MONGO_COMMAND_KEY] | ||
expect(command_hash).not_to include(request_id => command) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would change this to not_to have_key(request_id)
as the key shouldn't be in at all and the value is not relevant to the test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a few more comments, otherwise lgtm
spec/integrations/mongo_spec.rb
Outdated
@mocked_mongo = true | ||
module ::Mongo | ||
module Monitoring | ||
COMMAND = 'command' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
'Command'
(capital C) to match the mongo gem
spec/integrations/mongo_spec.rb
Outdated
expect(subscriber.send(:sanitize_filter_hash, {:a => 1, :b => 2})).to eq({:a => '?', :b => '?'}) | ||
end | ||
|
||
it "defaults the depth to 0" do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The tests on lines 185-192 no longer provide much value, so I would remove them.
spec/integrations/mongo_spec.rb
Outdated
end | ||
|
||
describe "#failed" do | ||
it "calls #leave_mongo_beradcrumb with the event_name and event" do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
leave_mongo_breadcrumb (spelling)
spec/integrations/mongo_spec.rb
Outdated
subscriber.send(:sanitize_filter_value, {:a => 1}, 0) | ||
end | ||
|
||
it "increments the depth for each call" do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doc string is no longer accurate; I would replace line 203 with line 212 as the input on line 212 recurses and remove this block (so the "is recursive and iterative for array values"
test uses [1, [2, [3]]]
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
* Breadcrumbs/Circular buffer: Added circular buffer class * Breadcrumbs/Circular buffer: Added initial buffer unit tests * Breadcrumbs/Circular buffer: Ensure test file is named correctly * Breadcrumbs/Circular buffer: Code tidy up * Breadcrumbs / Breadcrumb class: Add first pass breadcrumb class with unit tests * Breadcrumbs/Circular buffer rubocop fixes * Breadcrumbs / Breadcrumb class: Rubocop fixes * Breadcrumbs/Cricular buffer: Improve test readability * Breadcrumbs/Breadcrumb: Simplified timestamp, improved tests * Breadcrumbs/Breadcrumb: Rubocop requirements * Breadcrumbs/Breadcrumb: Added doc comments * Breadcrumbs/Circular buffer: Fixed RSpec/Rspec typo * Breadcrumbs/Breadcrumb: Using explicit RSpec.describe call * Breadcrumbs/Validator: Added validator and tests * Breadcrumbs/Validator: Rubocop fixes * Breadcrumbs/Validator: Tidying up from initial review * Breadcrumbs/Validator: Re-add renamed types file * Breadcrumbs/Breadcrumb: Change message => name, add yard comments * Breadcrumbs/Circular buffer: Added yard docs * Breadcrumbs/Breadcrumb: Improve tests, convert to utc sooner * Breadcrumbs/Configuration: Initial configuration changes * Breadcrumbs/Report: Initial report changes to support breadcrumbs * Breadcrumbs/Breadcrumb: Test timestamps match up * Breadcrumbs/Validator: Change 'message' to 'name', use name and meta_data clones * Breadcrumbs/Validator: Refactor toward instance_double & let * Breadcrumbs/Configuration: Add in missing import statement * Breadcrumbs/Circular buffer docs: Improve readability * Breadcrumbs/Configuration: Add configuration tests * Breadcrumbs/Configuration: Added yardoc comments * Breadcrumbs/Configuration: Simplify before_breadcrumb_callbacks, allow automatic_breadcrumb_types editting * Breadcrumbs/Configuration: Add automatic breadcrumbs types editablilty test * Breadcrumbs/Configuration: Add types to Yardoc arrays * Breadcrumbs/Validator: Use an instance_double to reflect Configuration better * Breadcrumbs/Validator: Add Yardoc comments * Breadcrumbs/Validator: Use select over each when filtering * Breadcrumbs/Validator: Clean up tests * Breadcrumbs/Validator: Use double for configuration * Breadcrumbs/Validator: Improved Yardoc * Breadcrumbs/Leave breadcrumb: Add implementation and tests * Breadcrumbs/Middleware: Add breadcrumb middleware * Breadcrumbs/Report: Rubify get_summary method name * Breadcrumbs/Report: Ensure message is set correctly * Breadcrumbs/Report: Change default breadcrumbs from Hash to Array * Breadcrumbs/Report: Ensure breadcrumb meta_data is filtered correctly * Breadcrumbs/Report: Fixed typo * Bugsnag/Report: Added tests * Breadcrumbs/Leave breadcrumb: Update rubocop to allow increased Bugsnag module length * Breadcrumbs/middleware: Rubocop fixes * Breadcrumbs/Leave-breadcrumb: Rubocop improvements/early return * Breadcrumbs/Leave-breadcrumb: Better block styling * Breadcrumbs/Report: Ensure report summary is consistent with dashboard * Breadcrumbs/Report: Better Rubocop report exclusions * Breadcrumbs/Report: Rspec improvements * Breadcrumbs/Notify-breadcrumb: Initial implementation and tests * Breadcrumbs/Notify breadcrumb: Add test against notifying a string * Breadcrumbs/Report: Better checking against Java error * Breadcrumbs/Report: Make summary more accurate * Breadcrumbs/Report update: make summary guard more readable * Breadcrumbs/Update automatic_breadcrumb_types to enabled_automatic_breadcrumb_types * Breadcrumbs: Rails Breadcrumbs (#514) * Breadcrumbs/Rails integration: Initial integration implemenetation * Breadcrumbs/Rails: Minor fixes * Breadcrumbs/Rails: Add initial maze tests * Breadcrumbs/Rails breadcrumbs: Remove plural event data * Breadcrumbs/Rails breadcrumbs: Ammend breadcrumb namespace * Breadcrumbs/Rails breadcrumbs: Remove event_id from allowed keys, change name to event_name * Breadcrumbs/Rails breadcrumbs: Require breadcrumb file when necessary to avoid namespace clashes * Breadcrumbs/Rails breadcrumbs: Fix rubocop issues * Breadcrumbs/Rails breadcrumbs: Clarify namespaces better * Breadcrumbs/Rails breadcrumbs: Remove invalid data from captured breadcrumb definitions * Breadcrumbs/Rails breadcrumbs: Added ActiveJob breadcrumb test * Breadcrumbs/Rails breadcrumbs: Ammended active_job event id * Breadcrumbs/Ruby breadcrumbs: Removed complex breadcrumb metadata * Breadcrumbs/Ruby: Added cache breadcrumb * Breadcrumbs/Ruby breadcrumbs: Fleshed out maze-test expectations where possible * Breadcrumbs/Rails breadcrumbs: Fixed broken test fixture, made fixtures more rails-y * Breadcrumbs: Add Mongo integration (#516) * Breadcrumbs/Rails integration: Initial integration implemenetation * Breadcrumbs/Rails: Minor fixes * Breadcrumbs/Rails: Add initial maze tests * Breadcrumbs/Mongo: Initial subscriber implementation * Breadcrumbs/Rails breadcrumbs: Remove plural event data * Breadcrumbs/Rails breadcrumbs: Ammend breadcrumb namespace * Breadcrumbs/Rails breadcrumbs: Remove event_id from allowed keys, change name to event_name * Breadcrumbs/Rails breadcrumbs: Require breadcrumb file when necessary to avoid namespace clashes * Breadcrumbs/Mongo integration: Fix module name issue * Breadcrumbs/Mongo integration: Maintain consistency with ActiveSupport arguments * Breadcrumbs/Rails breadcrumbs: Fix rubocop issues * Breadcrumbs/Rails breadcrumbs: Clarify namespaces better * Breadcrumbs/Mongo integration: Add integration tests * Breadcrumbs/Mongo integration: Remove example commits * Breadcrumbs/Rails breadcrumbs: Remove invalid data from captured breadcrumb definitions * Breadcrumbs/Mongo integration: Rubocop fixes * Breadcrumbs/Mongo integration: Better doc comments * Breadcrumbs/Mongo integration: Improve tests, add 'failure' test * Breadcrumbs/Mongo integration: Re-add 'success' test * Breadcrumbs/Rails breadcrumbs: Added ActiveJob breadcrumb test * Breadcrumbs/Rails breadcrumbs: Ammended active_job event id * Breadcrumbs/Ruby breadcrumbs: Removed complex breadcrumb metadata * Breadcrumbs/Ruby: Added cache breadcrumb * Breadcrumbs/Ruby breadcrumbs: Fleshed out maze-test expectations where possible * Breadcrumbs/Mongo integration: Rubocop fixes * Breadcrumbs/Rails breadcrumbs: Fixed broken test fixture, made fixtures more rails-y * Breadcrumbs/Mongo integration: Maze fixes * Breadcrumbs/Validator: Allow nil as an acceptable meta_data type (#520) * Breadcrumbs/Mongo: Remove 'started' breadcrumb, add collection detail (#519) * Breadcrumbs/Mongo integration: Remove 'started' event, add 'collection' data * Breadcrumbs/Mongo: Add filter keys * Breadcrumbs/Mongo integration: Made filters consistent with other breadcrumb implementations * Breadcrumbs/Mongo integration: Rubocop fixes * Breadcrumbs/Mongo integration: Address feedback * Breadcrumbs: Additional Rails ActiveRecord data (#521) * Breadcrumbs/Rails breadcrumbs: Capture event_id * Breadcrumbs/Rails breadcrumbs: Capture 'sql' string and redacted bindings * Breadcrumbs/Rails integration: Redact SQL string values * Breadcrumbs/Rails integration: Remove SQL string acquisition * Breadcrumbs/Rails improvements: Remove unnecessary require * Breadcrumbs/Rails integration: Add integration tests for bindings * Breadcrumbs/Ruby integration: lock nokogiri version * Breadcrumbs/Ruby integration: Better meta_data naming, Ruby < 2.1 compatability * Breadcrumbs/Mongo integration: Handle 'any_of' cases (#522) * Breadcrumbs/Mongo integration: Handle 'any_of' cases * Breadcrumbs/Mongo Integration: Improve filter sanitization process and add unit tests * Breadcrumbs/Mongo integration: Ensure tests are notated correctly * Breadcrumbs/Mongo filters: Add 1.9.3 & 2.0.0 compatability * Breadcrumbs/Mongo integration: Appease rubocop * Breadcrumbs/Mongo integration: Minor test and efficieny improvements * Breadcrumbs/Mongo integration: Test improvements * Breadcrumbs/Mongo integrations: Test spelling * Breadcrumbs: Fix rubocop issues (excluding configuration from ClassLength until refactor)
Goal
Enables mongo breadcrumb filters to include nested keys, such as in
any_of
cases.