Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mongodb: Use OP_QUERY for buildinfo for wire proto < 6 #330

Merged
merged 1 commit into from
Oct 13, 2021
Merged

mongodb: Use OP_QUERY for buildinfo for wire proto < 6 #330

merged 1 commit into from
Oct 13, 2021

Conversation

dabdine
Copy link
Contributor

@dabdine dabdine commented Oct 12, 2021

Enables the mongodb scanner to collect info from mongodb versions < 3. Also fixes a wire protocol version check that is inconsistent with documentation on MongoDB's official Git

How to Test

for MONGO_VERSION in "2.4" "2.6" "3.2" "3.6" "4.1" "4.2" "4.4" "5.0"; do { CID=$(docker run --rm --name test-mongo -d -p 1111:27017 mongo:$MONGO_VERSION) && sleep 1 && echo -n "127.0.0.1,," | ./zgrab2 mongodb -p 1111 | jq . && docker kill $CID; } done;

Output:

INFO[0000] started grab at 2021-10-12T09:50:38-07:00    
INFO[0000] finished grab at 2021-10-12T09:50:38-07:00   
{"statuses":{"mongodb":{"successes":1,"failures":0}},"start":"2021-10-12T09:50:38-07:00","end":"2021-10-12T09:50:38-07:00","duration":"7.059171ms"}
{
  "ip": "127.0.0.1",
  "data": {
    "mongodb": {
      "status": "success",
      "protocol": "mongodb",
      "result": {
        "is_master": {
          "is_master": true,
          "max_bson_object_size": 16777216,
          "max_message_size_bytes": 48000000,
          "read_only": false
        },
        "build_info": {
          "version": "2.4.14",
          "git_version": "05bebf9ab15511a71bfbded684bb226014c0a553",
          "build_environment": {}
        }
      },
      "timestamp": "2021-10-12T09:50:38-07:00"
    }
  }
}
ae5fdf5b0648bf28c9dbb54cd87c4c3c1dbd47045091e1f7a38156a9ef29668a
INFO[0000] started grab at 2021-10-12T09:50:40-07:00    
INFO[0000] finished grab at 2021-10-12T09:50:40-07:00   
{"statuses":{"mongodb":{"successes":1,"failures":0}},"start":"2021-10-12T09:50:40-07:00","end":"2021-10-12T09:50:40-07:00","duration":"7.197025ms"}
{
  "ip": "127.0.0.1",
  "data": {
    "mongodb": {
      "status": "success",
      "protocol": "mongodb",
      "result": {
        "is_master": {
          "is_master": true,
          "max_wire_version": 2,
          "max_bson_object_size": 16777216,
          "max_write_batch_size": 1000,
          "max_message_size_bytes": 48000000,
          "read_only": false
        },
        "build_info": {
          "version": "2.6.12",
          "git_version": "d73c92b1c85703828b55c2916a5dd4ad46535f6a",
          "build_environment": {}
        }
      },
      "timestamp": "2021-10-12T09:50:40-07:00"
    }
  }
}
bacde47c20b4cff75694d72f5c982572074113d139a031c69b5b79571b571a66
INFO[0000] started grab at 2021-10-12T09:50:42-07:00    
INFO[0000] finished grab at 2021-10-12T09:50:42-07:00   
{"statuses":{"mongodb":{"successes":1,"failures":0}},"start":"2021-10-12T09:50:42-07:00","end":"2021-10-12T09:50:42-07:00","duration":"7.601838ms"}
{
  "ip": "127.0.0.1",
  "data": {
    "mongodb": {
      "status": "success",
      "protocol": "mongodb",
      "result": {
        "is_master": {
          "is_master": true,
          "max_wire_version": 4,
          "max_bson_object_size": 16777216,
          "max_write_batch_size": 1000,
          "max_message_size_bytes": 48000000,
          "read_only": false
        },
        "build_info": {
          "version": "3.2.21",
          "git_version": "1ab1010737145ba3761318508ff65ba74dfe8155",
          "build_environment": {
            "dist_mod": "debian81",
            "dist_arch": "x86_64",
            "cc": "/opt/mongodbtoolchain/bin/gcc: gcc (GCC) 4.8.2",
            "cc_flags": "-fno-omit-frame-pointer -fPIC -fno-strict-aliasing -ggdb -pthread -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -Werror -O2 -Wno-unused-local-typedefs -Wno-unused-function -Wno-deprecated-declarations -Wno-unused-but-set-variable -Wno-missing-braces -fno-builtin-memcmp",
            "cxx": "/opt/mongodbtoolchain/bin/g++: g++ (GCC) 4.8.2",
            "cxx_flags": "-Wnon-virtual-dtor -Woverloaded-virtual -Wno-maybe-uninitialized -std=c++11",
            "link_flags": "-fPIC -pthread -Wl,-z,now -rdynamic -fuse-ld=gold -Wl,-z,noexecstack -Wl,--warn-execstack",
            "target_arch": "x86_64",
            "target_os": "linux"
          }
        }
      },
      "timestamp": "2021-10-12T09:50:42-07:00"
    }
  }
}
ec111d1bd154ae19021e54f7188fc1f7856ecd913e247d7afe5d3ea4e7cb0f0f
INFO[0000] started grab at 2021-10-12T09:50:44-07:00    
INFO[0000] finished grab at 2021-10-12T09:50:44-07:00   
{"statuses":{"mongodb":{"successes":1,"failures":0}},"start":"2021-10-12T09:50:44-07:00","end":"2021-10-12T09:50:44-07:00","duration":"7.646831ms"}
{
  "ip": "127.0.0.1",
  "data": {
    "mongodb": {
      "status": "success",
      "protocol": "mongodb",
      "result": {
        "is_master": {
          "is_master": true,
          "max_wire_version": 6,
          "max_bson_object_size": 16777216,
          "max_write_batch_size": 100000,
          "logical_session_timeout_minutes": 30,
          "max_message_size_bytes": 48000000,
          "read_only": false
        },
        "build_info": {
          "version": "3.6.23",
          "git_version": "d352e6a4764659e0d0350ce77279de3c1f243e5c",
          "build_environment": {
            "dist_mod": "ubuntu1604",
            "dist_arch": "x86_64",
            "cc": "/opt/mongodbtoolchain/v2/bin/gcc: gcc (GCC) 5.4.0",
            "cc_flags": "-fno-omit-frame-pointer -fno-strict-aliasing -ggdb -pthread -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -Werror -O2 -Wno-unused-local-typedefs -Wno-unused-function -Wno-deprecated-declarations -Wno-unused-but-set-variable -Wno-missing-braces -fstack-protector-strong -fno-builtin-memcmp",
            "cxx": "/opt/mongodbtoolchain/v2/bin/g++: g++ (GCC) 5.4.0",
            "cxx_flags": "-Woverloaded-virtual -Wno-maybe-uninitialized -std=c++14",
            "link_flags": "-pthread -Wl,-z,now -rdynamic -Wl,--fatal-warnings -fstack-protector-strong -fuse-ld=gold -Wl,--build-id -Wl,--hash-style=gnu -Wl,-z,noexecstack -Wl,--warn-execstack -Wl,-z,relro",
            "target_arch": "x86_64",
            "target_os": "linux"
          }
        }
      },
      "timestamp": "2021-10-12T09:50:44-07:00"
    }
  }
}
8ab5e1b54be3455830256c88c9427f68d47bae60493a6ee8b213d4c937a06950
INFO[0000] started grab at 2021-10-12T09:50:46-07:00    
INFO[0000] finished grab at 2021-10-12T09:50:46-07:00   
{"statuses":{"mongodb":{"successes":1,"failures":0}},"start":"2021-10-12T09:50:46-07:00","end":"2021-10-12T09:50:46-07:00","duration":"7.09127ms"}
{
  "ip": "127.0.0.1",
  "data": {
    "mongodb": {
      "status": "success",
      "protocol": "mongodb",
      "result": {
        "is_master": {
          "is_master": true,
          "max_wire_version": 8,
          "max_bson_object_size": 16777216,
          "max_write_batch_size": 100000,
          "logical_session_timeout_minutes": 30,
          "max_message_size_bytes": 48000000,
          "read_only": false
        },
        "build_info": {
          "version": "4.1.13",
          "git_version": "441714bc4c70699950f3ac51a5cac41dcd413eaa",
          "build_environment": {
            "dist_mod": "ubuntu1804",
            "dist_arch": "x86_64",
            "cc": "/opt/mongodbtoolchain/v3/bin/gcc: gcc (GCC) 8.2.0",
            "cc_flags": "-fno-omit-frame-pointer -fno-strict-aliasing -ggdb -pthread -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -Werror -O2 -Wno-unused-local-typedefs -Wno-unused-function -Wno-deprecated-declarations -Wno-unused-const-variable -Wno-unused-but-set-variable -Wno-missing-braces -fstack-protector-strong -fno-builtin-memcmp",
            "cxx": "/opt/mongodbtoolchain/v3/bin/g++: g++ (GCC) 8.2.0",
            "cxx_flags": "-Woverloaded-virtual -Wno-maybe-uninitialized -fsized-deallocation -std=c++17",
            "link_flags": "-pthread -Wl,-z,now -rdynamic -Wl,--fatal-warnings -fstack-protector-strong -fuse-ld=gold -Wl,--build-id -Wl,--hash-style=gnu -Wl,-z,noexecstack -Wl,--warn-execstack -Wl,-z,relro",
            "target_arch": "x86_64",
            "target_os": "linux"
          }
        }
      },
      "timestamp": "2021-10-12T09:50:46-07:00"
    }
  }
}
77019854b4e4a354edda35b095b42bcd8bc4dabb7ae2e6433cb444424753855a
INFO[0000] started grab at 2021-10-12T09:50:48-07:00    
INFO[0000] finished grab at 2021-10-12T09:50:48-07:00   
{"statuses":{"mongodb":{"successes":1,"failures":0}},"start":"2021-10-12T09:50:48-07:00","end":"2021-10-12T09:50:48-07:00","duration":"8.083791ms"}
{
  "ip": "127.0.0.1",
  "data": {
    "mongodb": {
      "status": "success",
      "protocol": "mongodb",
      "result": {
        "is_master": {
          "is_master": true,
          "max_wire_version": 8,
          "max_bson_object_size": 16777216,
          "max_write_batch_size": 100000,
          "logical_session_timeout_minutes": 30,
          "max_message_size_bytes": 48000000,
          "read_only": false
        },
        "build_info": {
          "version": "4.2.17",
          "git_version": "be089838c55d33b6f6039c4219896ee4a3cd704f",
          "build_environment": {
            "dist_mod": "ubuntu1804",
            "dist_arch": "x86_64",
            "cc": "/opt/mongodbtoolchain/v3/bin/gcc: gcc (GCC) 8.5.0",
            "cc_flags": "-fno-omit-frame-pointer -fno-strict-aliasing -ggdb -pthread -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -Werror -O2 -Wno-unused-local-typedefs -Wno-unused-function -Wno-deprecated-declarations -Wno-unused-const-variable -Wno-unused-but-set-variable -Wno-missing-braces -fstack-protector-strong -fno-builtin-memcmp",
            "cxx": "/opt/mongodbtoolchain/v3/bin/g++: g++ (GCC) 8.5.0",
            "cxx_flags": "-Woverloaded-virtual -Wno-maybe-uninitialized -fsized-deallocation -std=c++17",
            "link_flags": "-pthread -Wl,-z,now -rdynamic -Wl,--fatal-warnings -fstack-protector-strong -fuse-ld=gold -Wl,--build-id -Wl,--hash-style=gnu -Wl,-z,noexecstack -Wl,--warn-execstack -Wl,-z,relro",
            "target_arch": "x86_64",
            "target_os": "linux"
          }
        }
      },
      "timestamp": "2021-10-12T09:50:48-07:00"
    }
  }
}
135d1d9df3936ab084c977dd2b09a6cdc8ee2a740525fb14b14503c94f6b999f
INFO[0000] started grab at 2021-10-12T09:50:50-07:00    
INFO[0000] finished grab at 2021-10-12T09:50:50-07:00   
{"statuses":{"mongodb":{"successes":1,"failures":0}},"start":"2021-10-12T09:50:50-07:00","end":"2021-10-12T09:50:50-07:00","duration":"6.035159ms"}
{
  "ip": "127.0.0.1",
  "data": {
    "mongodb": {
      "status": "success",
      "protocol": "mongodb",
      "result": {
        "is_master": {
          "is_master": true,
          "max_wire_version": 9,
          "max_bson_object_size": 16777216,
          "max_write_batch_size": 100000,
          "logical_session_timeout_minutes": 30,
          "max_message_size_bytes": 48000000,
          "read_only": false
        },
        "build_info": {
          "version": "4.4.9",
          "git_version": "b4048e19814bfebac717cf5a880076aa69aba481",
          "build_environment": {
            "dist_mod": "ubuntu2004",
            "dist_arch": "x86_64",
            "cc": "/opt/mongodbtoolchain/v3/bin/gcc: gcc (GCC) 8.5.0",
            "cc_flags": "-fno-omit-frame-pointer -fno-strict-aliasing -fasynchronous-unwind-tables -ggdb -pthread -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -Werror -O2 -Wno-unused-local-typedefs -Wno-unused-function -Wno-deprecated-declarations -Wno-unused-const-variable -Wno-unused-but-set-variable -Wno-missing-braces -fstack-protector-strong -fno-builtin-memcmp",
            "cxx": "/opt/mongodbtoolchain/v3/bin/g++: g++ (GCC) 8.5.0",
            "cxx_flags": "-Woverloaded-virtual -Wno-maybe-uninitialized -fsized-deallocation -std=c++17",
            "link_flags": "-pthread -Wl,-z,now -rdynamic -Wl,--fatal-warnings -fstack-protector-strong -fuse-ld=gold -Wl,--no-threads -Wl,--build-id -Wl,--hash-style=gnu -Wl,-z,noexecstack -Wl,--warn-execstack -Wl,-z,relro -Wl,-z,origin -Wl,--enable-new-dtags",
            "target_arch": "x86_64",
            "target_os": "linux"
          }
        }
      },
      "timestamp": "2021-10-12T09:50:50-07:00"
    }
  }
}
9cad5db18ccbd7797584c870c72225aecb65d2553f612508a6f2a11423aec8cb
INFO[0000] started grab at 2021-10-12T09:50:52-07:00    
INFO[0000] finished grab at 2021-10-12T09:50:52-07:00   
{"statuses":{"mongodb":{"successes":1,"failures":0}},"start":"2021-10-12T09:50:52-07:00","end":"2021-10-12T09:50:52-07:00","duration":"7.095128ms"}
{
  "ip": "127.0.0.1",
  "data": {
    "mongodb": {
      "status": "success",
      "protocol": "mongodb",
      "result": {
        "is_master": {
          "is_master": true,
          "max_wire_version": 13,
          "max_bson_object_size": 16777216,
          "max_write_batch_size": 100000,
          "logical_session_timeout_minutes": 30,
          "max_message_size_bytes": 48000000,
          "read_only": false
        },
        "build_info": {
          "version": "5.0.3",
          "git_version": "657fea5a61a74d7a79df7aff8e4bcf0bc742b748",
          "build_environment": {
            "dist_mod": "ubuntu2004",
            "dist_arch": "x86_64",
            "cc": "/opt/mongodbtoolchain/v3/bin/gcc: gcc (GCC) 8.5.0",
            "cc_flags": "-Werror -include mongo/platform/basic.h -fasynchronous-unwind-tables -ggdb -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -fno-omit-frame-pointer -fno-strict-aliasing -O2 -march=sandybridge -mtune=generic -mprefer-vector-width=128 -Wno-unused-local-typedefs -Wno-unused-function -Wno-deprecated-declarations -Wno-unused-const-variable -Wno-unused-but-set-variable -Wno-missing-braces -fstack-protector-strong -Wa,--nocompress-debug-sections -fno-builtin-memcmp",
            "cxx": "/opt/mongodbtoolchain/v3/bin/g++: g++ (GCC) 8.5.0",
            "cxx_flags": "-Woverloaded-virtual -Wno-maybe-uninitialized -fsized-deallocation -std=c++17",
            "link_flags": "-Wl,--fatal-warnings -pthread -Wl,-z,now -fuse-ld=gold -fstack-protector-strong -Wl,--no-threads -Wl,--build-id -Wl,--hash-style=gnu -Wl,-z,noexecstack -Wl,--warn-execstack -Wl,-z,relro -Wl,--compress-debug-sections=none -Wl,-z,origin -Wl,--enable-new-dtags",
            "target_arch": "x86_64",
            "target_os": "linux"
          }
        }
      },
      "timestamp": "2021-10-12T09:50:52-07:00"
    }
  }
}
073ade978aae4261e42d3b4a5935a3631eded272661a7cc8189b0c4a0d74bdd6

Prior to this patch, 2.4 and 2.6 would error (EOF, remote end sends FIN):

{
  "ip": "127.0.0.1",
  "data": {
{"statuses":{"mongodb":{"successes":0,"failures":1}},"start":"2021-10-12T09:52:27-07:00","end":"2021-10-12T09:52:27-07:00","duration":"7.028057ms"}
    "mongodb": {
      "status": "io-timeout",
      "protocol": "mongodb",
      "result": {
        "is_master": {
          "is_master": true,
          "max_bson_object_size": 16777216,
          "max_message_size_bytes": 48000000,
          "read_only": false
        }
      },
      "timestamp": "2021-10-12T09:52:27-07:00",
      "error": "EOF"
    }
  }
}
8f0d3a2ccb33c4e5111501095452d7d8d00d96cc6eeec6235f6a4ceb222cd4bc
INFO[0000] started grab at 2021-10-12T09:52:28-07:00    
INFO[0000] finished grab at 2021-10-12T09:52:29-07:00   
{"statuses":{"mongodb":{"successes":0,"failures":1}},"start":"2021-10-12T09:52:28-07:00","end":"2021-10-12T09:52:29-07:00","duration":"10.529352ms"}
{
  "ip": "127.0.0.1",
  "data": {
    "mongodb": {
      "status": "io-timeout",
      "protocol": "mongodb",
      "result": {
        "is_master": {
          "is_master": true,
          "max_wire_version": 2,
          "max_bson_object_size": 16777216,
          "max_write_batch_size": 1000,
          "max_message_size_bytes": 48000000,
          "read_only": false
        }
      },
      "timestamp": "2021-10-12T09:52:28-07:00",
      "error": "EOF"
    }
  }
}

Notes & Caveats

Noticed via missing results in search.censys.io.

codyprime
codyprime approved these changes Oct 12, 2021
View reviewed changes
Copy link
Member

@codyprime codyprime left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested against several versions & MaxWireVersions. Did not notice any regressions in results, and strict improvement for services with a MaxWireVersion < 3.

@codyprime codyprime merged commit a1fba22 into zmap:master Oct 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@codyprime codyprime codyprime approved these changes

@dadrian dadrian dadrian approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@dabdine @dadrian @codyprime