Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

-u --update機能 #391

Closed
YamatoSecurity opened this issue Feb 3, 2022 · 3 comments · Fixed by #392
Closed

-u --update機能 #391

YamatoSecurity opened this issue Feb 3, 2022 · 3 comments · Fixed by #392
Assignees
Labels
enhancement New feature or request Priority:High
Milestone

Comments

@YamatoSecurity
Copy link
Collaborator

rulesフォルダが空の場合は、ユーザがhayabusa-rulesリポからルールをダウンロードしていない可能性が非常に高いです。git submodule update --initコマンド等でダウンロードできますが、gitをインストールしていないユーザも多いので、--update機能で最初のルールダウンロードもできると便利です。

まず、hayabusaを--rules--updateオプション指定なしで実行した際に(通常のスキャンの際に)、rulesフォルダが空の場合は以下のエラーを出力します:
"You currently do not have any rules in your rules folder.
You can download them from https://github.com/Yamato-Security/hayabusa-rules or run the -u / --update command to get the latest rules."

--updateコマンドを実行する場合:
まず、他のオプションがあるかどうかチェックし、他のオプションが指定された場合(例: hayabusa.exe --update -d sample/ -o hoge.csv等々)は、コマンドオプションがおかしいので、エラーメッセージを出力します:
"You cannot specify any other options when using -u / --update"

rulesディレクトリ配下にhayabusaとsigmaディレクトリが無い場合は、https://github.com/Yamato-Security/hayabusa-rulesからルールをダウンロードして、./rules配下にルールを置きます。

rulesディレクトリ配下にhayabusaとsigmaディレクトリがある場合は、まず以下の警告を出力します:
"All of your rule files in the rules folder will be deleted and replaced by the latest rules https://github.com/Yamato-Security/hayabusa-rules. Is this ok? (y/N)"

ユーザが"y"を入力すると、hayabusaとsigmaルールディレクトリを消して、新しいルールをダウンロードします。(rsyncのように差分だけをダウンロードできたら理想です。)
ユーザが例えば、rules/testingrules/my-rulesというhayabusaとsigmaディレクトリ以外のフォルダがある場合は上書きされないようにしたいです。

後で、Readmeで自分のルールが上書きされないように、別のディレクトリ(/rules/my-rules等)に置いて下さいというように記載しておきます。

@hitenkoku
Copy link
Collaborator

https://docs.rs/git2/latest/git2/ を使って対応できないか確認する。

@hitenkoku hitenkoku self-assigned this Feb 3, 2022
@hitenkoku
Copy link
Collaborator

git clone [hayabusa-rules] rules でrulesフォルダをクローンする機能をつける。rulesフォルダで重複した場合はエラーが出るはず(ライブラリの実装を確認する)

@hitenkoku hitenkoku added this to the v1.2 milestone Feb 3, 2022
@hitenkoku hitenkoku added enhancement New feature or request Priority:High labels Feb 3, 2022
hitenkoku added a commit that referenced this issue Feb 3, 2022
hitenkoku added a commit that referenced this issue Feb 3, 2022
hitenkoku added a commit that referenced this issue Feb 3, 2022
@hitenkoku hitenkoku linked a pull request Feb 3, 2022 that will close this issue
hitenkoku added a commit that referenced this issue Feb 4, 2022
@hitenkoku hitenkoku modified the milestones: v1.2, v1.1 Feb 4, 2022
hitenkoku added a commit that referenced this issue Feb 13, 2022
hitenkoku added a commit that referenced this issue Feb 16, 2022
hitenkoku added a commit that referenced this issue Feb 24, 2022
* changed rules update from clone and pull to submodule update #391

* fixed document

* changed unnecessary clone recursively to clone only

* English message update. ( 4657c35 cherry-pick)

* added create rules folder when rules folder is not exist

* fixed gitmodules github-rules url from ssh to https

Co-authored-by: Tanaka Zakku <[email protected]>
hitenkoku added a commit that referenced this issue Feb 26, 2022
* add git2 crate #391

* added Update option #391

* updated readme #391

* fixed cargo.lock

* fixed option if-statement #391

* changed utc short option and rule-update short option #391

* updated readme

* updated readme

* fixed -u long option & version number update #391

* added fast-forwarding rules repository #391

* updated command line option #391

* moved output logo prev update rule

* fixed readme #391

* removed recursive option in readme

* English message update.

* cargo fmt

* Added update command#391 submodule ver (#401)

* changed rules update from clone and pull to submodule update #391

* fixed document

* changed unnecessary clone recursively to clone only

* English message update. ( 4657c35 cherry-pick)

* added create rules folder when rules folder is not exist

* fixed gitmodules github-rules url from ssh to https

Co-authored-by: Tanaka Zakku <[email protected]>

* added caution case of update failed in readme #391

* fixed document

* added output error in case of loaded rule count is 0  #391 #392

 #392 (comment)

* --update-rules typo

* removed unused library call

Co-authored-by: Tanaka Zakku <[email protected]>
@hitenkoku hitenkoku added the waiting merge main branch already resolved develop branch. label Mar 2, 2022
@hitenkoku
Copy link
Collaborator

close by #425 merged

@hitenkoku hitenkoku removed the waiting merge main branch already resolved develop branch. label Mar 3, 2022
@hitenkoku hitenkoku added this to the v1.1 milestone Mar 7, 2022
hitenkoku added a commit that referenced this issue Mar 29, 2022
* add git2 crate #391

* added Update option #391

* updated readme #391

* fixed cargo.lock

* fixed option if-statement #391

* changed utc short option and rule-update short option #391

* updated readme

* updated readme

* fixed -u long option & version number update #391

* added fast-forwarding rules repository #391

* updated command line option #391

* moved output logo prev update rule

* fixed readme #391

* removed recursive option in readme

* changed rules update from clone and pull to submodule update #391

* fixed document

* changed unnecessary clone recursively to clone only

* English message update.

* cargo fmt

* English message update. ( 4657c35 cherry-pick)

* added create rules folder when rules folder is not exist

* fixed gitmodules github-rules url from ssh to https

* added output of updated file #420

* fixed error #410

* changed update rule list seq

* added test

* fixed output #410

* fixed output and fixed output date field  when  modified field is lacked #410

* fixed compile error

* fixed output

- added enter after Latest rule update output
- added output when no exist new rule
- fixed Latest rule update date format
- changed output from 'Latest rule update' to 'Latest rules update'

* fixed compile error

* changed modified date source from rules folder to each yml rule file

* formatting use chrono in main.rs

* merge develop clippy ci

* fixed output when no update rule #410

- removed Latest rule update

- no output "Rules update successfully" when No rule changed

* Change English

Co-authored-by: Tanaka Zakku <[email protected]>
hitenkoku added a commit that referenced this issue Apr 15, 2022
* Fix/fix clippy warn (#434)

- Fixed following Clippy Warnings(previous warning count: 671 -> after: 4)
  - clippy::needless_return
  - clippy::println_empty_string
  - clippy::redundant_field_names
  - clippy::single_char_pattern
  - clippy::len_zero
  - clippy::iter_nth_zero
  - clippy::bool_comparison
  - clippy::question_mark
  - clippy::needless_collect
  - clippy::unnecessary_unwrap
  - clippy::ptr_arg
  - clippy::needless_collect
  - clippy::needless_borrow
  - clippy::new_without_default
  - clippy::assign_op_pattern
  - clippy::bool_assert_comparison
  - clippy::into_iter_on_ref
  - clippy::deref_addrof
  - clippy::while_let_on_iterator
  - clippy::match_like_matches_macro
  - clippy::or_fun_call
  - clippy::useless_conversion
  - clippy::let_and_return
  - clippy::redundant_clone
  - clippy::redundant_closure
  - clippy::cmp_owned
  - clippy::upper_case_acronyms
  - clippy::map_identity
  - clippy::unused_io_amount
  - clippy::assertions_on_constants
  - clippy::op_ref
  - clippy::useless_vec
  - clippy::vec_init_then_push
  - clippy::useless_format
  - clippy::bind_instead_of_map
  - clippy::bool_comparison
  - clippy::clone_on_copy
  - clippy::too_many_arguments
  - clippy::module_inception
  - fixed clippy::needless_lifetimes
  - fixed clippy::borrowed_box (Thanks for helping by hach1yon!)

* Merge main and output fix#443#444 (#445)

* removed tools/sigmac (#441)

* removed tools/sigmac

- moved tools/sigmac to hayabusa-rules repo

* fixed doc link tools/sigmac

* fixed submodule track

* fixed submodule track from latest to v1.1.0 tag

* fixed link

* erased enter #444

* erased enter #444

* reverted logo enter

* fixed rules submodule target commit #444

Co-authored-by: Yamato Security <[email protected]>

* readme update screenshots etc (#448)

* Opensslを静的にコンパイルするためにCargo.tomlの設定変更 (#437)

* cargo update - openssl static

* updated cargo

* macos2apple

* cargo update

* cargo update

* aliasキーがない場合もEvent.EventDataを自動で走査する (#442)

* add no event key

* support not-register-alias search

* added checking EventData when key do not match in alias #290

- added checking key in Event.EventData, if key is not exist in eventkey_alias.txt.

* cargo fmt

* fixed panic when filter files does not exists

* fixed errorlog format when filter config files does not exist

Co-authored-by: DustInDark <[email protected]>

* changed downcast library from mopa to downcast_rs #447 (#450)

* Fixed Clippy Warnings (#451)

* fixed clippy warn

* fixed cargo clippy warnging

* fixed clippy warngings in clippy ver 0.1.59

* fixed clippy warnings clippy::unnecessary_to_owned

* added temporary blackhat arsenal badge

* added rust report card badges #453

* added repository maintenance levels badge #453

* documentation update macOS usage etc

* update

* added clippy workflow #428 (#429)

* added clippy workflow #428

* fixed action yaml to run clippy #428

* fixed indent

* fixed workflow

* fixed workflow error

* fixed indent

* changed no annotation #428

* adujusted annotation version

* fixed clippy::needless_match

* remove if let exception

* removed unnecessary permission check #428

* statistics event id update (#457)

* Feature/#440 refactoring #395 (#464)

* updated submodule

* fix degrade for pull req #464 (#468)

* fix degrade for pull req #464

* add trim

* Fearture/ added output update result#410 (#452)

* add git2 crate #391

* added Update option #391

* updated readme #391

* fixed cargo.lock

* fixed option if-statement #391

* changed utc short option and rule-update short option #391

* updated readme

* updated readme

* fixed -u long option & version number update #391

* added fast-forwarding rules repository #391

* updated command line option #391

* moved output logo prev update rule

* fixed readme #391

* removed recursive option in readme

* changed rules update from clone and pull to submodule update #391

* fixed document

* changed unnecessary clone recursively to clone only

* English message update.

* cargo fmt

* English message update. ( 4657c35 cherry-pick)

* added create rules folder when rules folder is not exist

* fixed gitmodules github-rules url from ssh to https

* added output of updated file #420

* fixed error #410

* changed update rule list seq

* added test

* fixed output #410

* fixed output and fixed output date field  when  modified field is lacked #410

* fixed compile error

* fixed output

- added enter after Latest rule update output
- added output when no exist new rule
- fixed Latest rule update date format
- changed output from 'Latest rule update' to 'Latest rules update'

* fixed compile error

* changed modified date source from rules folder to each yml rule file

* formatting use chrono in main.rs

* merge develop clippy ci

* fixed output when no update rule #410

- removed Latest rule update

- no output "Rules update successfully" when No rule changed

* Change English

Co-authored-by: Tanaka Zakku <[email protected]>

* Remove unnecessary code from timeline_event_info and rename files for… (#470)

* Remove unnecessary code from timeline_event_info and rename files for issue462

* Remove unnecessary code #462

* add equalsfield pipe (#467)

* Enhancement: add config config #456 (#471)

* added config option #456

* added process of option to speicifed config folder #456

following files adjust config option.

* noisy_rules.txt

* exclude_rules.txt

* fixed usage in readme

* updated rules submodule:

* fixed process when yml file exist in .git folder

* ignore when yml file exist in .git folder

* Add: --level-tuning option's outline

* Add: read Rule files

* Add: input rule_level.txt files & read rules

* cargo fmt

* Add: level-tuning function

* Reface: split to options file

* WIP: Text overwrite failed...

* Fix: Text overwrite was failed

* Add: Error handlings

* Add: id, level validation

* mv: IDS_REGEX to configs file

* fix: level tuning's file name

* Cargo fmt

* Pivot Keyword List機能の追加 (#412)

* add get_pivot_keyword() func

* change function name and call it's function

* [WIP] support config file

* compilete output

* cargo fmt

* [WIP] add test

* add test

* support -o option in pivot

* add pivot mod

* fix miss

* pass test in pivot.rs

* add comment

* pass all test

* add fast return

* fix output

* add test config file

* review

* rebase

* cargo fmt

* test pass

* fix clippy in my commit

* cargo fmt

* little refactor

* change file input logic and config format

* [WIP] change output

* [wip] change deta structure

* change output & change data structure

* pass test

* add config

* cargo fmt & clippy & rebase

* fix cllipy

* delete /rules/ in .gitignore

* clean comment

* clean

* clean

* fix rebase miss

* fix rebase miss

* fix clippy

* file name output on -o to stdout

* add pivot_keywords.txt to ./config

* updated english

* Documentation update

* cargo fmt and clean

* updated translate japanese

* readme update

* readme update

Co-authored-by: DustInDark <[email protected]>
Co-authored-by: Tanaka Zakku <[email protected]>

* Add: test

* Add: README.md

* Cargo fmt

* Use
#[cfg(test)]

* Fixed output stop when  control char exist in windows terminal (#485)

* added control character filter in details #382

* fixed document

- removed fixed windows teminal caution in readme

* fixed level tuning test and added test files #390

* changed level_tuning.txt header from next_level to new_level

* fixed convert miss change to low level

* added run args rules path to check test easy #390

* fixed comment out processing in level_tuning.txt

* fixed config to show level-tuning option

* fixed level-tuning option usage from required to option

* reduce output mitre attack detail tachnique No. by config file (#483)

* reduced mitre attck tag output by config file #477

* prepared 1.2.0 version toml

* added test files and mitre attck strategy tag file #477

* fixed cargo.toml version

* updated cargo.lock

* output tag english update

* cargo fmt

Co-authored-by: Tanaka Zakku <[email protected]>

* Fix: test file's path was incorrect

* Add: add test_files/config/level_tuning.txt

* Add: Flush method.

* inserted debug data

* reverted config usage

* fixed test yaml file path

* Feature/#216 output allfields csvnewcolumn (#469)

* refactoring

* refactoring

* under constructing

* underconstructing

* under construction

* underconstructing

* fix existing testcase

* finish implement

* fmt

* add option

* change name

* fix control code bug

* fix disp

* change format and fix testcase

* fix help

* Fix: show usage when hayabusa has no args

* rm: debug line

* Enhance/warning architecture#478 (#482)

* added  enhance of architecture check #478

* changed check architecture process after output logo #478

* English msg update

* fixed detect method of os-bit to windows and linux

* removed mac and unix architecture and binary and updated its process of windows

* fix clippy

* added check on Wow64 env #478

* Update contributors.txt

Co-authored-by: Tanaka Zakku <[email protected]>

* added --level-tuning option to usage

* Revert "added --level-tuning option to usage"

This reverts commit e6a7409.

* readme update

* Update README-Japanese.md

* readme, version, cargo update

* typo fix

* typo fix

* rm: duplicated test & fix test name

* Add: show logo, and some infos

* small english fix

* twitter link fix (#486)

* added feature of tag output reducing to agg condition #477 (#488)

* changed level output from informational to info #491

* updated rules submodule

* v1.2 changelog update (#473)

* changelog update

* Update CHANGELOG.md

added contributor in "Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData."

ref #442

* Update CHANGELOG-Japanese.md

Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData.

added contributor in "Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData."

ref #442

* Update CHANGELOG.md

added bug fixes (#444) and `Performance and. accuracy`  add contributor ref(#395)

* Update CHANGELOG-Japanese.md

* Translated v1.2 change log to Japanese

v1.2の内容を日本語に修正

* fixed typo

added lacked back quote.

* added description

added following issue and pr description to readme

- #216 / #469 L8
- #390 / #459 L9
- #478 / #482 L19
- #477/ #483 L20

* added description README.md

added following issue and pr description to readme

- #216 / #469 L8
- #390 / #459 L9
- #478 / #482 L19
- #477/ #483 L20

* changelog update

* changelog update

* update

Co-authored-by: DustInDark <[email protected]>

* updated rules #493 (#494)

* Resolve conflict develop (#496)

* removed tools/sigmac (#441)

* removed tools/sigmac

- moved tools/sigmac to hayabusa-rules repo

* fixed doc link tools/sigmac

* fixed submodule track

* fixed submodule track from latest to v1.1.0 tag

* fixed link

* fixed rules submodule targe #444

* updated submodule

* updated rules submodule

Co-authored-by: Yamato Security <[email protected]>

Co-authored-by: Yamato Security <[email protected]>
Co-authored-by: kazuminn <[email protected]>
Co-authored-by: James / hach1yon <[email protected]>
Co-authored-by: garigariganzy <[email protected]>
Co-authored-by: itiB <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Priority:High
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants