Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--updateの結果表示で更新されたファイルの一覧を出力する #410

Closed
hitenkoku opened this issue Feb 23, 2022 · 5 comments · Fixed by #452
Closed

--updateの結果表示で更新されたファイルの一覧を出力する #410

hitenkoku opened this issue Feb 23, 2022 · 5 comments · Fixed by #452
Assignees
Labels
enhancement New feature or request Priority:Low
Milestone

Comments

@hitenkoku
Copy link
Collaborator

あともう一つの(もっと簡単な?)方法として、(詳細は分からないけど)、例えばルールファイルにあるmodified_dateを比較するのはどうですか?
全てのmodified_dateをチェックして、一番最新の日付で以下のように出力する:

Current latest rule: 2020/02/22
Updated rules successfully.
Current latest rule 2022/02/23

等にしたら、ユーザがどのぐらい古いルールを使っていたか、現在の最新のルール等が分かります。
ま、優先度は低で良いと思います。

Originally posted by @YamatoSecurity in #401 (comment)

@hitenkoku hitenkoku changed the title --update で更新されたファイルの一覧を出力する --updateの結果表示で更新されたファイルの一覧を出力する Feb 23, 2022
@hitenkoku hitenkoku self-assigned this Feb 23, 2022
@hitenkoku hitenkoku added the enhancement New feature or request label Feb 23, 2022
@hitenkoku hitenkoku added this to the v1.2 milestone Feb 23, 2022
@hitenkoku
Copy link
Collaborator Author

方針としてはrules配下のファイルの、ファイル名とmodified_dateの組の情報を取得し、
update後に再度取り直して比較することで対応をするようにする。ファイル名としてパイプを使うことはあまりないからHashSetでそのようにつなげてしまったほうが検索としては早いかも。

@YamatoSecurity
Copy link
Collaborator

追加:ルール数も入れましょう。(rules/配下のymlファイルのcount)
例:

Number of Hayabusa rules: 200
Number of Sigma rules: 1000
Current latest rule: 2020/02/22
Updated rules successfully.
Number of Hayabusa rules: 400
Number of Sigma rules: 1500
Current latest rule: 2022/02/23

@hitenkoku
Copy link
Collaborator Author

@YamatoSecurity ymlファイルのカウントはできますが、念のため既存のyamlファイル読み込み処理のところが、ファイルパスを取得出来ているか確認させてください。

また、ルール数を入れるという話ですが、これは更新対象のルール数の話ではなく、対象フォルダ配下に存在するymlファイルのルール数ということでよろしいでしょうか。

表示方法についてですが、どのファイルが更新されたかを明らかにするため表示方法として以下のような形はいかがでしょうか。
Current latest ruleはファイルの最も新しい更新日時を出すというロジックならばできますが、hayabusa-rulesのgitの最新コミット日とはずれる可能性がありますが問題ないでしょうか。

[Rule Updated] {ファイル名}
...
Updated number of xxx rules: 100
Current latest rule: YYYY/MM/DD

@YamatoSecurity
Copy link
Collaborator

お返事が遅くなって、ごめんなさい!
上記のようにrulesフォルダ配下に存在するymlファイルのルール数を出したいです。その差分(例:Updated rules: 100)も入れた方がどれぐらい更新されたか分かりやすいので、それもあると良さそうですね。

更新されたルールの一覧表示については、それもmodifiedフィールドを確認して出力するのはどうですか?
例えば、更新前のmodified日付が2022/02/22で更新後の最新のmodified日付が2022/03/1だったら、modified日付が2/22~3/1の間のルールのtitleフィールドを出力します。(ファイル名より、タイトル名の方が分かりやすいため)

@hitenkoku
Copy link
Collaborator Author

承りました。それでは頂いた仕様で対応できるかは確認してみます。

タイトル名だけを出してもわかりづらいとは思いますので、
タイトル名の後ろにカッコ書きで末尾にファイル名をつけるようにしたいと思います。

hitenkoku added a commit that referenced this issue Mar 16, 2022
@hitenkoku hitenkoku linked a pull request Mar 17, 2022 that will close this issue
hitenkoku added a commit that referenced this issue Mar 19, 2022
hitenkoku added a commit that referenced this issue Mar 25, 2022
- removed Latest rule update

- no output "Rules update successfully" when No rule changed
hitenkoku added a commit that referenced this issue Mar 29, 2022
* add git2 crate #391

* added Update option #391

* updated readme #391

* fixed cargo.lock

* fixed option if-statement #391

* changed utc short option and rule-update short option #391

* updated readme

* updated readme

* fixed -u long option & version number update #391

* added fast-forwarding rules repository #391

* updated command line option #391

* moved output logo prev update rule

* fixed readme #391

* removed recursive option in readme

* changed rules update from clone and pull to submodule update #391

* fixed document

* changed unnecessary clone recursively to clone only

* English message update.

* cargo fmt

* English message update. ( 4657c35 cherry-pick)

* added create rules folder when rules folder is not exist

* fixed gitmodules github-rules url from ssh to https

* added output of updated file #420

* fixed error #410

* changed update rule list seq

* added test

* fixed output #410

* fixed output and fixed output date field  when  modified field is lacked #410

* fixed compile error

* fixed output

- added enter after Latest rule update output
- added output when no exist new rule
- fixed Latest rule update date format
- changed output from 'Latest rule update' to 'Latest rules update'

* fixed compile error

* changed modified date source from rules folder to each yml rule file

* formatting use chrono in main.rs

* merge develop clippy ci

* fixed output when no update rule #410

- removed Latest rule update

- no output "Rules update successfully" when No rule changed

* Change English

Co-authored-by: Tanaka Zakku <[email protected]>
@hitenkoku hitenkoku added the waiting merge main branch already resolved develop branch. label Mar 30, 2022
hitenkoku added a commit that referenced this issue Apr 15, 2022
* Fix/fix clippy warn (#434)

- Fixed following Clippy Warnings(previous warning count: 671 -> after: 4)
  - clippy::needless_return
  - clippy::println_empty_string
  - clippy::redundant_field_names
  - clippy::single_char_pattern
  - clippy::len_zero
  - clippy::iter_nth_zero
  - clippy::bool_comparison
  - clippy::question_mark
  - clippy::needless_collect
  - clippy::unnecessary_unwrap
  - clippy::ptr_arg
  - clippy::needless_collect
  - clippy::needless_borrow
  - clippy::new_without_default
  - clippy::assign_op_pattern
  - clippy::bool_assert_comparison
  - clippy::into_iter_on_ref
  - clippy::deref_addrof
  - clippy::while_let_on_iterator
  - clippy::match_like_matches_macro
  - clippy::or_fun_call
  - clippy::useless_conversion
  - clippy::let_and_return
  - clippy::redundant_clone
  - clippy::redundant_closure
  - clippy::cmp_owned
  - clippy::upper_case_acronyms
  - clippy::map_identity
  - clippy::unused_io_amount
  - clippy::assertions_on_constants
  - clippy::op_ref
  - clippy::useless_vec
  - clippy::vec_init_then_push
  - clippy::useless_format
  - clippy::bind_instead_of_map
  - clippy::bool_comparison
  - clippy::clone_on_copy
  - clippy::too_many_arguments
  - clippy::module_inception
  - fixed clippy::needless_lifetimes
  - fixed clippy::borrowed_box (Thanks for helping by hach1yon!)

* Merge main and output fix#443#444 (#445)

* removed tools/sigmac (#441)

* removed tools/sigmac

- moved tools/sigmac to hayabusa-rules repo

* fixed doc link tools/sigmac

* fixed submodule track

* fixed submodule track from latest to v1.1.0 tag

* fixed link

* erased enter #444

* erased enter #444

* reverted logo enter

* fixed rules submodule target commit #444

Co-authored-by: Yamato Security <[email protected]>

* readme update screenshots etc (#448)

* Opensslを静的にコンパイルするためにCargo.tomlの設定変更 (#437)

* cargo update - openssl static

* updated cargo

* macos2apple

* cargo update

* cargo update

* aliasキーがない場合もEvent.EventDataを自動で走査する (#442)

* add no event key

* support not-register-alias search

* added checking EventData when key do not match in alias #290

- added checking key in Event.EventData, if key is not exist in eventkey_alias.txt.

* cargo fmt

* fixed panic when filter files does not exists

* fixed errorlog format when filter config files does not exist

Co-authored-by: DustInDark <[email protected]>

* changed downcast library from mopa to downcast_rs #447 (#450)

* Fixed Clippy Warnings (#451)

* fixed clippy warn

* fixed cargo clippy warnging

* fixed clippy warngings in clippy ver 0.1.59

* fixed clippy warnings clippy::unnecessary_to_owned

* added temporary blackhat arsenal badge

* added rust report card badges #453

* added repository maintenance levels badge #453

* documentation update macOS usage etc

* update

* added clippy workflow #428 (#429)

* added clippy workflow #428

* fixed action yaml to run clippy #428

* fixed indent

* fixed workflow

* fixed workflow error

* fixed indent

* changed no annotation #428

* adujusted annotation version

* fixed clippy::needless_match

* remove if let exception

* removed unnecessary permission check #428

* statistics event id update (#457)

* Feature/#440 refactoring #395 (#464)

* updated submodule

* fix degrade for pull req #464 (#468)

* fix degrade for pull req #464

* add trim

* Fearture/ added output update result#410 (#452)

* add git2 crate #391

* added Update option #391

* updated readme #391

* fixed cargo.lock

* fixed option if-statement #391

* changed utc short option and rule-update short option #391

* updated readme

* updated readme

* fixed -u long option & version number update #391

* added fast-forwarding rules repository #391

* updated command line option #391

* moved output logo prev update rule

* fixed readme #391

* removed recursive option in readme

* changed rules update from clone and pull to submodule update #391

* fixed document

* changed unnecessary clone recursively to clone only

* English message update.

* cargo fmt

* English message update. ( 4657c35 cherry-pick)

* added create rules folder when rules folder is not exist

* fixed gitmodules github-rules url from ssh to https

* added output of updated file #420

* fixed error #410

* changed update rule list seq

* added test

* fixed output #410

* fixed output and fixed output date field  when  modified field is lacked #410

* fixed compile error

* fixed output

- added enter after Latest rule update output
- added output when no exist new rule
- fixed Latest rule update date format
- changed output from 'Latest rule update' to 'Latest rules update'

* fixed compile error

* changed modified date source from rules folder to each yml rule file

* formatting use chrono in main.rs

* merge develop clippy ci

* fixed output when no update rule #410

- removed Latest rule update

- no output "Rules update successfully" when No rule changed

* Change English

Co-authored-by: Tanaka Zakku <[email protected]>

* Remove unnecessary code from timeline_event_info and rename files for… (#470)

* Remove unnecessary code from timeline_event_info and rename files for issue462

* Remove unnecessary code #462

* add equalsfield pipe (#467)

* Enhancement: add config config #456 (#471)

* added config option #456

* added process of option to speicifed config folder #456

following files adjust config option.

* noisy_rules.txt

* exclude_rules.txt

* fixed usage in readme

* updated rules submodule:

* fixed process when yml file exist in .git folder

* ignore when yml file exist in .git folder

* Add: --level-tuning option's outline

* Add: read Rule files

* Add: input rule_level.txt files & read rules

* cargo fmt

* Add: level-tuning function

* Reface: split to options file

* WIP: Text overwrite failed...

* Fix: Text overwrite was failed

* Add: Error handlings

* Add: id, level validation

* mv: IDS_REGEX to configs file

* fix: level tuning's file name

* Cargo fmt

* Pivot Keyword List機能の追加 (#412)

* add get_pivot_keyword() func

* change function name and call it's function

* [WIP] support config file

* compilete output

* cargo fmt

* [WIP] add test

* add test

* support -o option in pivot

* add pivot mod

* fix miss

* pass test in pivot.rs

* add comment

* pass all test

* add fast return

* fix output

* add test config file

* review

* rebase

* cargo fmt

* test pass

* fix clippy in my commit

* cargo fmt

* little refactor

* change file input logic and config format

* [WIP] change output

* [wip] change deta structure

* change output & change data structure

* pass test

* add config

* cargo fmt & clippy & rebase

* fix cllipy

* delete /rules/ in .gitignore

* clean comment

* clean

* clean

* fix rebase miss

* fix rebase miss

* fix clippy

* file name output on -o to stdout

* add pivot_keywords.txt to ./config

* updated english

* Documentation update

* cargo fmt and clean

* updated translate japanese

* readme update

* readme update

Co-authored-by: DustInDark <[email protected]>
Co-authored-by: Tanaka Zakku <[email protected]>

* Add: test

* Add: README.md

* Cargo fmt

* Use
#[cfg(test)]

* Fixed output stop when  control char exist in windows terminal (#485)

* added control character filter in details #382

* fixed document

- removed fixed windows teminal caution in readme

* fixed level tuning test and added test files #390

* changed level_tuning.txt header from next_level to new_level

* fixed convert miss change to low level

* added run args rules path to check test easy #390

* fixed comment out processing in level_tuning.txt

* fixed config to show level-tuning option

* fixed level-tuning option usage from required to option

* reduce output mitre attack detail tachnique No. by config file (#483)

* reduced mitre attck tag output by config file #477

* prepared 1.2.0 version toml

* added test files and mitre attck strategy tag file #477

* fixed cargo.toml version

* updated cargo.lock

* output tag english update

* cargo fmt

Co-authored-by: Tanaka Zakku <[email protected]>

* Fix: test file's path was incorrect

* Add: add test_files/config/level_tuning.txt

* Add: Flush method.

* inserted debug data

* reverted config usage

* fixed test yaml file path

* Feature/#216 output allfields csvnewcolumn (#469)

* refactoring

* refactoring

* under constructing

* underconstructing

* under construction

* underconstructing

* fix existing testcase

* finish implement

* fmt

* add option

* change name

* fix control code bug

* fix disp

* change format and fix testcase

* fix help

* Fix: show usage when hayabusa has no args

* rm: debug line

* Enhance/warning architecture#478 (#482)

* added  enhance of architecture check #478

* changed check architecture process after output logo #478

* English msg update

* fixed detect method of os-bit to windows and linux

* removed mac and unix architecture and binary and updated its process of windows

* fix clippy

* added check on Wow64 env #478

* Update contributors.txt

Co-authored-by: Tanaka Zakku <[email protected]>

* added --level-tuning option to usage

* Revert "added --level-tuning option to usage"

This reverts commit e6a7409.

* readme update

* Update README-Japanese.md

* readme, version, cargo update

* typo fix

* typo fix

* rm: duplicated test & fix test name

* Add: show logo, and some infos

* small english fix

* twitter link fix (#486)

* added feature of tag output reducing to agg condition #477 (#488)

* changed level output from informational to info #491

* updated rules submodule

* v1.2 changelog update (#473)

* changelog update

* Update CHANGELOG.md

added contributor in "Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData."

ref #442

* Update CHANGELOG-Japanese.md

Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData.

added contributor in "Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData."

ref #442

* Update CHANGELOG.md

added bug fixes (#444) and `Performance and. accuracy`  add contributor ref(#395)

* Update CHANGELOG-Japanese.md

* Translated v1.2 change log to Japanese

v1.2の内容を日本語に修正

* fixed typo

added lacked back quote.

* added description

added following issue and pr description to readme

- #216 / #469 L8
- #390 / #459 L9
- #478 / #482 L19
- #477/ #483 L20

* added description README.md

added following issue and pr description to readme

- #216 / #469 L8
- #390 / #459 L9
- #478 / #482 L19
- #477/ #483 L20

* changelog update

* changelog update

* update

Co-authored-by: DustInDark <[email protected]>

* updated rules #493 (#494)

* Resolve conflict develop (#496)

* removed tools/sigmac (#441)

* removed tools/sigmac

- moved tools/sigmac to hayabusa-rules repo

* fixed doc link tools/sigmac

* fixed submodule track

* fixed submodule track from latest to v1.1.0 tag

* fixed link

* fixed rules submodule targe #444

* updated submodule

* updated rules submodule

Co-authored-by: Yamato Security <[email protected]>

Co-authored-by: Yamato Security <[email protected]>
Co-authored-by: kazuminn <[email protected]>
Co-authored-by: James / hach1yon <[email protected]>
Co-authored-by: garigariganzy <[email protected]>
Co-authored-by: itiB <[email protected]>
@hitenkoku hitenkoku removed the waiting merge main branch already resolved develop branch. label May 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Priority:Low
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants