Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hayabusa crashes with -n option #493

Closed
YamatoSecurity opened this issue Apr 14, 2022 · 7 comments · Fixed by #494
Closed

Hayabusa crashes with -n option #493

YamatoSecurity opened this issue Apr 14, 2022 · 7 comments · Fixed by #494
Assignees

Comments

@YamatoSecurity
Copy link
Collaborator

Describe the bug
Hayabusa crashes when I specify -n

Step to Reproduce
./target/release/hayabusa -d ../hayabusa-sample-evtx -r ../hayabusa-rules -C ../hayabusa-rules/config -o sample-evtx.csv -n

Expected behavior
No crash

Screenshots

./target/release/hayabusa -d ../hayabusa-sample-evtx  -n -q
Analyzing event files: 573
Hayabusa rules: 116
Sigma rules: 1358
Ignored rules: 73
Rule parsing errors: 0
Total enabled detection rules: 1474

33 / 573 [==========>-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 5.76 % 44s thread 'yea-thread' panicked at 'called `Option::unwrap()` on a `None` value', src/detections/rule/matchers.rs:216:33
stack backtrace:
   0:        0x10ce184b6 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h4cae82d438451481
   1:        0x10ccbda2b - core::fmt::write::hb68c3045179d0cad
   2:        0x10ce16e30 - std::io::Write::write_fmt::haf84c797e63d79f0
   3:        0x10ce1bea2 - std::panicking::default_hook::{{closure}}::he18137441e51da1f
   4:        0x10ce1cecc - std::panicking::rust_panic_with_hook::h429a7ddefa5f0258
   5:        0x10ce1c90f - std::panicking::begin_panic_handler::{{closure}}::h9b033a6b15b84a74
   6:        0x10ce1c8a9 - std::sys_common::backtrace::__rust_end_short_backtrace::hcdd3bec8e0e38aa6
   7:        0x10ce1c865 - _rust_begin_unwind
   8:        0x10d020e63 - core::panicking::panic_fmt::hf7d6e5207e013f69
   9:        0x10d020f97 - core::panicking::panic::hdb1992cacba5f03a
  10:        0x10cd05c87 - <hayabusa::detections::rule::matchers::DefaultMatcher as hayabusa::detections::rule::matchers::LeafMatcher>::is_match::hdf8bbff68b4431fd
  11:        0x10cd05d69 - <hayabusa::detections::rule::selectionnodes::AndSelectionNode as hayabusa::detections::rule::selectionnodes::SelectionNode>::select::h4b13d6caa1dc3240
  12:        0x10cd05dc9 - <hayabusa::detections::rule::selectionnodes::OrSelectionNode as hayabusa::detections::rule::selectionnodes::SelectionNode>::select::h7df8ae3a4548decc
  13:        0x10cd062d1 - <hayabusa::detections::rule::selectionnodes::NotSelectionNode as hayabusa::detections::rule::selectionnodes::SelectionNode>::select::hf01e0c4ed4c1f45e
  14:        0x10cd05d69 - <hayabusa::detections::rule::selectionnodes::AndSelectionNode as hayabusa::detections::rule::selectionnodes::SelectionNode>::select::h4b13d6caa1dc3240
  15:        0x10cd05dc9 - <hayabusa::detections::rule::selectionnodes::OrSelectionNode as hayabusa::detections::rule::selectionnodes::SelectionNode>::select::h7df8ae3a4548decc
  16:        0x10cd500d0 - hayabusa::detections::rule::RuleNode::select::h6b42b5eb13638e58
  17:        0x10cd27959 - hayabusa::detections::detection::Detection::execute_rule::hffaf02a3d03c5fe9
  18:        0x10cd4a05e - tokio::runtime::task::harness::poll_future::h27681604da9f6e9e
  19:        0x10cd14ec7 - tokio::runtime::task::raw::poll::h64b3a71ff5b80b90
  20:        0x10ce34b04 - tokio::runtime::thread_pool::worker::Context::run_task::ha92dae581f8a3c7c
  21:        0x10ce39310 - tokio::runtime::task::raw::poll::ha94ef325c6f91a44
  22:        0x10ce287b4 - std::sys_common::backtrace::__rust_begin_short_backtrace::h8273369e087fee37
  23:        0x10ce21b7a - core::ops::function::FnOnce::call_once{{vtable.shim}}::h598f01d5c32c50f0
  24:        0x10ce1e609 - std::sys::unix::thread::Thread::new::thread_start::h9b70539161da4eaf
  25:     0x7ff809f8f4e1 - __pthread_start
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: JoinError::Panic(...)', src/detections/detection.rs:147:41
stack backtrace:
   0:        0x10ce184b6 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h4cae82d438451481
   1:        0x10ccbda2b - core::fmt::write::hb68c3045179d0cad
   2:        0x10ce16e30 - std::io::Write::write_fmt::haf84c797e63d79f0
   3:        0x10ce1bea2 - std::panicking::default_hook::{{closure}}::he18137441e51da1f
   4:        0x10ce1cecc - std::panicking::rust_panic_with_hook::h429a7ddefa5f0258
   5:        0x10ce1c934 - std::panicking::begin_panic_handler::{{closure}}::h9b033a6b15b84a74
   6:        0x10ce1c8a9 - std::sys_common::backtrace::__rust_end_short_backtrace::hcdd3bec8e0e38aa6
   7:        0x10ce1c865 - _rust_begin_unwind
   8:        0x10d020e63 - core::panicking::panic_fmt::hf7d6e5207e013f69
   9:        0x10d021175 - core::result::unwrap_failed::h95d9e30ede493473
  10:        0x10cd2508b - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h9f66300fdbc651ff
  11:        0x10cd255af - hayabusa::detections::detection::Detection::start::h99f0019509df7914
  12:        0x10cc51cab - hayabusa::App::analysis_files::hcf4fc3d1f55392e3
  13:        0x10cc41ca1 - hayabusa::App::exec::h6f96d2c6434794df
  14:        0x10cc3df83 - hayabusa::main::h4919bcdeb6ff6565
  15:        0x10cd737b6 - std::sys_common::backtrace::__rust_begin_short_backtrace::hb0cfaf3909830e29
  16:        0x10cc5e6ad - _main

Environment (please complete the following information):

  • OS: Mac
  • hayabusa version 1.2.0
@YamatoSecurity YamatoSecurity added bug Something isn't working Priority:High labels Apr 14, 2022
@YamatoSecurity YamatoSecurity added this to the v1.2 milestone Apr 14, 2022
@hitenkoku hitenkoku added the under-investigation under investigation to develop label Apr 15, 2022
@hitenkoku
Copy link
Collaborator

以下コマンドでWindows機でも確認できました。 -n をなくしたらエラーが出ていないので、特定のルールで何かしら問題が起こっていそうです。一旦、原因となっているルールがどれかを確認します。

./target/release/hayabusa -d ../hayabusa-sample-evtx -n -q

@hitenkoku
Copy link
Collaborator

-vをつけた際に ./hayabusa-sample-evtx\EVTX-ATTACK-SAMPLES\Command and Control\DE_sysmon-3-rdp-tun.evtx でエラーが出ていることを確認済み

./target/release/hayabusa -d ../hayabusa-sample-evtx -n -q -v
33 / 509 [=========>----------------------------------------------------------------------------------------------------------------------------------------] 6.48 % 2m
Checking target evtx FilePath: "./hayabusa-sample-evtx\EVTX-ATTACK-SAMPLES\Command and Control\DE_sysmon-3-rdp-tun.evtx"

@hitenkoku
Copy link
Collaborator

以下2つのnoisy_ruleを検知ルールとして加えたときにエラーが出ることを確認しました。

0090ea60-f4a2-43a8-8657-3a9a4ddcf547 # Sysmon 6 Driver Loaded - Unknown Signature. It is not possible to list up all good signatures so will be many false positives.
8b8db936-172e-4bb7-9f84-ccc954d51d93 # Sysmon 7 DLL Loaded - No Alerts

@hitenkoku hitenkoku self-assigned this Apr 15, 2022
@YamatoSecurity
Copy link
Collaborator Author

原因特定ありがとうございます!
Sysmon 6 DriverのルールのSignature: の設定されていないところが怪しいので、修正してみます。
しかし、./target/release/hayabusa -d ../hayabusa-sample-evtx -r ../hayabusa-rules/hayabusa/sysmon/events/ -n を実行しても、無事に終わるので、不思議・・

@YamatoSecurity
Copy link
Collaborator Author

@hitenkoku
多分、これで直ります。Yamato-Security/hayabusa-rules#49
ご確認下さい。
ルールミスで失礼しました。

@hitenkoku
Copy link
Collaborator

ご対応ありがとうございます。こちらでも確認して問題なければこのissueはクローズしておきます

@hitenkoku hitenkoku removed this from the v1.2 milestone Apr 15, 2022
@hitenkoku
Copy link
Collaborator

@YamatoSecurity 確認できました。hayabusa側のrulesの紐づけを変更するpull requestを出しますので承認のほどよろしくお願いいたします

@hitenkoku hitenkoku removed bug Something isn't working under-investigation under investigation to develop labels Apr 15, 2022
hitenkoku added a commit that referenced this issue Apr 15, 2022
@hitenkoku hitenkoku linked a pull request Apr 15, 2022 that will close this issue
hitenkoku added a commit that referenced this issue Apr 15, 2022
@hitenkoku hitenkoku added the waiting merge main branch already resolved develop branch. label Apr 15, 2022
hitenkoku added a commit that referenced this issue Apr 15, 2022
* Fix/fix clippy warn (#434)

- Fixed following Clippy Warnings(previous warning count: 671 -> after: 4)
  - clippy::needless_return
  - clippy::println_empty_string
  - clippy::redundant_field_names
  - clippy::single_char_pattern
  - clippy::len_zero
  - clippy::iter_nth_zero
  - clippy::bool_comparison
  - clippy::question_mark
  - clippy::needless_collect
  - clippy::unnecessary_unwrap
  - clippy::ptr_arg
  - clippy::needless_collect
  - clippy::needless_borrow
  - clippy::new_without_default
  - clippy::assign_op_pattern
  - clippy::bool_assert_comparison
  - clippy::into_iter_on_ref
  - clippy::deref_addrof
  - clippy::while_let_on_iterator
  - clippy::match_like_matches_macro
  - clippy::or_fun_call
  - clippy::useless_conversion
  - clippy::let_and_return
  - clippy::redundant_clone
  - clippy::redundant_closure
  - clippy::cmp_owned
  - clippy::upper_case_acronyms
  - clippy::map_identity
  - clippy::unused_io_amount
  - clippy::assertions_on_constants
  - clippy::op_ref
  - clippy::useless_vec
  - clippy::vec_init_then_push
  - clippy::useless_format
  - clippy::bind_instead_of_map
  - clippy::bool_comparison
  - clippy::clone_on_copy
  - clippy::too_many_arguments
  - clippy::module_inception
  - fixed clippy::needless_lifetimes
  - fixed clippy::borrowed_box (Thanks for helping by hach1yon!)

* Merge main and output fix#443#444 (#445)

* removed tools/sigmac (#441)

* removed tools/sigmac

- moved tools/sigmac to hayabusa-rules repo

* fixed doc link tools/sigmac

* fixed submodule track

* fixed submodule track from latest to v1.1.0 tag

* fixed link

* erased enter #444

* erased enter #444

* reverted logo enter

* fixed rules submodule target commit #444

Co-authored-by: Yamato Security <[email protected]>

* readme update screenshots etc (#448)

* Opensslを静的にコンパイルするためにCargo.tomlの設定変更 (#437)

* cargo update - openssl static

* updated cargo

* macos2apple

* cargo update

* cargo update

* aliasキーがない場合もEvent.EventDataを自動で走査する (#442)

* add no event key

* support not-register-alias search

* added checking EventData when key do not match in alias #290

- added checking key in Event.EventData, if key is not exist in eventkey_alias.txt.

* cargo fmt

* fixed panic when filter files does not exists

* fixed errorlog format when filter config files does not exist

Co-authored-by: DustInDark <[email protected]>

* changed downcast library from mopa to downcast_rs #447 (#450)

* Fixed Clippy Warnings (#451)

* fixed clippy warn

* fixed cargo clippy warnging

* fixed clippy warngings in clippy ver 0.1.59

* fixed clippy warnings clippy::unnecessary_to_owned

* added temporary blackhat arsenal badge

* added rust report card badges #453

* added repository maintenance levels badge #453

* documentation update macOS usage etc

* update

* added clippy workflow #428 (#429)

* added clippy workflow #428

* fixed action yaml to run clippy #428

* fixed indent

* fixed workflow

* fixed workflow error

* fixed indent

* changed no annotation #428

* adujusted annotation version

* fixed clippy::needless_match

* remove if let exception

* removed unnecessary permission check #428

* statistics event id update (#457)

* Feature/#440 refactoring #395 (#464)

* updated submodule

* fix degrade for pull req #464 (#468)

* fix degrade for pull req #464

* add trim

* Fearture/ added output update result#410 (#452)

* add git2 crate #391

* added Update option #391

* updated readme #391

* fixed cargo.lock

* fixed option if-statement #391

* changed utc short option and rule-update short option #391

* updated readme

* updated readme

* fixed -u long option & version number update #391

* added fast-forwarding rules repository #391

* updated command line option #391

* moved output logo prev update rule

* fixed readme #391

* removed recursive option in readme

* changed rules update from clone and pull to submodule update #391

* fixed document

* changed unnecessary clone recursively to clone only

* English message update.

* cargo fmt

* English message update. ( 4657c35 cherry-pick)

* added create rules folder when rules folder is not exist

* fixed gitmodules github-rules url from ssh to https

* added output of updated file #420

* fixed error #410

* changed update rule list seq

* added test

* fixed output #410

* fixed output and fixed output date field  when  modified field is lacked #410

* fixed compile error

* fixed output

- added enter after Latest rule update output
- added output when no exist new rule
- fixed Latest rule update date format
- changed output from 'Latest rule update' to 'Latest rules update'

* fixed compile error

* changed modified date source from rules folder to each yml rule file

* formatting use chrono in main.rs

* merge develop clippy ci

* fixed output when no update rule #410

- removed Latest rule update

- no output "Rules update successfully" when No rule changed

* Change English

Co-authored-by: Tanaka Zakku <[email protected]>

* Remove unnecessary code from timeline_event_info and rename files for… (#470)

* Remove unnecessary code from timeline_event_info and rename files for issue462

* Remove unnecessary code #462

* add equalsfield pipe (#467)

* Enhancement: add config config #456 (#471)

* added config option #456

* added process of option to speicifed config folder #456

following files adjust config option.

* noisy_rules.txt

* exclude_rules.txt

* fixed usage in readme

* updated rules submodule:

* fixed process when yml file exist in .git folder

* ignore when yml file exist in .git folder

* Add: --level-tuning option's outline

* Add: read Rule files

* Add: input rule_level.txt files & read rules

* cargo fmt

* Add: level-tuning function

* Reface: split to options file

* WIP: Text overwrite failed...

* Fix: Text overwrite was failed

* Add: Error handlings

* Add: id, level validation

* mv: IDS_REGEX to configs file

* fix: level tuning's file name

* Cargo fmt

* Pivot Keyword List機能の追加 (#412)

* add get_pivot_keyword() func

* change function name and call it's function

* [WIP] support config file

* compilete output

* cargo fmt

* [WIP] add test

* add test

* support -o option in pivot

* add pivot mod

* fix miss

* pass test in pivot.rs

* add comment

* pass all test

* add fast return

* fix output

* add test config file

* review

* rebase

* cargo fmt

* test pass

* fix clippy in my commit

* cargo fmt

* little refactor

* change file input logic and config format

* [WIP] change output

* [wip] change deta structure

* change output & change data structure

* pass test

* add config

* cargo fmt & clippy & rebase

* fix cllipy

* delete /rules/ in .gitignore

* clean comment

* clean

* clean

* fix rebase miss

* fix rebase miss

* fix clippy

* file name output on -o to stdout

* add pivot_keywords.txt to ./config

* updated english

* Documentation update

* cargo fmt and clean

* updated translate japanese

* readme update

* readme update

Co-authored-by: DustInDark <[email protected]>
Co-authored-by: Tanaka Zakku <[email protected]>

* Add: test

* Add: README.md

* Cargo fmt

* Use
#[cfg(test)]

* Fixed output stop when  control char exist in windows terminal (#485)

* added control character filter in details #382

* fixed document

- removed fixed windows teminal caution in readme

* fixed level tuning test and added test files #390

* changed level_tuning.txt header from next_level to new_level

* fixed convert miss change to low level

* added run args rules path to check test easy #390

* fixed comment out processing in level_tuning.txt

* fixed config to show level-tuning option

* fixed level-tuning option usage from required to option

* reduce output mitre attack detail tachnique No. by config file (#483)

* reduced mitre attck tag output by config file #477

* prepared 1.2.0 version toml

* added test files and mitre attck strategy tag file #477

* fixed cargo.toml version

* updated cargo.lock

* output tag english update

* cargo fmt

Co-authored-by: Tanaka Zakku <[email protected]>

* Fix: test file's path was incorrect

* Add: add test_files/config/level_tuning.txt

* Add: Flush method.

* inserted debug data

* reverted config usage

* fixed test yaml file path

* Feature/#216 output allfields csvnewcolumn (#469)

* refactoring

* refactoring

* under constructing

* underconstructing

* under construction

* underconstructing

* fix existing testcase

* finish implement

* fmt

* add option

* change name

* fix control code bug

* fix disp

* change format and fix testcase

* fix help

* Fix: show usage when hayabusa has no args

* rm: debug line

* Enhance/warning architecture#478 (#482)

* added  enhance of architecture check #478

* changed check architecture process after output logo #478

* English msg update

* fixed detect method of os-bit to windows and linux

* removed mac and unix architecture and binary and updated its process of windows

* fix clippy

* added check on Wow64 env #478

* Update contributors.txt

Co-authored-by: Tanaka Zakku <[email protected]>

* added --level-tuning option to usage

* Revert "added --level-tuning option to usage"

This reverts commit e6a7409.

* readme update

* Update README-Japanese.md

* readme, version, cargo update

* typo fix

* typo fix

* rm: duplicated test & fix test name

* Add: show logo, and some infos

* small english fix

* twitter link fix (#486)

* added feature of tag output reducing to agg condition #477 (#488)

* changed level output from informational to info #491

* updated rules submodule

* v1.2 changelog update (#473)

* changelog update

* Update CHANGELOG.md

added contributor in "Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData."

ref #442

* Update CHANGELOG-Japanese.md

Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData.

added contributor in "Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData."

ref #442

* Update CHANGELOG.md

added bug fixes (#444) and `Performance and. accuracy`  add contributor ref(#395)

* Update CHANGELOG-Japanese.md

* Translated v1.2 change log to Japanese

v1.2の内容を日本語に修正

* fixed typo

added lacked back quote.

* added description

added following issue and pr description to readme

- #216 / #469 L8
- #390 / #459 L9
- #478 / #482 L19
- #477/ #483 L20

* added description README.md

added following issue and pr description to readme

- #216 / #469 L8
- #390 / #459 L9
- #478 / #482 L19
- #477/ #483 L20

* changelog update

* changelog update

* update

Co-authored-by: DustInDark <[email protected]>

* updated rules #493 (#494)

* Resolve conflict develop (#496)

* removed tools/sigmac (#441)

* removed tools/sigmac

- moved tools/sigmac to hayabusa-rules repo

* fixed doc link tools/sigmac

* fixed submodule track

* fixed submodule track from latest to v1.1.0 tag

* fixed link

* fixed rules submodule targe #444

* updated submodule

* updated rules submodule

Co-authored-by: Yamato Security <[email protected]>

Co-authored-by: Yamato Security <[email protected]>
Co-authored-by: kazuminn <[email protected]>
Co-authored-by: James / hach1yon <[email protected]>
Co-authored-by: garigariganzy <[email protected]>
Co-authored-by: itiB <[email protected]>
@hitenkoku hitenkoku removed the waiting merge main branch already resolved develop branch. label May 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants