Skip to content

Releases: vthib/boreal

v0.9.0

11 Oct 20:08
Compare
Choose a tag to compare

This release brings several memory optimizations and small API improvements.

Memory optimizations comes in two forms:

  • Generic optimizations to reduce the memory footprint of compiled rules, useful in all
    cases when the Scanner object is kept for a long time.
  • The introduction of a new profile that can be set in the compiler, which will compile
    rules to optimize for memory usage rather than scanning speed.

boreal

Breaking changes:

  • A memory pool was introduced to greatly reduce the memory footprint of compiled rules,
    notably when the same meta strings are used in all rules. This introduces two breaking
    changes:

    • The Metadata and MetadataValue objects are no longer re-exported from boreal-parser
      but are new types.
    • To retrieve strings and byte-strings from those objects, the new Scanner::get_bytes_symbol
      and Scanner::get_string_symbol must be used.
  • A new CompilerBuilder object is introduced, to be able to configure a Compiler before
    any rule is added.

  • Added UnwindSafe and RefUnwindSafe trait bounds on module datas:

    • add UnwindSafe traits to module private datas 43502307
    • add UnwindSafe traits for module user datas 56111d77
  • MSRV is bumped from 1.65 to 1.66 825aaab

Added

  • Add CompilerBuilder object to add modules and configure compiler profile: 261b11c2
  • Add compiler profile to pick between memory usage or scanning speed: #167.
  • Add compiler param to disable includes: #170.
  • Update compatibility with YARA 4.5.2: #172.

Changed

  • Add bytes intern pool to reduce memory consumption: #165.
  • Guarantee Scanner is UnwindSafe and RefUnwindSafe: #171.
  • Update windows-sys dependency to version 0.59 ff996f77
  • Update tlsh2 dependency to version 0.4.0 29097dc8

Fixed

  • Fix unused warning on statistics in default features config: #168.

boreal-cli

Added

  • Added option --profile to select memory or speed profile: c3a89c29.

v0.8.0

09 Jun 21:30
Compare
Choose a tag to compare

This release consists of several changes to make the library easier to use in any context
or target:

  • The dependency on OpenSSL (through the authenticode feature) is removed and replaced by pure-Rust dependencies, through the use of two features:

    • The authenticode feature is retained but is now enabled by default. It uses two new dependencies to parse the authenticode signatures.
    • A new authenticode-verify feature is added to handle the pe.is_signed, pe.signatures[*].verified and pe.signatures[*].countersignatures[*].verified fields. See the dedicated documentation for details.
  • The patched version of object has been removed, making the use of the library much easier.

Those changes make boreal depend only on Rust libraries (except for the magic feature), which means the library can be used with any targets and is much easier to integrate.

In addition, this release brings full compatibility with YARA 4.5.1.

⚠ Breaking changes

  • The authenticode feature has been revamped. It is now split into two features:

    • The authenticode feature, which implements all the pe.signatures field except the ones related to signature verification. This feature is now enabled by default.
    • The authenticode-verify feature, which implements the pe.is_signed and *.verified fields. This feature is disabled by default. See the dedicated documentation for details.
  • The Compiler API has been reworked to remove all the ugly workarounds that were needed due to the unsafety brought by the OpenSSL dependency. The Compiler::new_with_pe_signatures and Compiler::new_without_pe_module functions has been removed.

Added

  • add authenticode-verify feature for signature verification 9ced02bf.

Changed

  • Remove hex dependency bb46e49e
  • Remove object patched version #159.
  • Replace authenticode-parser dependency with a custom impl f9521c5c
  • Remove authenticode-parser dependency and clean API 21c5cd74
  • Enable hash dependencies when authenticode feature is enabled b88fedb6

YARA 4.5.1 compatibility:

  • only consider valid ascii bytes for pe dll names c219245e.
  • add some safety checks in pe module for corrupted values 00235005
  • update rva resolution in pe module 66c2d5f4
  • list dotnet resources that are not located in the file b2fa436d

Fixed

  • limit size of version info key and value in pe module 4a20f5c4
  • fix parsing issues in version_info of pe module 8c00218a

v0.7.0

05 May 20:09
Compare
Choose a tag to compare

This release adds the last missing modules from YARA: magic, dex and cuckoo.
It also fixes some bugs related to the use of global rules.

Added:

  • The magic module is now available behind the magic feature (not enabled by default). #139.
  • The dex module is now available behind the object feature (enabled by default). #141.
  • The cuckoo module is now available behind the cuckoo feature (not enabled by default). #143, #144.

Fixed:

  • Fix evaluation bug when global rules were declared after non-global rules. #146.
    If the global rules had any strings, it would make the evaluation of the rules that followed it invalid.
  • Fix application of global rules to namespaces. #147, #149.
    Global rules were applied to all namespaces instead of only their own namespaces.

Changed:

  • The type of boreal::module::StaticValue::Function and of the callback declared in the console module has changed from Arc<Box<...>> to Arc<...>. #142.
  • Error reporting has been improved on IO error on the rules file. #140.

v0.6.0

14 Apr 10:02
Compare
Choose a tag to compare

This release mainly adds the dotnet module and simplifies a few dependencies.

boreal

Added:

  • The dotnet module is now available behind the object feature (enabled by default). #127, #131, #133, #135.

Fixed:

  • Fixed compilation when using --no-default-features and other feature combinations. #129, #130.
  • Fixed exposure of some optional dependencies as their own features. #128.
  • Added CI jobs to ensure common combinations of features compile and run tests properly. #132.

Changed:

  • The bitmap dependency has been removed and replaced by an custom implementation for our very limited usecase. #120.
  • The windows dependency has been replaced by windows-sys. #137.
  • All dependencies have been updated to their latest versions.

Thanks to @demoray for their contributions.

v0.5.0

16 Feb 22:27
Compare
Choose a tag to compare

This release mainly consists of Yara 4.5 compatibility features and fixes:

Added:

YARA 4.5 support:

  • New Warning on unknown escape sequences in regexes. See PR #68.
    This warning is more broad than the YARA one from YARA 4.5.
  • always expose pe.is_signed 97d1d11
  • Do not report strings whose name starts with _ as unused 1a8a8cd
  • Add pe.export_details[*].rva field 7597d3f
  • math.count and math.percentage now returns an undefined value when given a
    value outside the [0; 255] range. 6a09ed2
  • Imported dlls are ignored if the dll name is longer than 255 bytes 28f8626
  • Fix endianness issue in macho.magic field, see the Yara fix 50d418d
  • filter imported functions with invalid name in pe module 5a0cb4e
  • bump limit on number of listed export symbols in pe module to 16384 98032b3

Changed:

  • crc32-fast dependency updated to 1.4 f1ae01a
  • authenticode-parser dependency updated e68dde7

Fixed:

  • Exclude test assets in package 24ca838.
    This avoids having the package be flagged by antiviruses, as unfortunately, some of the binaries copied from the yara repository
    and used for testing seems trigger false positives.

v0.4.0

11 Feb 19:30
Compare
Choose a tag to compare

This release introduces process memory scanning, implemented on Windows, Linux and macOS. In addition, different modes of scanning are available, documenting the exact semantics of scanning a process memory. This allows picking a mode that is less surpresing and faster than the default mode which reproduces YARA's behavior. See FragmentedScanMode for more details, as well as the updated updated benchmarks.

In addition, an API to scan fragmented memory is now available. This is the API which is used during process scanning, and allows custom handling of which memory blocks to scan.

Finally, a few additional features have been added, including an API to mmap files to scan, and the ability to get partial results when the scanning fails, for example due to a timeout.

v0.3.1

12 Nov 09:19
Compare
Choose a tag to compare

Quick release to add a missing feature in boreal: tags and metadatas of matched rules were not available in the scan results.

Detailed changelog:

Boreal:

  • Add rule metadata and tags in results of scans. Only the rule name and namespace was listed, which was an oversight.
    In addition, the Metadata and MetadataValue structs from boreal-parser are re-exported, to avoid having to depend on it to
    inspect matched rules metadatas.
    See PR #85.

v0.3.0

12 Sep 20:20
Compare
Choose a tag to compare

This is a huge release containing several months of work, including:

  • Full compatibility with Yara 4.3. All the new features from Yara 4.3 are available.

  • A complete rewrite of the strings compilation algorithm. Performance has been improved dramatically when using a lot of rules or when using strings of lesser quality. See the updated benchmarks.

  • New tools to debug and improve performances of rules scanning, which new flags to display several kind of statistics.

    • Strings statistics can now be computed: how are strings compiled, the quality of the extracted atoms, ...
    • Evaluation duration statistics can now be computed, detailing how long each evaluation step takes. This is only available if the new profiling feature is enabled, to not impact evaluation performance if not set.
  • Improved testing on modules and on the boreal-cli binary.

Here are some more details on the new YARA features:

Yara 4.3:

  • Negation in hex strings, eg { ~C3 ~?F }.
  • New to_string function in math module.
  • New string module with to_int and length functions.
  • rva field in imported functions in pe module.
  • pe.import_rva and pe.delayed_import_rva functions.
  • pe.rich_signature.version_data field.
  • Iterator on bytes literal, eg for any s in ("foo", "bar"): (...).
  • at for expression, eg any of them at 0.
  • New functions import_md5 and telfhash in elf module.
  • Use of the authenticode-parser lib to parse signatures in pe module. This adds a lot of fields in pe.signatures.

Here are the changes grouped by crate:

Boreal

Added

  • Yara 4.3 compatibility. Too many features to list, see above for a short recap of the main new features.
  • New profiling feature, needed to compute evaluation statistics.

Changed

  • Rewrite of the strings compilation algorithm to significantly improve statistics.
  • openssl feature removed, replaced with the authenticode feature.
  • Using the pe module with the signatures parsing now requires calling the unsafe function Compiler::new_with_pe_signatures.
  • All dependencies updated. regex has been removed in favor of regex-automata.

Fixed

  • Improved handling on invalid ranges in '$a in (from..to)' expression.
  • Fixed minor differences in edge cases in elf.dynamic_section_entries and elf.number_of_sections (e639df643b05).
  • Fixed == operator on boolean values (cec439eee19f).
  • Fixed some bugs occuring when using the fullword keyword with both the wide and ascii modifiers, see PR #51.
  • Fix compilation of rules following the failed compilation of a rule using a rule dependency. I doubt this actually impacted anyone, see PR #60.
  • Change regex behavior to allow non ascii bytes in regexes. See PR #62. A warning has however been added to warn against this situation.
  • Fixed string comparison in the pe.imports and pe.(delayed_)import_rva functions to be case-insensitive, See PR #69.

boreal-cli

Added

  • New -M flag to a list of available modules.
  • New --string-stats flag to display strings' compilation statistics.
  • New --scan-stats flag to display evaluation duration statistics.

Changed

  • Number of dependencies reduced by removing any use of proc macros.
  • boreal updated to 0.3, see boreal changes.

boreal-parser

Added

  • Parsing of negation in hex strings, eg { ~C3 ~?F } (9c21fd446).
  • Parsing of at for expression, eg any of them at 0 (b26fbc3b6).
  • parse_regex and parse_hex_string added to public API (d6a7afc98).

Changed

  • Exports of the crate have been entirely reworked. Objects are now nested in relevant modules (3e8682bec).
  • Removal of bitflags dependency, rework of VariableModifiers object (05877aae4).
  • Regex now accepts non ascii bytes when not in a class. See PR #62.
  • AST for bytes and characters in a regex has been updated to provide escaping information and span location. See PR #68.

Fixed

  • Some public objects were not properly exposed publicly, this should now be fixed (3e8682bec).

Release 0.2.0

12 Feb 20:57
Compare
Choose a tag to compare

Main new feature is hardening of rules parsing and evaluation. This ensures that untrusted rules can be used, and maliciously formed rules or scanned files cannot trigger crashes or bring deteriorated performances.

In addition, a timeout can be specified during the scan to abort scans that last too long.

Lastly, a few differences with YARA found in corner-case situations have been fixed.

For a more detailed list of changes, see https://github.com/vthib/boreal/blob/master/CHANGELOG.md#020---2023-02-12

v0.1.0

04 Dec 11:39
Compare
Choose a tag to compare

Initial release for boreal