Releases: vthib/boreal
v0.9.0
This release brings several memory optimizations and small API improvements.
Memory optimizations comes in two forms:
- Generic optimizations to reduce the memory footprint of compiled rules, useful in all
cases when theScanner
object is kept for a long time. - The introduction of a new profile that can be set in the compiler, which will compile
rules to optimize for memory usage rather than scanning speed.
boreal
Breaking changes:
-
A memory pool was introduced to greatly reduce the memory footprint of compiled rules,
notably when the same meta strings are used in all rules. This introduces two breaking
changes:- The
Metadata
andMetadataValue
objects are no longer re-exported fromboreal-parser
but are new types. - To retrieve strings and byte-strings from those objects, the new
Scanner::get_bytes_symbol
andScanner::get_string_symbol
must be used.
- The
-
A new
CompilerBuilder
object is introduced, to be able to configure aCompiler
before
any rule is added. -
Added
UnwindSafe
andRefUnwindSafe
trait bounds on module datas: -
MSRV is bumped from 1.65 to 1.66 825aaab
Added
- Add CompilerBuilder object to add modules and configure compiler profile: 261b11c2
- Add compiler profile to pick between memory usage or scanning speed: #167.
- Add compiler param to disable includes: #170.
- Update compatibility with YARA 4.5.2: #172.
Changed
- Add bytes intern pool to reduce memory consumption: #165.
- Guarantee
Scanner
isUnwindSafe
andRefUnwindSafe
: #171.
- Update memory benchmarks 68a1e046
- Update windows-sys dependency to version 0.59 ff996f77
- Update tlsh2 dependency to version 0.4.0 29097dc8
Fixed
- Fix unused warning on statistics in default features config: #168.
boreal-cli
Added
- Added option
--profile
to select memory or speed profile: c3a89c29.
v0.8.0
This release consists of several changes to make the library easier to use in any context
or target:
-
The dependency on OpenSSL (through the
authenticode
feature) is removed and replaced by pure-Rust dependencies, through the use of two features:- The
authenticode
feature is retained but is now enabled by default. It uses two new dependencies to parse the authenticode signatures. - A new
authenticode-verify
feature is added to handle thepe.is_signed
,pe.signatures[*].verified
andpe.signatures[*].countersignatures[*].verified
fields. See the dedicated documentation for details.
- The
-
The patched version of
object
has been removed, making the use of the library much easier.
Those changes make boreal
depend only on Rust libraries (except for the magic
feature), which means the library can be used with any targets and is much easier to integrate.
In addition, this release brings full compatibility with YARA 4.5.1.
⚠ Breaking changes
-
The
authenticode
feature has been revamped. It is now split into two features:- The
authenticode
feature, which implements all thepe.signatures
field except the ones related to signature verification. This feature is now enabled by default. - The
authenticode-verify
feature, which implements thepe.is_signed
and*.verified
fields. This feature is disabled by default. See the dedicated documentation for details.
- The
-
The
Compiler
API has been reworked to remove all the ugly workarounds that were needed due to the unsafety brought by the OpenSSL dependency. TheCompiler::new_with_pe_signatures
andCompiler::new_without_pe_module
functions has been removed.
Added
- add authenticode-verify feature for signature verification 9ced02bf.
Changed
- Remove
hex
dependency bb46e49e - Remove
object
patched version #159. - Replace authenticode-parser dependency with a custom impl f9521c5c
- Remove authenticode-parser dependency and clean API 21c5cd74
- Enable hash dependencies when authenticode feature is enabled b88fedb6
YARA 4.5.1 compatibility:
- only consider valid ascii bytes for pe dll names c219245e.
- add some safety checks in pe module for corrupted values 00235005
- update rva resolution in pe module 66c2d5f4
- list dotnet resources that are not located in the file b2fa436d
Fixed
v0.7.0
This release adds the last missing modules from YARA: magic
, dex
and cuckoo
.
It also fixes some bugs related to the use of global rules.
Added:
- The
magic
module is now available behind themagic
feature (not enabled by default). #139. - The
dex
module is now available behind theobject
feature (enabled by default). #141. - The
cuckoo
module is now available behind thecuckoo
feature (not enabled by default). #143, #144.
Fixed:
- Fix evaluation bug when global rules were declared after non-global rules. #146.
If the global rules had any strings, it would make the evaluation of the rules that followed it invalid. - Fix application of global rules to namespaces. #147, #149.
Global rules were applied to all namespaces instead of only their own namespaces.
Changed:
v0.6.0
This release mainly adds the dotnet
module and simplifies a few dependencies.
boreal
Added:
- The
dotnet
module is now available behind theobject
feature (enabled by default). #127, #131, #133, #135.
Fixed:
- Fixed compilation when using
--no-default-features
and other feature combinations. #129, #130. - Fixed exposure of some optional dependencies as their own features. #128.
- Added CI jobs to ensure common combinations of features compile and run tests properly. #132.
Changed:
- The
bitmap
dependency has been removed and replaced by an custom implementation for our very limited usecase. #120. - The
windows
dependency has been replaced bywindows-sys
. #137. - All dependencies have been updated to their latest versions.
Thanks to @demoray for their contributions.
v0.5.0
This release mainly consists of Yara 4.5 compatibility features and fixes:
Added:
YARA 4.5 support:
- New Warning on unknown escape sequences in regexes. See PR #68.
This warning is more broad than the YARA one from YARA 4.5. - always expose
pe.is_signed
97d1d11 - Do not report strings whose name starts with
_
as unused 1a8a8cd - Add
pe.export_details[*].rva
field 7597d3f math.count
andmath.percentage
now returns an undefined value when given a
value outside the[0; 255]
range. 6a09ed2- Imported dlls are ignored if the dll name is longer than 255 bytes 28f8626
- Fix endianness issue in
macho.magic
field, see the Yara fix 50d418d - filter imported functions with invalid name in pe module 5a0cb4e
- bump limit on number of listed export symbols in pe module to 16384 98032b3
Changed:
Fixed:
- Exclude test assets in package 24ca838.
This avoids having the package be flagged by antiviruses, as unfortunately, some of the binaries copied from the yara repository
and used for testing seems trigger false positives.
v0.4.0
This release introduces process memory scanning, implemented on Windows, Linux and macOS. In addition, different modes of scanning are available, documenting the exact semantics of scanning a process memory. This allows picking a mode that is less surpresing and faster than the default mode which reproduces YARA's behavior. See FragmentedScanMode for more details, as well as the updated updated benchmarks.
In addition, an API to scan fragmented memory is now available. This is the API which is used during process scanning, and allows custom handling of which memory blocks to scan.
Finally, a few additional features have been added, including an API to mmap files to scan, and the ability to get partial results when the scanning fails, for example due to a timeout.
v0.3.1
Quick release to add a missing feature in boreal: tags and metadatas of matched rules were not available in the scan results.
Detailed changelog:
Boreal:
- Add rule metadata and tags in results of scans. Only the rule name and namespace was listed, which was an oversight.
In addition, theMetadata
andMetadataValue
structs fromboreal-parser
are re-exported, to avoid having to depend on it to
inspect matched rules metadatas.
See PR #85.
v0.3.0
This is a huge release containing several months of work, including:
-
Full compatibility with Yara 4.3. All the new features from Yara 4.3 are available.
-
A complete rewrite of the strings compilation algorithm. Performance has been improved dramatically when using a lot of rules or when using strings of lesser quality. See the updated benchmarks.
-
New tools to debug and improve performances of rules scanning, which new flags to display several kind of statistics.
- Strings statistics can now be computed: how are strings compiled, the quality of the extracted atoms, ...
- Evaluation duration statistics can now be computed, detailing how long each evaluation step takes. This is only available if the new
profiling
feature is enabled, to not impact evaluation performance if not set.
-
Improved testing on modules and on the
boreal-cli
binary.
Here are some more details on the new YARA features:
Yara 4.3:
- Negation in hex strings, eg
{ ~C3 ~?F }
. - New
to_string
function inmath
module. - New
string
module withto_int
andlength
functions. rva
field in imported functions inpe
module.pe.import_rva
andpe.delayed_import_rva
functions.pe.rich_signature.version_data
field.- Iterator on bytes literal, eg
for any s in ("foo", "bar"): (...)
. at
for expression, egany of them at 0
.- New functions
import_md5
andtelfhash
inelf
module. - Use of the
authenticode-parser
lib to parse signatures inpe
module. This adds a lot of fields inpe.signatures
.
Here are the changes grouped by crate:
Boreal
Added
- Yara 4.3 compatibility. Too many features to list, see above for a short recap of the main new features.
- New
profiling
feature, needed to compute evaluation statistics.
Changed
- Rewrite of the strings compilation algorithm to significantly improve statistics.
openssl
feature removed, replaced with theauthenticode
feature.- Using the
pe
module with thesignatures
parsing now requires calling the unsafe functionCompiler::new_with_pe_signatures
. - All dependencies updated.
regex
has been removed in favor ofregex-automata
.
Fixed
- Improved handling on invalid ranges in '$a in (from..to)' expression.
- Fixed minor differences in edge cases in
elf.dynamic_section_entries
andelf.number_of_sections
(e639df643b05). - Fixed
==
operator on boolean values (cec439eee19f). - Fixed some bugs occuring when using the
fullword
keyword with both thewide
andascii
modifiers, see PR #51. - Fix compilation of rules following the failed compilation of a rule using a rule dependency. I doubt this actually impacted anyone, see PR #60.
- Change regex behavior to allow non ascii bytes in regexes. See PR #62. A warning has however been added to warn against this situation.
- Fixed string comparison in the
pe.imports
andpe.(delayed_)import_rva
functions to be case-insensitive, See PR #69.
boreal-cli
Added
- New
-M
flag to a list of available modules. - New
--string-stats
flag to display strings' compilation statistics. - New
--scan-stats
flag to display evaluation duration statistics.
Changed
- Number of dependencies reduced by removing any use of proc macros.
boreal
updated to 0.3, seeboreal
changes.
boreal-parser
Added
- Parsing of negation in hex strings, eg
{ ~C3 ~?F }
(9c21fd446). - Parsing of
at
for expression, egany of them at 0
(b26fbc3b6). parse_regex
andparse_hex_string
added to public API (d6a7afc98).
Changed
- Exports of the crate have been entirely reworked. Objects are now nested in relevant modules (3e8682bec).
- Removal of
bitflags
dependency, rework ofVariableModifiers
object (05877aae4). - Regex now accepts non ascii bytes when not in a class. See PR #62.
- AST for bytes and characters in a regex has been updated to provide escaping information and span location. See PR #68.
Fixed
- Some public objects were not properly exposed publicly, this should now be fixed (3e8682bec).
Release 0.2.0
Main new feature is hardening of rules parsing and evaluation. This ensures that untrusted rules can be used, and maliciously formed rules or scanned files cannot trigger crashes or bring deteriorated performances.
In addition, a timeout can be specified during the scan to abort scans that last too long.
Lastly, a few differences with YARA found in corner-case situations have been fixed.
For a more detailed list of changes, see https://github.com/vthib/boreal/blob/master/CHANGELOG.md#020---2023-02-12
v0.1.0
Initial release for boreal