config: yaml_parser: Handle $log_level element for special case

[cosmo0920]

Which issue(s) this PR fixes:

Closes #4463.

What this PR does / why we need it:

This is because @log_level is invalid for YAML.
Instead, we should interpret $log_level as @log_level on YAML parser.

Docs Changes:

Needed. Will handle on it.

fluent/fluentd-docs-gitbook#497

Release Note:

Same as title.

[daipom]

Thanks! I will check this tomorrow!

[Shingo-Nakayama]

I confirmed that it works fine.

config:
  - source:
      $type: tail
      tag: sample
      path: /tmp/test.log
      pos_file: /tmp/tail-test.pos
      $log_level: debug
      parse: 
        $type: none

  - match:
      $tag: '**'
      $type: stdout

$ bundle exec fluentd -c /Users/config.yaml
2024-05-02 12:19:15 +0900 [info]: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2024-05-02 12:19:15 +0900 [info]: parsing config file is succeeded path="/Users/config.yaml"
2024-05-02 12:19:15 +0900 [info]: gem 'fluentd' version '1.16.5'
2024-05-02 12:19:15 +0900 [warn]: define <match fluent.**> to capture fluentd logs in top level is deprecated. Use <label @FLUENT_LOG> instead
2024-05-02 12:19:15 +0900 [info]: using configuration file: <ROOT>
  <source>
    @type tail
    @log_level "debug"
    tag "sample"
    path "/tmp/test.log"
    pos_file "/tmp/tail-test.pos"
    <parse>
      @type "none"
      unmatched_lines 
    </parse>
  </source>
  <match **>
    @type stdout
  </match>
</ROOT>
2024-05-02 12:19:15 +0900 [info]: starting fluentd-1.16.5 pid=36950 ruby="3.3.0"
2024-05-02 12:19:15 +0900 [info]: spawn command to main:  cmdline=["/opt/homebrew/Cellar/ruby/3.3.0/bin/ruby", "-r/opt/homebrew/Cellar/ruby/3.3.0/lib/ruby/3.3.0/bundler/setup", "-Eascii-8bit:ascii-8bit", "/opt/homebrew/lib/ruby/gems/3.3.0/bin/fluentd", "-c", "/Users/nakayamashingo/Desktop/desktop/fluentd/config.yaml", "--under-supervisor"]
2024-05-02 12:19:15 +0900 [info]: #0 init worker0 logger path=nil rotate_age=nil rotate_size=nil
2024-05-02 12:19:15 +0900 [info]: adding match pattern="**" type="stdout"
2024-05-02 12:19:15 +0900 [info]: adding source type="tail"
2024-05-02 12:19:15 +0900 [warn]: #0 define <match fluent.**> to capture fluentd logs in top level is deprecated. Use <label @FLUENT_LOG> instead
2024-05-02 12:19:15 +0900 [info]: #0 starting fluentd worker pid=36953 ppid=36950 worker=0
2024-05-02 12:19:15 +0900 [debug]: #0 Compacted entries: ["/tmp/test.log"]
2024-05-02 12:19:15 +0900 [debug]: #0 Remove missing entries. existing_targets=["/tmp/test.log"] entries_after_removing=["/tmp/test.log"]
2024-05-02 12:19:15 +0900 [debug]: #0 tailing paths: target = /tmp/test.log | existing = 
2024-05-02 12:19:15 +0900 [info]: #0 following tail of /tmp/test.log
2024-05-02 12:19:15 +0900 [info]: #0 fluentd worker is now running worker=0
2024-05-02 12:19:15.954638000 +0900 fluent.info: {"pid":36953,"ppid":36950,"worker":0,"message":"starting fluentd worker pid=36953 ppid=36950 worker=0"}
2024-05-02 12:19:15.955250000 +0900 fluent.debug: {"message":"Compacted entries: [\"/tmp/test.log\"]"}
2024-05-02 12:19:15.955269000 +0900 fluent.debug: {"existing_targets":["/tmp/test.log"],"entries_after_removing":["/tmp/test.log"],"message":"Remove missing entries. existing_targets=[\"/tmp/test.log\"] entries_after_removing=[\"/tmp/test.log\"]"}
2024-05-02 12:19:15.955695000 +0900 fluent.debug: {"message":"tailing paths: target = /tmp/test.log | existing = "}
2024-05-02 12:19:15.955829000 +0900 fluent.info: {"message":"following tail of /tmp/test.log"}
2024-05-02 12:19:15.955898000 +0900 fluent.info: {"worker":0,"message":"fluentd worker is now running worker=0"}
^C2024-05-02 12:19:18 +0900 [info]: Received graceful stop
2024-05-02 12:19:18 +0900 [info]: #0 fluentd worker is now stopping worker=0
2024-05-02 12:19:18.996849000 +0900 fluent.info: {"worker":0,"message":"fluentd worker is now stopping worker=0"}
2024-05-02 12:19:18 +0900 [info]: #0 shutting down fluentd worker worker=0
2024-05-02 12:19:18 +0900 [info]: #0 shutting down input plugin type=:tail plugin_id="object:cf8"
2024-05-02 12:19:18 +0900 [info]: #0 shutting down output plugin type=:stdout plugin_id="object:cd0"
2024-05-02 12:19:19 +0900 [info]: Worker 0 finished with status 0

[daipom]

@Shingo-Nakayama Thanks!

[daipom]

LGTM! Thanks!

[ashie]

Thanks!

[cosmo0920]

BTW, Which versions will this feature be enabled? 1.17.1 or 1.16.6?

[daipom]

I will backport this to 1.16.6 if there is no objection.

[cosmo0920]

OK. I don't have any objections. Let's backport this to v1.16.6.