Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/valkey]: Init new component #37005

Open
wants to merge 27 commits into
base: main
Choose a base branch
from

Conversation

rogercoll
Copy link
Contributor

@rogercoll rogercoll commented Jan 2, 2025

Description

Base receiver code for the new Valkey receiver (metadata, client interface, unit and integration tests). Just the valkey.client.connection.count metric will be available after this PR. Based on https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/database-metrics.md#metric-dbclientconnectioncount

Configuration options: Valkey endpoint + TLS options

Link to tracking issue

Fixes #33787

Testing

  • Unit and integration tests using testdata + contrib golden package.

Documentation

@github-actions github-actions bot added the cmd/otelcontribcol otelcontribcol command label Jan 2, 2025
@rogercoll rogercoll force-pushed the init_valkey_receiver branch from b991b59 to a82e488 Compare January 2, 2025 13:52
@rogercoll rogercoll force-pushed the init_valkey_receiver branch from a82e488 to 1644903 Compare January 2, 2025 14:03
@rogercoll rogercoll marked this pull request as ready for review January 2, 2025 14:30
@rogercoll rogercoll requested a review from a team as a code owner January 2, 2025 14:30
@rogercoll rogercoll requested a review from fatsheep9146 January 2, 2025 14:30
@rogercoll
Copy link
Contributor Author

cc @dmitryax

receiver/valkeyreceiver/client.go Outdated Show resolved Hide resolved
receiver/valkeyreceiver/client.go Outdated Show resolved Hide resolved
receiver/valkeyreceiver/receiver.go Outdated Show resolved Hide resolved
receiver/valkeyreceiver/receiver.go Show resolved Hide resolved
receiver/valkeyreceiver/client.go Outdated Show resolved Hide resolved
Copy link
Contributor

@dehaansa dehaansa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grouping keys by section is a minor concern, not important for this initial version, inability to recover if a client is closed is a larger concern that should be addressed.

}

result, err := vs.client.retrieveInfo(ctx)
if err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there errors that should be considered differently - IE errors that show the client is no longer valid and we need to construct a new one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

attrs := make(map[string]string)
lines := strings.Split(data, attrDelimiter)
for _, line := range lines {
if len(line) == 0 || strings.HasPrefix(line, "#") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lines with a # prefix appear to start a named section of key/value pairs. Is there a promise in the API that there will be no name conflicts across categories? If no, we may need to consider grouping the parsed pairs by category, not in one shared map.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since 1.0.0, there do not appear to have been any conflicts in the sections. See the documentation: https://valkey.io/commands/info/

Although I like the idea of grouping the data by section, I would suggest keeping this feature for a future PR if needed. The current exported client metrics are not affected by the tested version (integration test 7.2)." Wdyt?


## Configuration

> :information_source: This receiver is in beta and configuration fields are subject to change.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this sentence is redundant, since the status of this component is in development, no one should assume that there will be no breaking changes to configuration.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, removed in 018a5e4

receiver the duration between runs. This value must be a string readable by
Golang's `ParseDuration` function (example: `1h30m`). Valid time units are
`ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`.
- `tls`:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, thanks 018a5e4

# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: 'enhancement'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be new_component

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed db230b1

Copy link
Contributor

@fatsheep9146 fatsheep9146 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you fix the conflict?
And there is also a failed lint
https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/12650162873/job/35248291762?pr=37005

Running target 'lint' in module 'receiver/valkeyreceiver' as part of group 'receiver-3'
make --no-print-directory -C receiver/valkeyreceiver lint
go: downloading go.opentelemetry.io/otel/metric v1.33.0
go: downloading go.opentelemetry.io/otel/trace v1.33.0
go: downloading github.com/valkey-io/valkey-go v1.0.45
go: downloading go.opentelemetry.io/otel v1.33.0
go: downloading google.golang.org/genproto/googleapis/rpc v0.0.0-202411182[336](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/12650162873/job/35248291762?pr=37005#step:8:337)22-e639e219e697
Check License finished successfully
running /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/.tools/misspell -error
/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/.tools/golangci-lint run --allow-parallel-runners --build-tags integration --timeout=30m --path-prefix valkeyreceiver
level=error msg="Running error: context loading failed: no go files to analyze: running `go mod tidy` may solve the problem"

@rogercoll rogercoll force-pushed the init_valkey_receiver branch from 29e109c to 8309e1c Compare January 9, 2025 16:50
next consumer. The `collection_interval` configuration option tells this
receiver the duration between runs. This value must be a string readable by
Golang's `ParseDuration` function (example: `1h30m`). Valid time units are
`ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nit: I think maybe it's not reasonable to set up a too shot collection interval to prevent too much stress on Valkey.
Therefore, we can set up to minimum duration check in code to validate it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable to me, which low bound would you propose? Maybe 1 second?

Do we have any other scraper that sets a minimum collection interval value?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I'm not sure about this, but 1s looks good to me, even though receiver can fetch metric the at such granularity, but I think in most cases, the user won't store or query the metrics in that granularity. For me I often query the metric in the granularity of one minute. It's enough for me in most cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New component: Valkey receiver
4 participants