Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(server): Use global config from file if provided #2458

Merged
merged 41 commits into from
Sep 12, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
dbc71f3
wip
TBS1996 Sep 3, 2023
a320a33
Merge branch 'master' into tor/static_config
TBS1996 Sep 3, 2023
dc8fe7a
wip
TBS1996 Sep 3, 2023
d631912
wip
TBS1996 Sep 4, 2023
c125ec5
wip
TBS1996 Sep 4, 2023
aed5e6e
wip
TBS1996 Sep 4, 2023
153c9f1
wip
TBS1996 Sep 4, 2023
9367f89
take relay modes into account
TBS1996 Sep 5, 2023
9068be4
merge
TBS1996 Sep 5, 2023
1a43645
Merge branch 'master' into tor/static_config
TBS1996 Sep 5, 2023
3776ec6
wip
TBS1996 Sep 5, 2023
2d56213
Merge branch 'tor/static_config' of https://github.com/getsentry/rela…
TBS1996 Sep 5, 2023
761198e
simplify match arms
TBS1996 Sep 6, 2023
c3221b7
move match to spawnhandler
TBS1996 Sep 6, 2023
8fa4521
ref
TBS1996 Sep 6, 2023
387c246
nit
TBS1996 Sep 6, 2023
0fadcea
self request
TBS1996 Sep 6, 2023
749306a
Merge branch 'master' into tor/static_config
TBS1996 Sep 6, 2023
5281b67
ref match
TBS1996 Sep 6, 2023
68b0bbf
move static config load to service
TBS1996 Sep 6, 2023
383d850
Merge branch 'tor/static_config' of https://github.com/getsentry/rela…
TBS1996 Sep 6, 2023
67d3f73
revert sleephandle logic
TBS1996 Sep 6, 2023
43e4b4c
Merge branch 'master' into tor/static_config
TBS1996 Sep 6, 2023
e9b7272
Merge branch 'tor/static_config' of https://github.com/getsentry/rela…
TBS1996 Sep 6, 2023
5e94b76
move globalconfig load
TBS1996 Sep 7, 2023
9e0f3e3
Merge branch 'master' into tor/static_config
TBS1996 Sep 7, 2023
4f87968
add tests
iker-barriocanal Sep 7, 2023
da63e18
undo rename to prevent bad error grouping
iker-barriocanal Sep 7, 2023
a5e9c03
Merge branch 'master' into tor/static_config
iker-barriocanal Sep 7, 2023
a7030d4
update docstrings
iker-barriocanal Sep 7, 2023
a5c0224
update changelog
TBS1996 Sep 11, 2023
a6efd91
Merge branch 'master' into tor/static_config
TBS1996 Sep 11, 2023
99f2acd
address some feedback
iker-barriocanal Sep 11, 2023
20a593e
wip
TBS1996 Sep 12, 2023
17eba98
Merge branch 'tor/static_config' of https://github.com/getsentry/rela…
TBS1996 Sep 12, 2023
a6ada9f
wip
TBS1996 Sep 12, 2023
a69eb2f
wip
TBS1996 Sep 12, 2023
a4f5a14
t->T
TBS1996 Sep 12, 2023
c0e767d
remove test feat/dep
TBS1996 Sep 12, 2023
c9f0ef9
Merge branch 'master' into tor/static_config
TBS1996 Sep 12, 2023
c39f12c
Merge branch 'master' into tor/static_config
TBS1996 Sep 12, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

**Features**:

- Use static global configuration if file is provided. ([#2453](https://github.com/getsentry/relay/pull/2453))
- Tag keys in error events and transaction events can now be up to `200` ASCII characters long. Before, tag keys were limited to 32 characters. ([#2453](https://github.com/getsentry/relay/pull/2453))

**Bug Fixes**:
Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions relay-config/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ num_cpus = "1.13.0"
relay-auth = { path = "../relay-auth" }
relay-common = { path = "../relay-common" }
relay-kafka = { path = "../relay-kafka" }
relay-dynamic-config = { path = "../relay-dynamic-config" }
relay-log = { path = "../relay-log", features = ["init"] }
relay-metrics = { path = "../relay-metrics" }
relay-redis = { path = "../relay-redis" }
Expand Down
11 changes: 11 additions & 0 deletions relay-config/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,14 @@ use std::io::Write;
use std::net::{IpAddr, SocketAddr, ToSocketAddrs};
use std::path::{Path, PathBuf};
use std::str::FromStr;
use std::sync::Arc;
use std::time::Duration;
use std::{env, fmt, fs, io};

use anyhow::Context;
use relay_auth::{generate_key_pair, generate_relay_id, PublicKey, RelayId, SecretKey};
use relay_common::Dsn;
use relay_dynamic_config::GlobalConfig;
use relay_kafka::{
ConfigError as KafkaConfigError, KafkaConfig, KafkaConfigParam, KafkaTopic, TopicAssignments,
};
Expand Down Expand Up @@ -1250,6 +1252,7 @@ impl ConfigObject for ConfigValues {
pub struct Config {
values: ConfigValues,
credentials: Option<Credentials>,
global_config: Option<Arc<GlobalConfig>>,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wasn't 100% sure if i should put it here or not. My reasoning was that if we fail to parse the file, it's better to fail early than later, and it just seems idiomatic within the codebase that we don't do these kind of risky IO-stuff in the services.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a precedent where we defer the deserialization of a static config file to a service:

/// Get filename for static project config.
pub fn project_configs_path(&self) -> PathBuf {
self.path.join("projects")
}

That service reloads the config every 10 seconds though, which I think is overkill. In short, I think your way of doing it is better because it fails early. I'll leave final review to @jan-auer.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I concur with this: We can load the global configs directly from within the actual service like we do it for project configs. That keeps the responsibility of the relay-config crate lower. The overall implementation in this PR does look good, so it would just have to be moved.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alright, I moved it now to the spawn handler, where I'll log an error if the file exists yet it fails to load it.

path: PathBuf,
}

Expand All @@ -1263,6 +1266,11 @@ impl fmt::Debug for Config {
}

impl Config {
/// globconf
pub fn global_config(&self) -> Option<Arc<GlobalConfig>> {
self.global_config.clone()
}

/// Loads a config from a given config folder.
pub fn from_path<P: AsRef<Path>>(path: P) -> anyhow::Result<Config> {
let path = env::current_dir()
Expand All @@ -1276,6 +1284,7 @@ impl Config {
} else {
None
},
global_config: GlobalConfig::from_file()?.map(Arc::new),
path: path.clone(),
};

Expand All @@ -1294,6 +1303,7 @@ impl Config {
values: serde_json::from_value(value)
.with_context(|| ConfigError::new(ConfigErrorKind::BadJson))?,
credentials: None,
global_config: GlobalConfig::from_file()?.map(Arc::new),
path: PathBuf::new(),
})
}
Expand Down Expand Up @@ -2022,6 +2032,7 @@ impl Default for Config {
values: ConfigValues::default(),
credentials: None,
path: PathBuf::new(),
global_config: None,
}
}
}
Expand Down
19 changes: 19 additions & 0 deletions relay-dynamic-config/src/global.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,22 @@ use serde::{Deserialize, Serialize};
#[derive(Default, Clone, Debug, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct GlobalConfig {}

impl GlobalConfig {
/// load it from a file
pub fn from_file() -> anyhow::Result<Option<GlobalConfig>> {
let path = std::env::current_dir()?
.join(".relay")
.join("global_config.json");

match path.exists() {
true => {
TBS1996 marked this conversation as resolved.
Show resolved Hide resolved
let file_contents = std::fs::read_to_string(path)?;
let global_config = serde_json::from_str::<GlobalConfig>(file_contents.as_str())?;
TBS1996 marked this conversation as resolved.
Show resolved Hide resolved

Ok(Some(global_config))
}
false => Ok(None),
}
}
}
32 changes: 21 additions & 11 deletions relay-server/src/actors/global_config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,7 @@ impl GlobalConfigService {
pub fn new(config: Arc<Config>, upstream: Addr<UpstreamRelay>) -> Self {
let (global_config_watch, _) = watch::channel(Arc::default());
let (internal_tx, internal_rx) = mpsc::channel(1);

Self {
config,
global_config_watch,
Expand Down Expand Up @@ -181,6 +182,7 @@ impl GlobalConfigService {
/// Requests a new global config from upstream.
///
/// We check if we have credentials before sending,
///
/// otherwise we would log an [`UpstreamRequestError::NoCredentials`] error.
fn update_global_config(&mut self) {
self.fetch_handle.reset();
Expand Down Expand Up @@ -243,17 +245,25 @@ impl Service for GlobalConfigService {

fn spawn_handler(mut self, mut rx: relay_system::Receiver<Self::Interface>) {
tokio::spawn(async move {
relay_log::info!("global config service starting");

if self.config.has_credentials() {
iker-barriocanal marked this conversation as resolved.
Show resolved Hide resolved
// NOTE(iker): if this first request fails it's possible the default
// global config is forwarded. This is not ideal, but we accept it
// for now.
self.update_global_config();
} else {
// NOTE(iker): not making a request results in the sleep handler
// not being reset, so no new requests are made.
relay_log::info!("fetching global configs disabled: no credentials configured");
match (self.config.has_credentials(), self.config.global_config()) {
(true, None) => {
// NOTE(iker): if this first request fails it's possible the default
// global config is forwarded. This is not ideal, but we accept it
// for now.
relay_log::info!("global config service starting");
self.update_global_config();
}
(false, None) => {
// NOTE(iker): not making a request results in the sleep handler
// not being reset, so no new requests are made.
relay_log::info!("global config service starting with fetching disabled: no credentials configured");
}
(_, Some(global_config)) => {
relay_log::info!("global config service starting with fetching disabled: using static global config");
self.global_config_watch
.send(global_config.clone())
.unwrap();
}
}

loop {
Expand Down