Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Enable .env file loading using env feature #1308

Merged
merged 10 commits into from
Aug 22, 2023
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,4 @@ version-compatibility/Cargo.lock
benches/benches-outputs/Cargo.lock
benches/benches-outputs/src/test_gas_costs_output.rs
.idea
.env
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Description of the upcoming release here.

### Added

- [#1308](https://github.com/FuelLabs/fuel-core/pull/1308): Add support for loading .env files when compiling with the `env` feature. This allows users to conveniently supply CLI arguments in a secure and IDE-agnostic way.
- [#1263](https://github.com/FuelLabs/fuel-core/pull/1263): Add gas benchmarks for `ED19` and `ECR1` instructions.
- [#1286](https://github.com/FuelLabs/fuel-core/pull/1286): Include readable names for test cases where missing.
- [#1304](https://github.com/FuelLabs/fuel-core/pull/1304): Implemented `submit_and_await_commit_with_receipts` method for `FuelClient`.
Expand Down
7 changes: 7 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions bin/fuel-core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ anyhow = { workspace = true }
clap = { workspace = true, features = ["derive", "env"] }
const_format = { version = "0.2", optional = true }
dirs = "4.0"
dotenvy = { version = "0.15", optional = true }
fuel-core = { workspace = true }
humantime = "2.1"
lazy_static = { workspace = true }
Expand All @@ -38,12 +39,13 @@ url = { version = "2.2", optional = true }
test-case = { workspace = true }

[features]
debug = ["fuel-core/debug"]
debug = ["env", "fuel-core/debug"]
default = ["debug", "metrics", "relayer", "rocksdb"]
env = ["dep:dotenvy"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it also should be part of the production and debug feature

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can understand adding it to the debug feature. I can definitely add it there.

I was looking at some of the deployment architecture to see if there would be any conflicts here, because I don't know how this will interact with existing environment variables like the ones set in K8s any configs or helm. It may override them. Is there any risk here?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't use .env, it shouldn't affect the current operator or chart deployment

metrics = ["fuel-core/metrics"]
p2p = ["fuel-core/p2p", "const_format"]
relayer = ["fuel-core/relayer", "dep:url", "dep:serde_json"]
rocksdb = ["fuel-core/rocksdb"]
rocksdb-production = ["fuel-core/rocksdb-production"]
# features to enable in production, but increase build times
production = ["metrics", "relayer", "rocksdb-production", "p2p"]
production = ["env", "metrics", "relayer", "rocksdb-production", "p2p"]
31 changes: 30 additions & 1 deletion bin/fuel-core/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,22 @@ use std::{
path::PathBuf,
str::FromStr,
};
use tracing::warn;
use tracing::{
info,
warn,
};
use tracing_subscriber::{
filter::EnvFilter,
layer::SubscriberExt,
registry,
Layer,
};

#[cfg(feature = "env")]
use dotenvy::dotenv;
#[cfg(feature = "env")]
use tracing::error;

lazy_static::lazy_static! {
pub static ref DEFAULT_DB_PATH: PathBuf = dirs::home_dir().unwrap().join(".fuel").join("db");
}
Expand Down Expand Up @@ -41,6 +49,22 @@ pub enum Fuel {
pub const LOG_FILTER: &str = "RUST_LOG";
pub const HUMAN_LOGGING: &str = "HUMAN_LOGGING";

#[cfg(feature = "env")]
fn init_environment() -> Option<PathBuf> {
match dotenv() {
Ok(path) => Some(path),
Err(e) => {
error!("Unable to load .env environment variables: {e}. Please check that you have created a .env file in your working directory.");
None
}
}
}

#[cfg(not(feature = "env"))]
fn init_environment() -> Option<PathBuf> {
None
}

pub async fn init_logging() -> anyhow::Result<()> {
let filter = match env::var_os(LOG_FILTER) {
Some(_) => {
Expand Down Expand Up @@ -87,6 +111,11 @@ pub async fn init_logging() -> anyhow::Result<()> {
}

pub async fn run_cli() -> anyhow::Result<()> {
init_logging().await?;
if let Some(path) = init_environment() {
let path = path.display();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to add a test to verify that it works properly with the .env file? Maybe even we can have a snapshot of .env file to track all future modificaitons?

Copy link
Contributor Author

@bvrooman bvrooman Aug 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm adding tests to ensure the basic behaviour of the CLI:

  • starts with valid args provided via CLI
  • starts with valid args provided via .env
  • errors with invalid args
  • errors without .env when env is set
  • etc.

I am re-reading your comment here, but I'm unsure if I understood what to use snapshot for here.

There should never be a .env that is shared among developers or stored in CI, since it can store private information. Everyone should have their own local .env.

If the goal of a snapshot is to help developers, one thing we can do is add an example.env that users copy and rename to .env locally.

info!("Loading environment variables from {path}");
}
let opt = Opt::try_parse();
if opt.is_err() {
let command = run::Command::try_parse();
Expand Down
2 changes: 0 additions & 2 deletions bin/fuel-core/src/cli/run.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
#![allow(unused_variables)]
use crate::{
cli::{
init_logging,
run::consensus::PoATriggerArgs,
DEFAULT_DB_PATH,
},
Expand Down Expand Up @@ -333,7 +332,6 @@ impl Command {
}

pub async fn exec(command: Command) -> anyhow::Result<()> {
init_logging().await?;
let config = command.get_config()?;
let network_name = {
#[cfg(feature = "p2p")]
Expand Down
4 changes: 2 additions & 2 deletions bin/fuel-core/src/cli/run/relayer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ pub struct RelayerArgs {
#[arg(long = "relayer", env)]
#[arg(required_if_eq("enable_relayer", "true"))]
#[arg(requires_if(IsPresent, "enable_relayer"))]
pub eth_client: Option<url::Url>,
pub relayer: Option<url::Url>,

/// Ethereum contract address. Create EthAddress into fuel_types
#[arg(long = "relayer-v2-listening-contracts", value_parser = parse_h160, env)]
Expand Down Expand Up @@ -71,7 +71,7 @@ impl RelayerArgs {
let config = Config {
da_deploy_height: DaBlockHeight(self.da_deploy_height),
da_finalization: DaBlockHeight(self.da_finalization),
eth_client: self.eth_client,
relayer: self.relayer,
eth_v2_listening_contracts: self.eth_v2_listening_contracts,
log_page_size: self.log_page_size,
sync_minimum_duration: Duration::from_secs(self.sync_minimum_duration_secs),
Expand Down
2 changes: 0 additions & 2 deletions bin/fuel-core/src/cli/snapshot.rs
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,6 @@ pub async fn exec(command: Command) -> anyhow::Result<()> {

#[cfg(any(feature = "rocksdb", feature = "rocksdb-production"))]
pub async fn exec(command: Command) -> anyhow::Result<()> {
use crate::cli::init_logging;
use anyhow::Context;
use fuel_core::{
chain_config::{
Expand All @@ -60,7 +59,6 @@ pub async fn exec(command: Command) -> anyhow::Result<()> {
},
database::Database,
};
init_logging().await?;
let path = command.database_path;
let data_source =
fuel_core::state::rocks_db::RocksDb::default_open(&path, None).context(
Expand Down
4 changes: 2 additions & 2 deletions crates/services/relayer/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ pub struct Config {
/// Number of da blocks after which messages/stakes/validators become finalized.
pub da_finalization: DaBlockHeight,
/// Uri address to ethereum client.
pub eth_client: Option<url::Url>,
pub relayer: Option<url::Url>,
// TODO: Create `EthAddress` into `fuel_core_types`.
/// Ethereum contract address.
pub eth_v2_listening_contracts: Vec<H160>,
Expand Down Expand Up @@ -58,7 +58,7 @@ impl Default for Config {
Self {
da_deploy_height: DaBlockHeight::from(Self::DEFAULT_DA_DEPLOY_HEIGHT),
da_finalization: DaBlockHeight::from(Self::DEFAULT_DA_FINALIZATION),
eth_client: None,
relayer: None,
eth_v2_listening_contracts: vec![H160::from_str(
"0x03E4538018285e1c03CCce2F92C9538c87606911",
)
Expand Down
2 changes: 1 addition & 1 deletion crates/services/relayer/src/service.rs
Original file line number Diff line number Diff line change
Expand Up @@ -348,7 +348,7 @@ pub fn new_service<D>(database: D, config: Config) -> anyhow::Result<Service<D>>
where
D: RelayerDb + Clone + 'static,
{
let url = config.eth_client.clone().ok_or_else(|| {
let url = config.relayer.clone().ok_or_else(|| {
anyhow::anyhow!(
"Tried to start Relayer without setting an eth_client in the config"
)
Expand Down
4 changes: 2 additions & 2 deletions tests/tests/relayer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ async fn relayer_can_download_logs() {
let eth_node = Arc::new(eth_node);
let eth_node_handle = spawn_eth_node(eth_node).await;

relayer_config.eth_client = Some(
relayer_config.relayer = Some(
format!("http://{}", eth_node_handle.address)
.as_str()
.try_into()
Expand Down Expand Up @@ -153,7 +153,7 @@ async fn messages_are_spendable_after_relayer_is_synced() {
let eth_node = Arc::new(eth_node);
let eth_node_handle = spawn_eth_node(eth_node).await;

relayer_config.eth_client = Some(
relayer_config.relayer = Some(
format!("http://{}", eth_node_handle.address)
.as_str()
.try_into()
Expand Down