Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Commit

Permalink
Add notes about creating parquet files in the development documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
joshuataylor committed Jun 23, 2022
1 parent 2f1a223 commit 664e39c
Showing 1 changed file with 26 additions and 0 deletions.
26 changes: 26 additions & 0 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,32 @@ This runs the tests of the crate without features. To run all features, use
cargo test --features full
```

If you get warnings about parquet files not existing, you can generate the test files by using Python:

```bash
# Ubuntu: sudo apt install python3-pip python3-venv
# Mac: brew install python3
# Archlinux: sudo pacman -Syu python

# Create a virtual environment for python, to keep dependencies contained
python3 -m venv venv

# Activate the virtual environment
source venv/bin/activate

# Make sure pip is up to date
pip install pip --upgrade

# Install pyarrow, version 6
pip install pyarrow==6

# Generate the parquet files (this might take some time, depending on your computer setup)
python parquet_integration/write_parquet.py

# Get out of venv, back to normal terminal
deactivate
```

during development of particular parts of the crate, it is usually faster
to reduce the feature set - the tests are gated to only the relevant tests
of that feature set. For example, if improving JSON, you can use
Expand Down

0 comments on commit 664e39c

Please sign in to comment.