cargo test only run unittests

[xudong963]

I run cargo test --features full locally, but only unit tests are run.

How to run integration tests (arrow2/tests) locally

[houqp]

you can find integration test commands based on CI tasks in .github/workflows, for example: https://github.com/jorgecarleitao/arrow2/blob/main/.github/workflows/integration-ipc.yml

[jorgecarleitao]

Hey @xudong963, arrow2/tests are run with that command:

$ cargo test --features full
   Compiling lexical-write-float v0.8.2
   ...
   Compiling arrow2 v0.6.2 (/home/azureuser/projects/arrow2)
    Finished test [unoptimized + debuginfo] target(s) in 2m 09s
     Running unittests (target/debug/deps/arrow2-b71d33dcbc02a483)

running 85 tests
test bitmap::utils::chunk_iterator::merge::tests::test_merge_reversed ... ok
...
test io::parquet::read::schema::convert::tests::test_parquet_lists ... ok

test result: ok. 85 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s

     Running tests/it/main.rs (target/debug/deps/it-3bbfa81fb882ace7)

running 799 tests
test array::binary::from_iter ... ok
...
test io::ipc::write::file::write_100_decimal ... ok

test result: ok. 797 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out; finished in 1.80s

   Doc-tests arrow2

running 144 tests
test src/array/struct_.rs - array::struct_::StructArray (line 15) ... ok
...
test src/types/bit_chunk.rs - types::bit_chunk::BitChunkIter (line 131) ... ok

test result: ok. 141 passed; 0 failed; 3 ignored; 0 measured; 0 filtered out; finished in 251.64s

To only run tests, use cargo test --tests --features full. To run integration tests of the Arrow specification (IPC, Flight, FFI) and/or parquet (pyarrow and pyspark), we need to run the tests that @houqp mentioned, which is not done solely by cargo.

[xudong963]

Thanks! Got it

I found the test need a data file and didn't find it in the project. Where can I fetch it?

    #[test]
    fn test_basic() -> Result<()> {
        if std::env::var("ARROW2_IGNORE_PARQUET").is_ok() {
            return Ok(());
        }
        let schema = read_schema("fixtures/pyarrow3/v1/basic_nullable_10.parquet")?;
        let names = schema
            .unwrap()
            .fields()
            .iter()
            .map(|x| x.name().clone())
            .collect::<Vec<_>>();
        assert_eq!(
            names,
            vec![
                "int64",
                "float64",
                "string",
                "bool",
                "date",
                "uint32",
                "string_large",
                "decimal_9",
                "decimal_18",
                "decimal_26"
            ]
        );
        Ok(())
    }

[jorgecarleitao]

Awesome. They are created via a Python script, See here for how we do it in the CI (I usually use a venv)

[xudong963]

I think it's better to rich readme about tests. I will do it！

[jorgecarleitao]

One thing I usually do is to run the tests under feature gates. E.g. if my changes only affect csv, I use cargo test --tests --features io_csv