support struct array in pretty display

[houqp]

Which issue does this PR close?

support struct array in pretty print

Rationale for this change

support display of nested fields in downstream projects that displays record batches in console.

What changes are included in this PR?

code to format struct arrays in pretty print
print null value as null string to be consistent with postgresql

Are there any user-facing changes?

better pretty print

[alamb]

Is there any way to add some test coverage to this code so can avoid introducing regressions (aka breaking your code in the future)?

[houqp]

Yep, I will add some tests. I was thinking about this for the arrow2 migration as well ;)

[codecov-commenter]

Codecov Report

Merging #579 (da79ba9) into master (68a4b5b) will decrease coverage by 0.05%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master     #579      +/-   ##
==========================================
- Coverage   82.46%   82.41%   -0.06%     
==========================================
  Files         167      167              
  Lines       46205    46232      +27     
==========================================
- Hits        38101    38100       -1     
- Misses       8104     8132      +28     

Impacted Files	Coverage Δ
arrow/src/util/display.rs	24.03% <0.00%> (-8.43%)	⬇️
parquet/src/encodings/encoding.rs	94.85% <0.00%> (-0.20%)	⬇️
arrow/src/ffi.rs	87.07% <0.00%> (ø)
arrow/src/compute/kernels/sort.rs	94.15% <0.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 68a4b5b...da79ba9. Read the comment docs.

[houqp]

@alamb added struct test, also changed null display to be consistent with postgres and mysql.

[alamb]

@houqp

THe struct tests look good 👍 thank you

also changed null display to be consistent with postgres / mysql / sqlite.

I am not sure the string"null" is consistent with postgres. Here is what happens for me with psql:

alamb=# create table bar(x int);
CREATE TABLE
alamb=# insert into bar values (1);
INSERT 0 1
alamb=# insert into bar values (null);
INSERT 0 1
alamb=# select * from bar;
 x 
---
 1
  
(2 rows)

Sqlite also uses an empty string:

(arrow_dev) alamb@MacBook-Pro:~/Software/influxdb_iox2$ sqlite3 
SQLite version 3.32.3 2020-06-18 14:16:19
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> create table bar(x int);
sqlite> insert into bar values (1);
sqlite> insert into bar values (null);
sqlite> select * from bar;
1

sqlite> 

mysql seems to use the string "NULL" rather than "null":

mysql> use foo;
Database changed
mysql> create table bar(x int);
Query OK, 0 rows affected (0.00 sec)

mysql> insert into bar values (1);
Query OK, 1 row affected (0.01 sec)

mysql> insert into bar values (null);
Query OK, 1 row affected (0.01 sec)

mysql> select * from bar;
+------+
| x    |
+------+
|    1 |
| NULL |
+------+
2 rows in set (0.00 sec)

[alamb]

this is cool -- it is almost like JSON

[houqp]

I am not sure the string"null" is consistent with postgres. Here is what happens for me with psql:

Good call, I was using a different postgres client to test the display. I have reverted that commit and updated the code to handle null only within struct display code.

[alamb]

Looks great @houqp -- thanks!

[jorgecarleitao]

it could have been useful to have a struct with a validity tested, to make sure we do not miss that case.

[houqp]

will send a follow up PR

[jorgecarleitao]

should we have a null in the fields' values but not in all other types?

[houqp]

I don't have a strong opinion on this. I went with this way because I feel like it looks more clear than "c11": ,. There is also the other option of skipping the column with null value entirely in the display. Let me know if you feel strongly on one way or the other, happy to send a PR to change the behavior as well.