2D & 3D geometry from CSV to DXF and vice versa

[namotec]

This is a follow-up ticket to #9060 "3D geometries from a CSV file to a DXF no longer works with version 3.8". It clarifies the ogr syntax for converting from a 3D CSV file to a 3D DXF. Many thanks for the help to @rouault and @atlight.

In the attached example (makefile) I have worked through four variants (L.dxf P.dxf LZ.dxf PZ.dxf). It examines how a DXF is generated from a spatial polygon (rectangle) in CSV format (import.csv) using the OGR commands LINESTRING, LINSTRINGZ, POLYGON and POLYGONZ. The sequence for the targets (L.dxf P.dxf LZ.dxf PZ.dxf) in the makefile first tries to generate a DXF from the source "import.csv" and then to generate a CSV from the DXF. Ideally, the output CSV should look exactly like the input CSV. As an example, the targets "make P.shp ZP.shp P.gpkg ZP.gpkg" run without errors in order to convert a CSV into an ESRI shapefile or geopackage and back.
Kind regards

Expected behavior and actual behavior.

As a result, I only get the variant "test_3.8.2_ID_LINESTRING_dxf.dxf" in 3-D format. Interestingly, the OGR command ogr2ogr -f "DXF" -nlt LINESTRING out_ID_LINESTRING_dxf.dxf import.csv -dialect sqlite -sql "SELECT ID as Layer, geometry from import"; is actually responsible for a 2D output. The three other variants generate 2D geometries; I would expect a 3D geometry for the targets LZ.dxf and PZ.dxf.

Steps to reproduce the problem.

An example can be found in the tar. The Makefile contains the commands I used.

2024.01.14 convert.error - dxf.tar.7z.zip

Operating system

chris@ber24:~$ inxi -SMrb

System: Host: ber24 Kernel: 5.10.0-26-amd64 x86_64 bits: 64 Desktop: Cinnamon 5.6.8 Distro: LMDE 5 Elsie
Machine: Type: Laptop System: Medion product: P7815 v: 1.0 serial:
Mobo: Medion model: P7815 v: 1.0 serial: UEFI: American Megatrends v: 504 date: 09/29/2012
CPU: Info: Quad Core Intel Core i7-3632QM [MT MCP] speed: 2178 MHz min/max: 1200/3200 MHz
Graphics: Device-1: Intel 3rd Gen Core processor Graphics driver: i915 v: kernel
Device-2: NVIDIA GK107M [GeForce GT 640M] driver: nvidia v: 470.199.02
Display: x11 server: X.Org 1.20.11 driver: loaded: modesetting,nouveau unloaded: fbdev,vesa
resolution: 1920x1080~60Hz
OpenGL: renderer: Mesa DRI Intel HD Graphics 4000 (IVB GT2) v: 4.2 Mesa 20.3.5
Network: Device-1: Intel Centrino Wireless-N 2230 driver: iwlwifi
Device-2: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet driver: r8169
Drives: Local Storage: total: 14.26 TiB used: 6.72 TiB (47.1%)
Repos: No active apt repos in: /etc/apt/sources.list
Active apt repos in: /etc/apt/sources.list.d/official-package-repositories.list
1: deb http://packages.linuxmint.com/ elsie main upstream import backport #id:linuxmint_main
2: deb https://deb.debian.org/debian bullseye main contrib non-free
3: deb https://deb.debian.org/debian bullseye-updates main contrib non-free
4: deb http://security.debian.org/ bullseye-security main contrib non-free
5: deb https://deb.debian.org/debian bullseye-backports main contrib non-free
Active apt repos in: /etc/apt/sources.list.d/qgis.sources
1: deb deb-src [arch=amd64] https://qgis.org/debian bullseye main
Active apt repos in: /etc/apt/sources.list.d/winehq-bullseye.sources
1: deb [arch=amd64 i386] https://dl.winehq.org/wine-builds/debian bullseye main
Info: Processes: 323 Uptime: 2d 18h 46m Memory: 7.64 GiB used: 3.47 GiB (45.4%) Shell: Bash inxi: 3.3.01

GDAL version and provenance

GDAL 3.8.2, released 2023/16/12 installed with conda/micromamba {micromamba create -n gdal gdal pcraster -y -c conda-forge}

[atlight]

Just so I'm not chasing my tail when I look into this, could you please clarify:

Interestingly, the OGR command ogr2ogr -f "DXF" -nlt LINESTRING out_ID_LINESTRING_dxf.dxf import.csv -dialect sqlite -sql "SELECT ID as Layer, geometry from import"; is actually responsible for a 2D output. The three other variants generate 2D geometries;

I assume one of these "2D" is meant to be "3D"?

[namotec]

...; the LINESTRING control word is actually responsible for 2D output.

2D-DXF:
make L.dxf LINESTRING should generate 2D DXF, but makes a 3D DXF. Nice, but not correct.
make P.dxf POLYGON should generate 2D-DXF and makes a 2D-DXF. That is correct.

3D-DXF:
make LZ.dxf LINESTRINGZ should generate 3D DXF, but makes a 2D DXF.
make PZ.dxf POLYGONZ should generate 3D DXF, but makes a 2D DXF.

[jratike80]

There seem to be both -dim and -nlt in the commands. One should be enough but you have used them in a consistent way so I guess that it is ok. But when the result is wrong I would test if it is in the same way wrong when using either/or.
When you test the CSV output remember to use the layer creation option -lco geometry=as_wkt.

[rouault]

What I get with 3.8.3 and master is:

• make L.dxf LINESTRING Z (44163.56 553352.05 247.727688,44163.57 553351.89 247.74,44163.88 553351.89 246.63,44163.86 553352.05 246.617688,44163.56 553352.05 247.727688)
• make P.dxf : POLYGON ((44163.56 553352.05,44163.57 553351.89,44163.88 553351.89,44163.86 553352.05,44163.56 553352.05))
• make LZ.dxf : POLYGON Z ((44163.56 553352.05 247.178844,44163.57 553351.89 247.178844,44163.88 553351.89 247.178844,44163.86 553352.05 247.178844,44163.56 553352.05 247.178844))
• make PZ.dxf : POLYGON Z ((44163.56 553352.05 247.178844,44163.57 553351.89 247.178844,44163.88 553351.89 247.178844,44163.86 553352.05 247.178844,44163.56 553352.05 247.178844))

so the potential debatable cases are -nlt LINESTRING and -nlt LINESTRINGZ that don't lead to expected geometry types

[rouault]

with #9083, I now get:

$ ogr2ogr /vsistdout/  "2024.01.14 convert.error - dxf/status.01 - open/import.csv" -dialect sqlite -sql "SELECT ID as Layer, geometry from import" -nlt linestring  -f CSV -lco geometry=as_wkt
GEOMETRY,Layer
"LINESTRING (44163.56 553352.05,44163.57 553351.89,44163.88 553351.89,44163.86 553352.05,44163.56 553352.05)","2005"

$ ogr2ogr /vsistdout/  "2024.01.14 convert.error - dxf/status.01 - open/import.csv" -dialect sqlite -sql "SELECT ID as Layer, geometry from import" -nlt linestringz  -f CSV -lco geometry=as_wkt
GEOMETRY,Layer
"LINESTRING Z (44163.56 553352.05 247.727688,44163.57 553351.89 247.74,44163.88 553351.89 246.63,44163.86 553352.05 246.617688,44163.56 553352.05 247.727688)","2005"

$ ogr2ogr /vsistdout/  "2024.01.14 convert.error - dxf/status.01 - open/import.csv" -dialect sqlite -sql "SELECT ID as Layer, geometry from import" -nlt polygon -f CSV -lco geometry=as_wkt
GEOMETRY,Layer
"POLYGON ((44163.56 553352.05,44163.57 553351.89,44163.88 553351.89,44163.86 553352.05,44163.56 553352.05))","2005"

$ ogr2ogr /vsistdout/  "2024.01.14 convert.error - dxf/status.01 - open/import.csv" -dialect sqlite -sql "SELECT ID as Layer, geometry from import" -nlt polygonz -f CSV -lco geometry=as_wkt
GEOMETRY,Layer
"POLYGON Z ((44163.56 553352.05 247.727688,44163.57 553351.89 247.74,44163.88 553351.89 246.63,44163.86 553352.05 246.617688,44163.56 553352.05 247.727688))","2005"

[namotec]

Does a Conda environment for gdal version 3.9.0 exist?

[rouault]

Does a Conda environment for gdal version 3.9.0 exist?

once this PR is merged, you'll be able to use https://gdal.org/download.html#gdal-master-conda-builds

[namotec]

Sorry for the delay.

The following development environment for gdal 3.9 works for me:

Automatic install micromamba:

[https://mamba.readthedocs.io/en/latest/installation/micromamba-installation.html?highlight=Automatic%20install#automatic-install]

"${SHELL}" <(curl -L micro.mamba.pm/install.sh); ### UX: install micromamba
source ~/.bashrc; ### (or ~/.zshrc, ~/.xonshrc, ~/.config/fish/config.fish, ...)
micromamba --version && micromamba env list;
micromamba update micromamba 	--name base --channel conda-forge  python_abi=*=*cp*'  --override-channels
micromamba create --name gdal_nightly && micromamba env list;
micromamba install gdal --name gdal_nightly --channel gdal-master;
micromamba activate --name gdal_nightly && gdalinfo --version; ### version 3.9 - here it is! 

GDAL 3.9.0dev-468125c1ce-dirty, released 2024/01/21

micromamba deactivate

Now to the actual analysis.

I have changed the geometry data in the input file import.csv in the Z values:

"POLYGON Z ((
44163.56 553352.05 777777.0111,
44163.57 553351.89 777777.02,
44163.88 553351.89 777777.03,
44163.86 553352.05 777777.04,
44163.56 553352.05 777777.0111))","2005"

Start GDAL test environment:

micromamba activate --name gdal_nightly && gdalinfo --version

The stream output is ok:

ogr2ogr /vsistdout/ ./import.csv -dialect sqlite -sql "SELECT ID as Layer, geometry from import" -nlt polygonz -f CSV -lco geometry=as_wkt;

GEOMETRY,Layer
"POLYGON Z ((44163.56 553352.05 777777.0111,44163.57 553351.89 777777.02,44163.88 553351.89 777777.03,44163.86 553352.05 777777.04,44163.56 553352.05 777777.0111))","2005"

Transformation from CSV to DXF and back:

ogr2ogr -f "DXF" ./export_POLYGONZ.dxf ./import.csv  -dialect sqlite -sql "SELECT  ID as Layer, geometry from import" -nlt polygonz;
ogrinfo -al ./export_POLYGONZ.dxf | egrep "Geo|POLYGON Z"

Geometry: Unknown (any)
POLYGON Z ((44163.56 553352.05 777777.02555,44163.57 553351.89 777777.02555,44163.88 553351.89 777777.02555,44163.86 553352.05 777777.02555,44163.56 553352.05 777777.02555))

ogr2ogr -f "CSV" ./export_POLYGONZ.csv ./export_POLYGONZ.dxf -lco geometry=as_wkt -nlt polygonz;
ogrinfo -al ./export_POLYGONZ.csv | egrep "Geo|POLYGON Z"

Geometry: Unknown (any)
WKT (String) = POLYGON Z ((44163.56 553352.05 777777.02555,44163.57 553351.89 777777.02555,44163.88 553351.89 777777.02555,44163.86 553352.05 777777.02555,44163.56 553352.05 777777.02555))
POLYGON Z ((44163.56 553352.05 777777.02555,44163.57 553351.89 777777.02555,44163.88 553351.89 777777.02555,44163.86 553352.05 777777.02555,44163.56 553352.05 777777.02555))

As a result, the Z values are lost in the transition to DXF. Only an averaged height (777777.02555) is calculated and only displayed in two dimensions in the DXF. Z values were expected, as declared in the import file (import.csv). A makefile and log are attached.
2024.01.22 3.9.0 milestone.tar.7z.zip

[rouault]

The following development environment for gdal 3.9 works for me:

note that this does not test the changes of pull request #9083, since it is not merged yet

[namotec]

Thank you. Now I understand.