Critical bug: ign-fuel download unusable after last update - creates regular files instead of directories

[peci1]

Environment

• OS Version: Ubuntu 18.04
• Source or binary build? binary 5.1.1

Description

• Expected behavior: ign fuel download works
• Actual behavior: It fails with errors like:

[Msg] Downloading model [https://fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials/textures]
[Err] [Filesystem.cc:459] Failed to create directory [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials/textures]: Not a directory
[Err] [Zip.cc:152] Error creating directory [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials/textures]. Do you have the right permissions?
[Err] [LocalCache.cc:415] Unable to unzip [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/ctu_cras_norlab_marv_sensor_config_1.zip]
[Err] [SystemPaths.cc:369] URI [https://fuel.ignitionrobotics.org/1.0/OpenRobotics/models/CTU_CRAS_NORLAB_MARV_SENSOR_CONFIG_1/tip/files/meshes/evetar_lens.dae] resolved to path [/meshes/evetar_lens.dae] but the path does not exist

The permission denied is there because the subdirectories Fuel should create are actually regular files:

$ ll "/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials"
-rw-r--r-- 1 robot robot 0 May  1 01:22 /home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials

Steps to reproduce

$ rm -rf "~/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1"
$ ign fuel download -v4 --url "https://fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1"
Downloading model: 
  Name: ctu_cras_norlab_marv_sensor_config_1
  Owner: openrobotics
  Server:
    URL: https://fuel.ignitionrobotics.org
    Version: 1.0

[Msg] Downloading model [https://fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials/textures]
[Err] [Filesystem.cc:459] Failed to create directory [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials/textures]: Not a directory
[Err] [Zip.cc:152] Error creating directory [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials/textures]. Do you have the right permissions?
[Err] [LocalCache.cc:415] Unable to unzip [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/ctu_cras_norlab_marv_sensor_config_1.zip]
Download failed.

I haven't found any relevant commit in ign-fuel. Might it be related to gazebosim/gz-common@cdfbe87 ?

[peci1]

@nkoenig This is a usability blocker, requires attention ASAP.

[chapulina]

Fix underway in gazebosim/gz-common#209

[peci1]

Workaround - gradually try to download the models, remove the wrong files, make the folders manually, and repeat.

$ rm -rf ~/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials 
$ mkdir -p ~/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials/textures
$ ign fuel download -v4 --url "https://fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1"
Downloading model: 
  Name: ctu_cras_norlab_marv_sensor_config_1
  Owner: openrobotics
  Server:
    URL: https://fuel.ignitionrobotics.org
    Version: 1.0

[Msg] Downloading model [https://fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials/textures]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials/textures/battery.png]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials/textures/chassis.png]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials/textures/chassis_emissive.png]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials/textures/flipper_color.png]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/materials/textures/ouster-os0.png]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/basler_ace.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/basler_ace2_pro.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/battery.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/belt.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/big_wheel.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/chassis.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/deck.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/evetar_fisheye_lens.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/evetar_lens.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/flipper.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/flipper_arm.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/flir_boson.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/item20.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/item_angle40x40.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/mobilicom.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/mote_deployer.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/motor_housing.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/motor_housing_mirrored.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/ouster-os0.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/reflector.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/small_wheel.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/meshes/tfmini_plus_lidar.dae]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/metadata.pbtxt]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/model.config]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/model.sdf]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/thumbnails]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/thumbnails/1.png]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/thumbnails/2.png]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/thumbnails/3.png]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/thumbnails/4.png]
[Dbg] [Zip.cc:175] Created file [/home/robot/.ignition/fuel/fuel.ignitionrobotics.org/openrobotics/models/ctu_cras_norlab_marv_sensor_config_1/3/thumbnails/5.png]
Download succeeded.

[nkoenig]

Fix in gazebosim/gz-common#209

[peci1]

Fix underway in ignitionrobotics/ign-common#209

Thanks!

[ahcorde]

I was reviewing the regression and to avoid this failure/regression on Linux this other PR https://github.com/ignitionrobotics/ign-fuel-tools/pull/178/files should be in too. There are some changes in Zip.cc that make the changes on ign-common compatible with ign-fuel-tools.

FYI @nkoenig @mjcarroll , just in case you want to revert again these changes and allow windows to use ign-fuel-tools.

[mjcarroll]

Addressed by #178