torchvision.transforms.v2.functional.convert_bounding_box_format is wrong

[eugeneteoh]

🐛 Describe the bug

Hi, unless I'm inputting the wrong data format, I found that the output of torchvision.transforms.v2.functional.convert_bounding_box_format is not consistent with torchvision.ops.box_convert. Please see the example below for reproduction:

import torch
from torchvision.transforms.v2.functional import convert_bounding_box_format
from torchvision.ops import box_convert

input = torch.tensor([[328.0770, 231.1015, 279.2261, 457.5734]])
out1 = convert_bounding_box_format(input, "CXCYWH", "XYXY")
out2 = box_convert(input, "cxcywh", "xyxy")

print(torch.allclose(out1, out2))
print((out1 - out2).norm())

Versions

PyTorch version: 2.2.0
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux 11 (bullseye) (x86_64)
GCC version: (Debian 10.2.1-6) 10.2.1 20210110
Clang version: Could not collect
CMake version: version 3.18.4
Libc version: glibc-2.31

Python version: 3.10.13 | packaged by conda-forge | (main, Dec 23 2023, 15:36:39) [GCC 12.3.0] (64-bit runtime)
Python platform: Linux-5.10.0-27-cloud-amd64-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: 12.1.105
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA L4
Nvidia driver version: 535.86.10
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 46 bits physical, 48 bits virtual
CPU(s): 2
On-line CPU(s) list: 0,1
Thread(s) per core: 1
Core(s) per socket: 2
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) CPU @ 2.20GHz
Stepping: 7
CPU MHz: 2200.180
BogoMIPS: 4400.36
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 64 KiB
L1i cache: 64 KiB
L2 cache: 2 MiB
L3 cache: 38.5 MiB
NUMA node0 CPU(s): 0,1
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
Vulnerability Retbleed: Mitigation; Enhanced IBRS
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Mitigation; Clear CPU buffers; SMT Host state unknown
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat avx512_vnni md_clear arch_capabilities

Versions of relevant libraries:
[pip3] numpy==1.26.3
[pip3] pytorch-lightning==2.1.4
[pip3] torch==2.2.0
[pip3] torchaudio==2.2.0
[pip3] torchmetrics==1.3.0.post0
[pip3] torchvision==0.17.0
[pip3] triton==2.2.0
[conda] blas 1.0 mkl conda-forge
[conda] ffmpeg 4.3 hf484d3e_0 pytorch
[conda] libjpeg-turbo 2.0.0 h9bf148f_0 pytorch
[conda] mkl 2023.1.0 h213fc3f_46344
[conda] numpy 1.26.3 py310hb13e2d6_0 conda-forge
[conda] pytorch 2.2.0 py3.10_cuda12.1_cudnn8.9.2_0 pytorch
[conda] pytorch-cuda 12.1 ha16c6d3_5 pytorch
[conda] pytorch-lightning 2.1.4 pypi_0 pypi
[conda] pytorch-mutex 1.0 cuda pytorch
[conda] torchaudio 2.2.0 py310_cu121 pytorch
[conda] torchmetrics 1.3.0.post0 pypi_0 pypi
[conda] torchtriton 2.2.0 py310 pytorch
[conda] torchvision 0.17.0 py310_cu121 pytorch

[mantasu]

Hey, convert_bounding_box_format does not accept string literals. Only enum values work:

import torch
from torchvision.ops import box_convert
from torchvision.transforms.v2.functional import convert_bounding_box_format
from torchvision.tv_tensors import BoundingBoxFormat as BBF

input = torch.tensor([[328.0770, 231.1015, 279.2261, 457.5734]])
out1 = convert_bounding_box_format(input, BBF.CXCYWH, BBF.XYXY)
out2 = box_convert(input, "cxcywh", "xyxy")

print(torch.allclose(out1, out2)) # True
print((out1 - out2).norm()) # tensor(0.)

[kargibora]

Shouldn't be there some assertions, error or warnings for passing a str? (although the expected format is obvious from the docs, still can happen and confuse people)

[mantasu]

I agree, I actually think both enums and strings (lowercase and uppercase) could be supported

[NicolasHug]

Thank you all for the report. This is a bug, convert_bounding_box_format should properly support strings. Right now the results are silently wrong and it's safer to just use enums. We'll push a fix soon (#8265). Hopefully it will be available soon in the bugfix release 0.17.1, otherwise it will be 0.18

[NicolasHug]

Hi folks, just FYI you can now install 0.17.1 with the fix!