PyQDecoder Ext not compiling

[EDesec]

After installing and successfully importing PyQDecoders the ext still isn't recognized through the compiling. The the issue with PyQDecoders first was the fact that the file path to usr/bin/local/python had to be defined within the .jl file and then the import 'using PyQDecoder' will run successfully but the REPL will not compile.

The PyQDecoders import should access the the extension but the current code seems to not compile correctly.

Minimal Reproducible Example 👇

(In Julia REPL)
using PyQDecoders
] precompile

Error & Stacktrace or other complete output produced by the MRE ⚠️

Precompiling project...
  ✗ QuantumClifford → QuantumCliffordPyQDecodersExt
  0 dependencies successfully precompiled in 3 seconds. 279 already precompiled.

Stacktrace from the use of a PyQDecoders Decoders:

ERROR: "The `PyMatchingDecoder` depends on the package `PyQDecoders` but you have not installed or imported `PyMatchingDecoder` yet. Immediately after you import `PyQDecoders`, the `PyMatchingDecoder` will be available."
Stacktrace:
 [1] PyMatchingDecoder(args::QuantumCliffordHeckeExt.LPCode; kwargs::@Kwargs{})
   @ QuantumClifford.ECC ~/Documents/QEC Work/QuantumClifford.jl/src/ecc/decoder_pipeline.jl:313
 [2] PyMatchingDecoder
   @ ~/Documents/QEC Work/QuantumClifford.jl/src/ecc/decoder_pipeline.jl:310 [inlined]
 [3] test_multiple_codes(codes::Vector{QuantumCliffordHeckeExt.LPCode}, mem_errors::Vector{Float64}, nsamples::Int64)
   @ Main ~/Documents/QEC Work/scratch_workspace/julia_playground.jl:115
 [4] make_decoder_figure_multiple_tests(phys_errors::Vector{Float64}, intial_li::Vector{Any}, n::Int64, title::String)
   @ Main ~/Documents/QEC Work/scratch_workspace/julia_playground.jl:81
 [5] top-level scope
   @ ~/Documents/QEC Work/scratch_workspace/julia_playground.jl:235

Environment:

• Output of using Pkg; Pkg.status()

Status `~/Documents/QEC Work/Project.toml`
⌃ [13f3f980] CairoMakie v0.12.15
  [a10d1c49] DBInterface v2.6.1
⌃ [3e1990a7] Hecke v0.34.6
  [3c486d74] LDPCDecoders v0.3.1
  [33c8b6b6] ProgressLogging v0.1.4
  [17f5de1a] PyQDecoders v0.2.1
  [0525e862] QuantumClifford v0.9.14 `QuantumClifford.jl`
⌃ [295af30f] Revise v3.6.2
  [0aa819cd] SQLite v1.6.1
  [5d786b92] TerminalLoggers v0.1.7
  [56ddb016] Logging

• Output of using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)

Status `~/Documents/QEC Work/Manifest.toml`
⌃ [c3fe647b] AbstractAlgebra v0.43.7
  [621f4979] AbstractFFTs v1.5.0
  [1520ce14] AbstractTrees v0.4.5
⌃ [79e6a3ab] Adapt v4.1.0
  [35492f91] AdaptivePredicates v1.2.0
  [66dad0bd] AliasTables v1.1.3
⌃ [27a7e980] Animations v0.4.1
  [ec485272] ArnoldiMethod v0.4.0
  [67c07d97] Automa v1.1.0
  [13072b0f] AxisAlgorithms v1.1.0
  [39de3d68] AxisArrays v0.4.7
  [62783981] BitTwiddlingConvenienceFunctions v0.1.6
  [fa961155] CEnum v0.5.0
⌃ [159f3aea] Cairo v1.1.0
⌃ [13f3f980] CairoMakie v0.12.15
  [d360d2e6] ChainRulesCore v1.25.0
  [da1fd8a2] CodeTracking v1.3.6
  [a2cac450] ColorBrewer v0.4.0
⌃ [35d6a980] ColorSchemes v3.27.0
⌅ [3da002f7] ColorTypes v0.11.5
⌅ [c3611d14] ColorVectorSpace v0.10.0
⌅ [5ae59095] Colors v0.12.11
  [861a8166] Combinatorics v1.0.2
  [f70d9fcc] CommonWorldInvalidations v1.0.0
  [34da2185] Compat v4.16.0
  [992eb4ea] CondaPkg v0.2.24
  [187b0558] ConstructionBase v1.5.8
  [d38c429a] Contour v0.6.3
  [a10d1c49] DBInterface v2.6.1
  [9a962f9c] DataAPI v1.16.0
  [864edb3b] DataStructures v0.18.20
  [e2d170a0] DataValueInterfaces v1.0.0
⌃ [927a84f5] DelaunayTriangulation v1.6.1
  [8bb1440f] DelimitedFiles v1.9.1
⌃ [31c24e10] Distributions v0.25.112
  [ffbed154] DocStringExtensions v0.9.3
  [4e289a0a] EnumX v1.0.4
  [429591f6] ExactPredicates v2.2.8
  [411431e0] Extents v0.1.4
  [7a1cc6ca] FFTW v1.8.0
⌃ [5789e2e9] FileIO v1.16.4
  [8fc22ac5] FilePaths v0.8.3
  [48062228] FilePathsBase v0.9.22
  [1a297f60] FillArrays v1.13.0
  [53c48c17] FixedPointNumbers v0.8.5
  [1fa38f19] Format v1.3.7
  [b38be410] FreeType v4.1.1
⌃ [663a7486] FreeTypeAbstraction v0.10.3
  [68eda718] GeoFormatTypes v0.4.2
⌃ [cf35fbd7] GeoInterface v1.3.7
⌅ [5c1252a2] GeometryBasics v0.4.11
⌃ [a2bd30eb] Graphics v1.1.2
  [86223c79] Graphs v1.12.0
⌃ [3955a311] GridLayoutBase v0.11.0
  [42e2da0e] Grisu v1.0.2
⌃ [3e1990a7] Hecke v0.34.6
  [3e5b6fbb] HostCPUFeatures v0.1.17
⌃ [34004b35] HypergeometricFunctions v0.3.24
⌃ [2cd5bd5f] ILog2 v0.2.4
  [615f187c] IfElse v0.1.1
⌃ [2803e5a7] ImageAxes v0.6.11
  [c817782e] ImageBase v0.1.7
⌃ [a09fc81d] ImageCore v0.10.2
⌃ [82e4d734] ImageIO v0.6.8
⌃ [bc367c6b] ImageMetadata v0.9.9
  [9b13fd28] IndirectArrays v1.0.0
  [d25df0c9] Inflate v0.1.5
  [842dd82b] InlineStrings v1.4.2
  [a98d9a8b] Interpolations v0.15.1
⌃ [d1acc4aa] IntervalArithmetic v0.22.18
  [8197267c] IntervalSets v0.7.10
  [3587e190] InverseFunctions v0.1.17
  [92d709cd] IrrationalConstants v0.2.2
  [f1662d9f] Isoband v0.1.1
  [c8e1da08] IterTools v1.10.0
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.6.1
  [682c06a0] JSON v0.21.4
  [0f8b85d8] JSON3 v1.14.1
  [b835a17e] JpegTurbo v0.1.5
⌃ [aa1ae85d] JuliaInterpreter v0.9.36
  [5ab0869b] KernelDensity v0.6.9
  [3c486d74] LDPCDecoders v0.3.1
  [b964fa9f] LaTeXStrings v1.4.0
  [8cdb02fc] LazyModules v0.3.1
  [1d6d02ad] LeftChildRightSiblingTrees v0.2.0
  [2ab3a3ac] LogExpFunctions v0.3.28
⌃ [6f1432cf] LoweredCodeUtils v3.0.5
  [1914dd2f] MacroTools v0.5.13
⌅ [ee78f7c6] Makie v0.21.15
⌅ [20f20a25] MakieCore v0.8.9
  [dbb5928d] MappedArrays v0.4.2
⌃ [0a4f8689] MathTeXEngine v0.6.1
  [0b3b1443] MicroMamba v0.1.14
  [e1d29d7a] Missings v1.2.0
  [e94cdb99] MosaicViews v0.3.4
  [77ba4419] NaNMath v1.0.2
⌃ [2edaba10] Nemo v0.47.2
  [f09324ee] Netpbm v1.1.1
  [510215fc] Observables v0.5.5
  [6fe1bfb0] OffsetArrays v1.14.1
⌃ [52e1d378] OpenEXR v0.3.2
⌃ [bac558e1] OrderedCollections v1.6.3
  [90014a1f] PDMats v0.11.31
  [f57f5aa1] PNGFiles v0.4.3
⌃ [19eb6ba3] Packing v0.5.0
  [5432bcbf] PaddedViews v0.5.12
  [69de0a69] Parsers v2.8.1
  [fa939f87] Pidfile v1.3.0
  [eebad327] PkgVersion v0.3.3
⌃ [995b91a9] PlotUtils v1.4.2
  [647866c9] PolygonOps v0.1.2
  [aea7be01] PrecompileTools v1.2.1
  [21216c6a] Preferences v1.4.3
  [33c8b6b6] ProgressLogging v0.1.4
  [92933f4c] ProgressMeter v1.10.2
  [43287f4e] PtrArrays v1.2.1
  [17f5de1a] PyQDecoders v0.2.1
  [6099a3de] PythonCall v0.9.23
⌃ [4b34888f] QOI v1.0.0
  [1fd47b50] QuadGK v2.11.1
  [0525e862] QuantumClifford v0.9.14 `QuantumClifford.jl`
  [5717a53b] QuantumInterface v0.3.6
  [fb686558] RandomExtensions v0.4.4
  [b3c3ace0] RangeArrays v0.3.2
  [c84ed2f1] Ratios v0.4.5
  [189a3867] Reexport v1.2.2
  [05181044] RelocatableFolders v1.0.1
  [ae029012] Requires v1.3.0
⌃ [295af30f] Revise v3.6.2
  [79098fc4] Rmath v0.8.0
  [5eaf0fd0] RoundingEmulator v0.2.1
  [af85af4c] RowEchelon v0.2.1
⌃ [fdea26ae] SIMD v3.6.0
  [0aa819cd] SQLite v1.6.1
  [6c6a2e73] Scratch v1.2.1
⌅ [65257c39] ShaderAbstractions v0.4.1
  [992d4aef] Showoff v1.0.3
  [73760f76] SignedDistanceFields v0.4.0
  [699a6c99] SimpleTraits v0.9.4
  [45858cf5] Sixel v0.1.3
  [a2af1166] SortingAlgorithms v1.2.1
  [276daf66] SpecialFunctions v2.4.0
  [860ef19b] StableRNGs v1.0.2
  [cae243ae] StackViews v0.1.1
  [aedffcd0] Static v1.1.1
  [90137ffa] StaticArrays v1.9.8
  [1e83bf80] StaticArraysCore v1.4.3
  [82ae8749] StatsAPI v1.7.0
  [2913bbd2] StatsBase v0.34.3
  [4c63d2b9] StatsFuns v1.3.2
⌅ [09ab397b] StructArrays v0.6.18
  [856f2bd8] StructTypes v1.11.0
  [8e1ec7a9] SumTypes v0.5.8
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.12.0
  [62fd8b95] TensorCore v0.1.1
  [5d786b92] TerminalLoggers v0.1.7
⌅ [731e570b] TiffImages v0.10.2
  [3bb67fe8] TranscodingStreams v0.11.3
  [981d1d27] TriplotBase v0.1.0
  [1cfade01] UnicodeFun v0.4.1
⌃ [1986cc42] Unitful v1.21.0
  [e17b2a0c] UnsafePointers v1.0.0
  [ea10d353] WeakRefStrings v1.4.2
  [efce3f68] WoodburyMatrices v1.0.0
  [6e34b625] Bzip2_jll v1.0.8+2
  [4e9b3aee] CRlibm_jll v1.0.1+0
  [83423d85] Cairo_jll v1.18.2+1
  [5ae413db] EarCut_jll v2.2.4+0
⌃ [2e619515] Expat_jll v2.6.2+0
  [b22a6f82] FFMPEG_jll v6.1.2+0
  [f5851436] FFTW_jll v3.3.10+1
  [e134572f] FLINT_jll v300.100.300+0
  [a3f928ae] Fontconfig_jll v2.13.96+0
⌃ [d7e528f0] FreeType2_jll v2.13.2+0
  [559328eb] FriBidi_jll v1.0.14+0
  [78b55507] Gettext_jll v0.21.0+0
⌃ [7746bdde] Glib_jll v2.80.5+0
⌃ [3b182d85] Graphite2_jll v1.3.14+0
  [2e76f6c2] HarfBuzz_jll v8.3.1+0
  [905a6f67] Imath_jll v3.1.11+0
  [1d5cc7b8] IntelOpenMP_jll v2024.2.1+0
  [aacddb02] JpegTurbo_jll v3.0.4+0
  [c1c5ebd0] LAME_jll v3.100.2+0
  [1d63c593] LLVMOpenMP_jll v18.1.7+0
  [dd4b983a] LZO_jll v2.10.2+1
⌅ [e9f186c6] Libffi_jll v3.2.2+1
  [d4300ac3] Libgcrypt_jll v1.11.0+0
  [7add5ba3] Libgpg_error_jll v1.50.0+0
  [94ce4f54] Libiconv_jll v1.17.0+1
⌃ [4b2f31a3] Libmount_jll v2.40.1+0
⌃ [38a345b3] Libuuid_jll v2.40.1+0
  [856f044c] MKL_jll v2024.2.0+0
  [e7412a2a] Ogg_jll v1.3.5+1
⌅ [656ef2d0] OpenBLAS32_jll v0.3.24+0
  [18a262bb] OpenEXR_jll v3.2.4+0
  [458c3c95] OpenSSL_jll v3.0.15+1
  [efe28fd5] OpenSpecFun_jll v0.5.5+0
  [91d4177d] Opus_jll v1.3.3+0
  [36c8627f] Pango_jll v1.54.1+0
⌅ [30392449] Pixman_jll v0.43.4+0
  [f50d1b31] Rmath_jll v0.5.1+0
  [76ed43ae] SQLite_jll v3.45.3+0
⌃ [02c8fc9c] XML2_jll v2.13.4+0
⌃ [aed1982a] XSLT_jll v1.1.41+0
⌃ [4f6342f7] Xorg_libX11_jll v1.8.6+0
⌃ [0c0b7dd1] Xorg_libXau_jll v1.0.11+0
⌃ [a3789734] Xorg_libXdmcp_jll v1.1.4+0
⌃ [1082639a] Xorg_libXext_jll v1.3.6+0
  [ea2f1a96] Xorg_libXrender_jll v0.9.11+0
⌃ [14d82f49] Xorg_libpthread_stubs_jll v0.1.1+0
⌃ [c7cfdc94] Xorg_libxcb_jll v1.17.0+0
⌃ [c5fb5394] Xorg_xtrans_jll v1.5.0+0
  [9a68df92] isoband_jll v0.2.3+0
  [a4ae2306] libaom_jll v3.9.0+0
  [0ac62f75] libass_jll v0.15.2+0
  [f638f0a6] libfdk_aac_jll v2.0.3+0
  [b53b4c65] libpng_jll v1.6.44+0
  [075b6546] libsixel_jll v1.10.3+1
  [f27f6e37] libvorbis_jll v1.3.7+2
  [f8abcde7] micromamba_jll v1.5.8+0
  [1317d2d5] oneTBB_jll v2021.12.0+0
  [1270edf5] x264_jll v10164.0.0+0
⌅ [dfaa095f] x265_jll v3.6.0+0
  [0dad84c5] ArgTools v1.1.1
  [56f22d72] Artifacts
  [2a0f44e3] Base64
  [8bf52ea8] CRC32c
  [ade2ca70] Dates
  [8ba89e20] Distributed
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching
  [b77e0a4c] InteractiveUtils
  [4af54fe1] LazyArtifacts
  [b27032c2] LibCURL v0.6.4
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.10.0
  [de0858da] Printf
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization
  [1a1011a3] SharedArrays
  [6462fe0b] Sockets
  [2f01184e] SparseArrays v1.10.0
  [10745b16] Statistics v1.10.0
  [4607b0f0] SuiteSparse
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
  [e66e0078] CompilerSupportLibraries_jll v1.1.1+0
  [781609d7] GMP_jll v6.2.1+6
  [deac9b47] LibCURL_jll v8.4.0+0
  [e37daf67] LibGit2_jll v1.6.4+0
  [29816b5a] LibSSH2_jll v1.11.0+1
  [3a97d323] MPFR_jll v4.2.0+1
  [c8ffd9c3] MbedTLS_jll v2.28.2+1
  [14a3606d] MozillaCACerts_jll v2023.1.10
  [4536629a] OpenBLAS_jll v0.3.23+4
  [05823500] OpenLibm_jll v0.8.1+2
  [efcefdf7] PCRE2_jll v10.42.0+1
  [bea87d4a] SuiteSparse_jll v7.2.1+1
  [83775a58] Zlib_jll v1.2.13+1
  [8e850b90] libblastrampoline_jll v5.11.0+0
  [8e850ede] nghttp2_jll v1.52.0+1
  [3f19e933] p7zip_jll v17.4.0+2

• Output of versioninfo()

Julia Version 1.10.5
Commit 6f3fdf7b362 (2024-08-27 14:19 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin22.4.0)
  CPU: 10 × Apple M1 Pro
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, apple-m1)
Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)
Environment:
  JULIA_EDITOR = code
  JULIA_NUM_THREADS = 
  JULIA_CONDAPKG_BACKEND = Null
  JULIA_PYTHONCALL_EXE = /usr/local/bin/python3.13

[Fe-r-oz]

Hi, @EDesec,

Apologies for the late response. It seems you are using Julia Version 1.10.5. Please update to Julia 1.11.1 from 1.10.5 because there were compatibility issues with hgp constructor , please see #394

Please try: using QuantumClifford; using QuantumClifford.ECC; import PyQDecoders

julia> using QuantumClifford; using QuantumClifford.ECC; import PyQDecoders
Precompiling PyQDecoders...
  3 dependencies successfully precompiled in 21 seconds. 50 already precompiled.
Precompiling QuantumCliffordPyQDecodersExt...
  1 dependency successfully precompiled in 6 seconds. 89 already precompiled.

Versioninfo() and package QuantumClifford v0.9.14

julia> versioninfo()
Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × Intel(R) Core(TM) i5-7300HQ CPU @ 2.50GHz
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, skylake)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

[Krastanov]

@Fe-r-oz , I do not believe your suggestion is correct. This should not be related to Julia versions: both QuantumClifford and PyQDecoders support Julia 1.10 (and even 1.9) as you can see from their Project.toml files. It also has nothing to do with the hgp function -- that function is neither used in this MWE, nor was there a released version that had an issue with hgp (that was only briefly on the master branch).

[Krastanov]

@EDesec , any chance there are any earlier error messages. I think the order of events is:

1. you are installing PyQDecoders
2. precompilation fails but it does not give a detailed error message about why it failed
3. you try to use a function that depends on PyQDecoders but you get an error message about PyQDecoders not being installed because of point 2

I am not sure, but I think currently you have posted the error message from step 2. Could you try the following: in a new environment install only PyQDecoders and try using PyQDecoders. Hopefully that would generate error messages specific to step 2.

[Fe-r-oz]

that function is neither used in this MWE, nor was there a released version that had an issue with hgp

Stacktrace:
 [1] PyMatchingDecoder(args::QuantumCliffordHeckeExt.LPCode; kwargs::@Kwargs{})
   @ QuantumClifford.ECC ~/Documents/QEC Work/QuantumClifford.jl/src/ecc/decoder_pipeline.jl:313
 [2] PyMatchingDecoder
   @ ~/Documents/QEC Work/QuantumClifford.jl/src/ecc/decoder_pipeline.jl:310 [inlined]

@Krastanov , I went on tangent (after seeing the above stacktrace) because that function is called under the hood anytime LPCode is called. That issue was because Julia 1.11.1 got released on Oct 17, the same day I posted the issue because errors started occurring on the master branch due to that. Anyhow, I should have started earlier message "As a side note, not directly related to issue here... " just in case.

@EDesec, sorry about going on the tangent.