Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

StackOverflowError when using callable syntax #24

Open
bencottier opened this issue Nov 22, 2021 · 5 comments
Open

StackOverflowError when using callable syntax #24

bencottier opened this issue Nov 22, 2021 · 5 comments
Labels
bug Something isn't working

Comments

@bencottier
Copy link
Contributor

bencottier commented Nov 22, 2021 •
edited
Loading

MWE:

julia> using AxisKeys, Distributions, KeyedDistributions

julia> keys = ([:a, :b, :c], );

julia> d = KeyedDistribution(MvNormal(rand(3)), keys)
KeyedMvNormal{ZeroMeanDiagNormal{Tuple{Base.OneTo{Int64}}}}(
d: ZeroMeanDiagNormal(
dim: 3
μ: Zeros(3)
Σ: [0.083005916265146 0.0 0.0; 0.0 0.6484896856503174 0.0; 0.0 0.0 0.04377962068806388]
)

keys: ([:a, :b, :c],)
)

julia> d(:a)
Internal error: encountered unexpected error in runtime:
StackOverflowError()
subtype_ccheck at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
var_gt at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
forall_exists_subtype at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)

<repeat above 3 lines many times>

subtype_ccheck at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
var_gt at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
forall_exists_equal at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
subtype at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
with_tvar at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
subtype at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
with_tvar at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
subtype at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
with_tvar at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
subtype at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
subtype_tuple_tail at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
subtype at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
with_tvar at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
subtype at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
with_tvar at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
subtype at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
with_tvar at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
subtype at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
with_tvar at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
subtype at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
forall_exists_subtype at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_subtype_env at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_type_intersection_env_s at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
ml_matches at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_matching_methods at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
_methods_by_ftype at ./reflection.jl:867 [inlined]
#findall#216 at ./compiler/methodtable.jl:57 [inlined]
findall##kw at ./compiler/methodtable.jl:54
jfptr_findallYY.YY.kw_13388.clone_1 at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
#218 at ./compiler/methodtable.jl:67
get! at ./iddict.jl:163 [inlined]
#findall#217 at ./compiler/methodtable.jl:66 [inlined]
findall##kw at ./compiler/methodtable.jl:65
jfptr_findallYY.YY.kw_13383.clone_1 at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:84
abstract_call_known at ./compiler/abstractinterpretation.jl:1033
abstract_call at ./compiler/abstractinterpretation.jl:1056
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call_known at ./compiler/abstractinterpretation.jl:1033
abstract_call at ./compiler/abstractinterpretation.jl:1056
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call_known at ./compiler/abstractinterpretation.jl:1033
abstract_call at ./compiler/abstractinterpretation.jl:1056
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call_known at ./compiler/abstractinterpretation.jl:1033
abstract_call at ./compiler/abstractinterpretation.jl:1056
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_ext at ./compiler/typeinfer.jl:892
typeinf_ext_toplevel at ./compiler/typeinfer.jl:925
typeinf_ext_toplevel at ./compiler/typeinfer.jl:921
jfptr_typeinf_ext_toplevel_13551.clone_1 at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_type_infer at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_generate_fptr at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_compile_method_internal at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
do_call at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
eval_body at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_interpret_toplevel_thunk at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
top-level scope at REPL[26]:1
jl_toplevel_eval_flex at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_toplevel_eval_flex at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_toplevel_eval_flex at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_toplevel_eval_in at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
eval at ./boot.jl:360 [inlined]
eval_user_input at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:139
repl_backend_loop at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:200
start_repl_backend at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:185
#run_repl#42 at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:317
run_repl at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:305
jl_apply_generic at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
#874 at ./client.jl:387
jfptr_YY.874_23727.clone_1 at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_f__call_latest at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
#invokelatest#2 at ./essentials.jl:708 [inlined]
invokelatest at ./essentials.jl:706 [inlined]
run_main_repl at ./client.jl:372
exec_options at ./client.jl:302
_start at ./client.jl:485
jfptr__start_37496.clone_1 at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
true_main at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
repl_entrypoint at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
KeyedDistribution{Univariate, Continuous, Normal{Float64}}(
d: Normal{Float64}(μ=0.0, σ=0.083005916265146)
keys: ([:a],)
)

Note that it still returns the correct result, but I have seen it cause total failure in downstream code.

I don't think this is covered by tests as they still pass locally without this error. It doesn't seem to be a problem for FullNormal. The code below is adapted from the "marginalising" testset:

using StableRNGs

X = rand(StableRNG(1234), 10, 3)
m = vec(mean(X; dims=1))
s = cov(X; dims=1)

d = KeyedDistribution(MvNormal(m, s), keys)

d(:a)

# output
KeyedDistribution{Univariate, Continuous, Normal{Float64}}(
d: Normal{Float64}(μ=0.5753194713591709, σ=0.0839957973357219)
keys: ([:a],)
)
@bencottier bencottier added the bug Something isn't working label Nov 22, 2021
@bencottier
Copy link
Contributor Author

It's the callable syntax in particular because if I index the distribution as implemented here it's ok:

julia> d[first(map(AxisKeys.findindex, (:a,), axiskeys(d)))]
KeyedDistribution{Univariate, Continuous, Normal{Float64}}(
d: Normal{Float64}(μ=0.0, σ=0.083005916265146)
keys: ([:a],)
)

@bencottier
Copy link
Contributor Author

Distributions v0.24 vs. v0.25 doesn't make a difference

@mzgubic
Copy link
Collaborator

mzgubic commented Nov 22, 2021 •
edited
Loading

Tested on Julia 1.6.4 and also same error

@iamed2
Copy link
Member

iamed2 commented Nov 22, 2021

I checked and this does not happen on Julia branches release-1.5, release-1.7, or master. Seems to be a 1.6-specific issue.

@bencottier bencottier mentioned this issue Nov 23, 2021
Open
@devmotion devmotion mentioned this issue Dec 16, 2021
Open
@AlexRobson
Copy link
Member

AlexRobson commented Jun 6, 2022 •
edited
Loading

I've started hitting this. Two comments fwiw:

  • It still appears in julia 1.6.6
  • I don't believe this is due to the callable syntax for the following reason:

Although I can recover all the errors as listed here including the syntax one, if we define:

_marginalise(d, keys) = d[first(map(AxisKeys.findindex, keys, axiskeys(d)))]

and replace this line

(d::$KeyedT)(_keys...) = _marginalise(d, _keys)`

then:

d(:a) # StackOverFlow
KeyedDistributions._marginalise(d, [:a, :b]) # StackOverFlow - bypasses callable struct 
d[first(map(AxisKeys.findindex, [:a,:b], axiskeys(d)))] # Fine

I suspect it worked because of some running-in-the-REPL reason which seems to affect type inference sometimes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@iamed2 @AlexRobson @mzgubic @bencottier