Indexing StdString returns Int8 which breaks most basic string functions

[Keluaa]

When indexing a StdString, a Int8 is returned, which is different from the indexing of Base.String, which returns a Char.

However, as mentioned in the AbstractString documentation, indexing a string should return an AbstractChar.

Therefore most functions operating on strings are broken:

julia> jl_s = "test"; cxx_s = CxxWrap.StdString(jl_s);

julia> jl_s == cxx_s  # comparison is explicitly defined by CxxWrap
true

julia> jl_s[1] == cxx_s[1]  # Internally compares a Char with a Int8
false

julia> occursin(jl_s, cxx_s)
false

julia> isnothing(findfirst('t', cxx_s))
true

I find this quite confusing, since StdString <: AbstractString, Julia happily treats cxx_s as a string yet fails to do any operations on it. Printing gives the correct string representation, == works as expected, but all else breaks.

Furthermore, UTF-8 characters completly break StdString:

julia> jl_s = "∘test"; cxx_s = CxxWrap.StdString(jl_s);

julia> length(jl_s)
5

julia> lengh(cxx_s)
7

julia> foreach(print, jl_s)
∘test

julia> foreach(print, cxx_s)
â

julia> jl_s[1], cxx_s[1]
('∘', -30)

julia> cxx_s[2]
-120

[fingolfin]

I think the best way to fix this is to change StdString to not inherit from AbstractString. The semantics of the two just don't fit; std::string doesn't support UTF-8 directly in any way; trying to make it work as an AbstractString would require implementing a lot of code, and introduce all sorts of inefficiencies.

[Keluaa]

I would agree that returning a non-AbstractString would make the most sense.

I should mention that there is ways to work with UTF-8 in C/C++. A familiar person wrote an article about it, but it is still quite a lot of work to correctly support StdString.

IMO, simply returning a Vector{UInt8} would already be enough. Or maybe a subtype AbstractVector{UInt8} which could be printed as a string. std::string is usually used when making copies of the string is not a problem, therefore needing to explicitly convert to String shouldn't be an issue.

[barche]

I think this is actually fixable, at least the examples shown above pass with the two commits mentioned above. It is unfortunate that at the AbstractString level unicode seems to be specified now, std::string will of course never have that. The objective is not to have StdString behave exactly the same as a Julia String, but keep the ability to pass it around transparently for simple cases, e.g. non-unicode filenames should be OK.