Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update character width tables according to Unicode 9 #294

Closed
wants to merge 2 commits into from

Commits on Jun 24, 2016

  1. Update character width tables according to Unicode 9

    Generated from the updated EastAsianWidth.txt using the following
    script:
    
    ```
    fullwidth = IOBuffer()
    fullwidthsingle = IOBuffer()
    ambiguouswidth = IOBuffer()
    ambiguouswidthsingle = IOBuffer()
    
    function print_to_correct_buffer(rangebuf, singlebuf, range, str)
        if length(range) == 1
            println(singlebuf, "[$str addCharactersInRange:NSMakeRange(0x$(hex(first(range))), 1)];")
        else
            f, l = hex(first(range)), hex(last(range))
            println(rangebuf, "[$str addCharactersInRange:NSMakeRange(0x$f, 0x$l - 0x$f + 1)];")
        end
    end
    
    ranges = Any[]
    for line in readlines(open("EastAsianWidth.txt"))
        #Strip comments
        line[1] == '#' && continue
        precomment = split(line, '#')[1]
        #Parse code point range and width code
        tokens = split(precomment, ';')
        length(tokens) >= 2 || continue
        charrange = tokens[1]
        width = strip(tokens[2])
        #Parse code point range into Julia UnitRange
        rangetokens = split(charrange, "..")
        charstart = parse(UInt32, "0x"*rangetokens[1])
        charend = parse(UInt32, "0x"*rangetokens[length(rangetokens)>1 ? 2 : 1])
        range = charstart:charend
    
        # Coalesce ranges
        if !isempty(ranges) && ranges[end][1] == width && last(ranges[end][2]) == first(range)-1
            ranges[end] = (width, first(ranges[end][2]):last(range))
        else
            push!(ranges, (width, range))
        end
    end
    
    for (width, range) in ranges
        if width=="W" || width=="F" # wide or full
            print_to_correct_buffer(fullwidth, fullwidthsingle, range, "sFullWidth")
        elseif width == "A"
            print_to_correct_buffer(ambiguouswidth, ambiguouswidthsingle, range, "sAmbiguousWidth")
        end
    end
    ```
    Keno committed Jun 24, 2016
    Configuration menu
    Copy the full SHA
    c51e49e View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2016

  1. Configuration menu
    Copy the full SHA
    c8e3bb7 View commit details
    Browse the repository at this point in the history