live_grep_glob feature request

[mikesart]

I added the below check for some ripgrep search flags to glob_parse and pass those through.

This allows me to do a live_grep_glob and add -w, -i, etc, etc on the fly to ripgrep, which is pretty useful. (I'm currently using vim-agriculture for this, but I like FzfLua better. :)

There might be a better way to do this, but for quick and dirty the below is working pretty well for me right now.

No pressure taking this or not, of course. Thanks!

M.glob_parse = function(opts, query)
  if not query or not query:find(opts.glob_separator) then
    return query, nil
  end
  local glob_args = ""
  local search_query, glob_str = query:match("(.*)"..opts.glob_separator.."(.*)")
  for _, s in ipairs(utils.strsplit(glob_str, "%s")) do
    if string.match(s, '^-%a$') then
      glob_args = glob_args .. s .. ' '
    else
      glob_args = glob_args .. ("%s %s ")
        :format(opts.glob_flag, vim.fn.shellescape(s))
    end
  end
  return search_query, glob_args
end

[mikesart]

Modified to support -t option...

M.glob_parse = function(opts, query)
  if not query or not query:find(opts.glob_separator) then
    return query, nil
  end
  local is_type = false
  local glob_args = ""
  local search_query, glob_str = query:match("(.*)"..opts.glob_separator.."(.*)")
  for _, s in ipairs(utils.strsplit(glob_str, "%s")) do
    if is_type == true then
      glob_args = glob_args .. '-t ' .. s .. ' '
      is_type = false
    elseif s == '-t' then
      is_type = true
    elseif string.match(s, '^-%a$') then
      glob_args = glob_args .. s .. ' '
    else
      glob_args = glob_args .. ("%s %s ")
        :format(opts.glob_flag, vim.fn.shellescape(s))
    end
  end
  return search_query, glob_args
end

[ibhagwan]

Hi @mikesart,

I see no harm in this enhancement as there aren’t many useful globs (if any) with a dash followed by a letter so it’s somewhat of a “free upgrade”.

That said, I’m not sure how many people are aware of all rg flags and how to use them, it’s hard enough as is to even explain what’s the difference grep and live grep lol

If this were to be added to the code I’d prefer it being more consistent in behavior, not sure how much I like a single hard coded exception for -t, would also be good if this also handled double dash flags (I.e --type).

I’m thinking maybe it’s best to just parse the string for an additional glob separator (use the same separator or different one?) and pass everything after it “as is” to the command? I.e. you can type foo -- *.h -- -i -t bar which will translate to rg -iglob=*.h -i -t bar 'foo'

[mikesart]

Could also make it so this function is overridable by the user if you’d prefer that? Thanks!

[ibhagwan]

Could also make it so this function is overridable by the user if you’d prefer that? Thanks!

Already thought about that but it’s not as straight forward as you’d think, for performance reasons most commands are run externally (with multiprocess=true) which means that the function override won’t exist in memory in the neovim headless instance (the instance doing the processing of the command input/output).

There are a couple of solutions for that but they aren’t pretty, the first would be to compile the user function to bytes, write it in a temp file and load it by the external instance which adds complexity and many other potential issues, the below is from packer.nvim which uses string.dump to save user functions:

NOTE: If you use a function value for config or setup keys in any plugin specifications, it must not have any upvalues (i.e. captures). We currently use Lua's string.dump to compile config/setup functions to bytecode, which has this limitation. Additionally, if functions are given for these keys, the functions will be passed the plugin name and information table as arguments.

The other option would be to perform the parsing in the main instance by executing the function over an RPC request, this would break the model of the external processing instance being independent of the main instance.

What I can try to do is query the user config in the pre-processing and pass the function override to the external instance as bytecode using string.dump this way it won’t have to query the main instance every keypress.

[mikesart]

On this page:

https://github.com/nanotee/nvim-lua-guide

There is a way to load a module and not error out if it doesn't exist. Ie:

local ok, _ = pcall(require, 'module_with_error')
if not ok then
  -- not loaded
end

Would something like this work where you use an override module if it exists and use yours if not?

You are way ahead of me on everything lua and neovim, so I'm for sure asking these questions for my own education. I very much appreciate your continued patience. :)

Also, I'm more than happy to just fork fzf-lua and keep it up to date with some minor tweaks I want that don't make sense for other folks. That's super easy as well.

Thanks ibhagwan!

[ibhagwan]

There is a way to load a module and not error out if it doesn't exist. Ie:
Would something like this work where you use an override module if it exists and use yours if not?

This is unrelated and won't work as these modules don't exist in the context of the neovim headless instance (it's a different process altogether), that's why I have the below "hacks" in libuv.lua to override the require statement to use local modules instead:

fzf-lua/lua/fzf-lua/libuv.lua

Lines 5 to 19 in 97a63e6

	-- path to current file
	local __FILE__ = debug.getinfo(1, 'S').source:gsub("^@", "")
	-- if loading this file as standalone ('--headless --clean')
	-- add the current folder to package.path so we can 'require'
	if not vim.g.fzf_lua_directory then
	-- prepend this folder first so our modules always get first
	-- priority over some unknown random module with the same name
	package.path = (";%s/?.lua;"):format(vim.fn.fnamemodify(__FILE__, ':h'))
	.. package.path
	-- override require to remove the 'fzf-lua.' part
	-- since all files are going to be loaded locally
	local _require = require
	require = function(s) return _require(s:gsub("^fzf%-lua%.", "")) end

The issue here is that your custom config module doesn't exist in this context and would require package.path to include your neovim config path and also load the function from the file as you can't use the func ref in the external instance.

Also, I'm more than happy to just fork fzf-lua and keep it up to date with some minor tweaks I want that don't make sense for other folks. That's super easy as well.

No need for that, I personally would hate that (having to keep updaging my own fork).

f3d0789 - this commit adds the ability to include custom function in your setup under grep.rg_glob_fn, the main instance is then queried (using vim.rpcrequest) and the function bytecode (using string.dump) is passed back to the external instance and loaded with lua's loadstring.

The only limiation is that you can't call upvalues (no require calls) so utils.strsplit will have to be replaced with vim.split (or your can create a local function inside 'rg_opts_fn'), no big deal and works great, add the below to your setup under grep:

require'fzf-lua'.setup {
  grep = {
    rg_glob           = true,
    rg_glob_fn        = function(opts, query)
      local glob_args = nil
      local search_query, glob_str = query:match("(.*)"..opts.glob_separator.."(.*)")
      for _, s in ipairs(vim.split(glob_str, " ")) do
        s = vim.trim(s)
        if #s>0 then
          glob_args = glob_args or ""
          if string.match(s, '^-%a$') then
            glob_args = glob_args .. s .. ' '
          else
            glob_args = glob_args .. ("%s %s ")
              :format(opts.glob_flag, vim.fn.shellescape(s))
          end
        end
      end
      -- UNCOMMENT TO DEBUG PRINT INTO FZF
      -- if glob_args then
      --   io.write(("q: %s -> flags: %s, query: %s\n"):format(
      --     query, glob_args, search_query))
      -- end
      return search_query, glob_args
    end,
  }
}

Just tested with -i flag for case-insensitive search:

[mikesart]

Interesting. Thanks for the description.

Also, your patch works great. I've got a rg_glob_fn in my init.lua and it's perfect.

Thank you very, very much ibhagwan!

[ibhagwan]

@mikesart quick note, turns out that when I implemented this I changed the way glob parsing worked by a tiny bit, 58320a2 - latest commit restores the functionality where the glob separator is removed from the search query immedietly after typing the separator.

I recommend you do this for your custom function too, all you need to do is change local glob_args = nil -> local glob_args = "", this way glob_args are processed even without typing a glob parameter so you get the results faster.

I found this out during a test I ran for #381 which you also might find interesting as you might have some uses for the filter option.

[mikesart]

Ah, very cool - I updated my filter. And thanks for the pointer to 381, learned some things from reading that.

Thanks for the pointer and have a great weekend ibhagwan!

[ibhagwan]

Hi @mikesart, quick FYI, I'm doing some major refactoring in order to make the plugin much easier to extend, as part of that I'm changing some of the API's, you can read about it more here (still WIP): https://github.com/ibhagwan/fzf-lua/wiki/Advanced

One important thing for you is that the signature of rg_glob_fn changes from:

rg_glob_fn = function(opts, query)

to:

rg_glob_fn = function(query, opts)

Keep an eye out for the next commit (still testing and making more changes in the develop branch), once that's commited just change the order of of the call args and place query as first argument and you'll be fine.

[mikesart]

Oh, awesome. Thank you very much for the heads up - much appreciated!

[ibhagwan]

Oh, awesome. Thank you very much for the heads up - much appreciated!

FYI, change was just pushed.

Also, if you're intestsed in the new API and making your own commands (rather easily I might say), read this:
https://github.com/ibhagwan/fzf-lua/wiki/Advanced

[mikesart]

This is great to hear. I've got the below right now and will look at moving it over to your new api when I get some free time.

Thanks very much, really appreciate all your work on this super useful plugin!

local shortcut_contents = {
  { text = "<leader>e"                           , data = "(netrw) Open :Lexplore" },

  { text = "<leader>l"                           , data = "set invlist" },
  { text = "<leader>c"                           , data = "Clear highlighting" },

  { text = "<leader>t"                           , data = "TabToggle" },

  { text = "<leader>rv"                          , data = "(vimrc) Reload .vimrc" },

  { text = ":%!xxd"                              , data = "(edit) Show buffer in hex" },
  { text = ":%!xxd -r"                           , data = "(edit) Convert from hex back to text" },
  { text = ":%!xxd -i"                           , data = "(edit) Convert to char array for C files" },

-- ...
}

local utils = require "fzf-lua.utils"

-- Previewer class inherits from base previewer
-- the only required method is 'populate_preview_buf'
local Previewer = {}

Previewer.base = require('fzf-lua.previewer.builtin').base
-- inheriting from 'buffer_or_file' gives us access to filetype
-- detection and syntax highlighting helpers for file based previews
-- Previewer.buffer_or_file = require('fzf-lua.previewer.builtin').buffer_or_file

function Previewer:new(o, opts, fzf_win)
  self = setmetatable(Previewer.base(o, opts, fzf_win), {
    __index = vim.tbl_deep_extend("keep",
      self, Previewer.base
      -- only if you need access to specific file methods
      -- self, Previewer.buffer_or_file, Previewer.base
    )})
  return self
end

local function SplitStr(s, delimiter)
    result = {};
    for match in (s..delimiter):gmatch("(.-)"..delimiter) do
        table.insert(result, match);
    end
    return result;
end

function Previewer:populate_preview_buf(entry_str)
  local lines = { SplitStr(entry_str, " ")[3] }

  -- vim.api.nvim_buf_set_option(self.preview_bufnr, 'modifiable', true)
  vim.api.nvim_buf_set_lines(self.preview_bufnr, 0, -1, false, lines)

  -- mark the buffer for unloading on next preview call
  self.preview_bufloaded = true

  -- enable syntax highlighting
  local filetype = 'cpp'
  vim.api.nvim_buf_set_option(self.preview_bufnr, 'filetype', filetype)
end

local my_ex_run = function(selected)
  vim.cmd("stopinsert")

  local cmd = SplitStr(selected[1], " ")[1]
  local keys = vim.api.nvim_replace_termcodes(cmd, true, false, true)
  vim.api.nvim_feedkeys(keys, "t", true)
  return cmd
end

function Shortcuts()
  -- this function feeds elements into fzf
  -- each call to `fzf_cb()` equals one line
  -- `fzf_cb(nil)` closes the pipe and marks EOL to fzf
  local fn = function(fzf_cb)
      local i = 1
      for _, e in ipairs(shortcut_contents) do
        local pad = string.rep(' ', 20 - #e.text)
        fzf_cb(("%s %s %s"):format(utils.ansi_codes.yellow(e.text), pad, e.data))
        i = i + 1
      end
      fzf_cb(nil)
    end

  local actions = {
    ["default"] = my_ex_run,
    ["ctrl-e"]  = my_ex_run,
  }

  coroutine.wrap(function()
    local selected = require('fzf-lua').fzf({
      prompt = 'Shortcuts❯ ',
      previewer = Previewer,
      actions = actions,
      fzf_opts = {
        ["--delimiter"] = '.',
        -- ["--header"] = arg_header("<CR>", "<Ctrl-e>", "execute"),
      },
    }, fn)
    require('fzf-lua').actions.act(actions, selected, {})
  end)()
end

[ibhagwan]

Ty @mikesart, let me know if you need help, btw I added lua type to your code snippet so it displays lua highlights.