Using Listen on Rails root spams errors on local symlinks

[wjordan]

Hello, I'm attempting to use spring-watcher-listen in a large Rails project containing several local symlinks, all of which serve a specific purpose and cannot be removed. Using spring-watcher-listen with recent versions of listen (since guard/listen#273), loading spring spams the following error once for every local symlink in the project:

        ** ERROR: directory is already being watched! **

        Directory: [path]/dir

        is already begin watched through: [path]/dir

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors

In response to the issue guard/listen#339 I opened, @e2 suggested that my case might in fact be better solved by patching spring-watcher-listen to call that library more appropriately:

I'm also pretty sure the spring Listen watcher doesn't need to watch the
whole project - so maybe it needs to be patched instead.

Does this suggestion sound correct? If so, any ideas on whether/how it would be possible to patch this library to watch a more selective set of files, in order to resolve this local-symlink error-spam issue in recent versions of Listen?

[e2]

https://github.com/jonleighton/spring-watcher-listen/blob/master/lib/spring/watcher/listen.rb#L58

You can probably try monkey patching this method to set your own set of dirs to avoid rewatching the symlinked dirs.

[wjordan]

Thanks for the suggestion @e2. I spent some time trying the suggested patch, but unfortunately it looks like avoiding a watch on the project root is not possible. Spring must watch Gemfile for changes in the project root directory, and since (as far as I can tell) the Listen API doesn't watch individual files for changes but only recursively watches directory trees, this means the only way to watch Gemfile for changes is to recursively watch the root project.

[wjordan]

Actually, I managed to workaround the Gemfile recursive-watch constraints by moving Gemfile and Gemfile.lock into a subfolder and adding a symlink to their expected location, e.g.:

mkdir .Gemfile
mv Gemfile* .Gemfile/
ln -s .Gemfile/Gemfile Gemfile
ln -s .Gemfile/Gemfile.lock Gemfile.lock

Along with this monkey-patch that removes the root from the base_directories array:

# Monkey-patch spring-watcher-listen to not watch the project root by default
require 'spring/watcher/listen'
module Spring
  module Watcher
    class Listen < Abstract
      def base_directories
        (files.map { |f| File.expand_path("#{f}/..") } +
          directories.to_a
        ).uniq.map { |path| Pathname.new(path) }
      end
    end
  end
end

This allows my project to successfully load spring-watcher-listen without the error spam and without suppressing the errors directly. I look forward to ideas on how to incorporate these fixes into a cleaner, less-hacky solution.

[e2]

Looks really good to me - guard-bundler actually does something simillar (expects Gemfile symlinked from config directory - or warning if not).

Removing watching the root directory is the right way to avoid performance problems on OSX, because sadly rb-fsevent is recursive.

In fact, the reason to use symlinks is to avoid watching the root directory for large Rails projects on OSX.

Summary: OSX-specific optimizations in Listen can mitigate the performance problems - and this would remove the need for warnings to begin with. Polling has a similar problem, though.

[goncalvesjoao]

Hi guys, I had the same problem that @wjordan had and after an unsuccessful monkey patch I ended up in here.

Should I be looking at guard/listen instead?

[e2]

@goncalvesjoao - figure out a way to not watch symlinked directories. I don't know what your project layout is, so I can't help much.

There might be a better way of organizing your project - but that's project specific. (E.g. why exactly do you need symlinks? Maybe you can configure one tool to reference the original files instead of telling it to use the symlinked location?).

As a quick (but detrimental?) workaround, just copy the files physically instead of symlinking them.

In short: watching directories and their symlinked version isn't supported. It's complex - rocket-science level. (It's impossible to handle symlinks in a way to make everyone happy).

So the rule of thumb is: avoid using symlinks when watching files.

(The other option is: you can hire someone to add support for your exact situation in Listen - just saying it's ungrateful work otherwise).

[ybart]

I had the following error on my development environment:

        ** ERROR: directory is already being watched! **

        Directory: ./vendor/cache/rails-6f9b01c056cd/activerecord/test/fixtures/all/admin

        is already being watched through: ./vendor/cache/rails-6f9b01c056cd/activerecord/test/fixtures/to_be_linked

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors

I solved it by removing my local bundler cache:

bundle config --delete BUNDLE_CACHE_ALL
rm -rf vendor/cache/

[jdabrowski1337]

I was able to get rid of the errors by using this monkey patch:

require 'listen/record/symlink_detector'
module Listen
  class Record
    class SymlinkDetector
      def _fail(_, _)
        raise Error, "Don't watch locally-symlinked directory"
      end
    end
  end
end