Feature overlapping markers

[swtormy]

Add OverlappingMarkerSpiderfier Plugin

Overview

In the pull request, I added a new OverlappingMarkerSpiderfier plugin to Folium. It solves the problem of overlapping markers on maps by “stretching” them into a distinct, scattered pattern. This feature will allow the user to work with adjacent markers, eliminating the problem described in #901.

Key Features

• Spiderfying Markers: Automatically spreads out overlapping markers when clicked, making them individually accessible.
• Customizable Options: Supports options like keepSpiderfied, nearbyDistance, and legWeight (description here).
• Popup Integration: Displaying their content when markers are clicked.

Changes in This PR

1. New Plugin: Added OverlappingMarkerSpiderfier in folium/plugins/overlapping_marker_spiderfier.py.
2. Documentation: Added a detailed usage guide in docs/user_guide/plugins/overlapping_marker_spiderfier.md.
3. Tests: Included unit tests in tests/plugins/test_overlapping_marker_spiderfier.py.
4. Import in Init: Updated folium/plugins/__init__.py to include the new plugin.

Example Usage

Here's a simple example to demonstrate the plugin:

import folium
from folium.plugins import OverlappingMarkerSpiderfier

# Create a map
m = folium.Map(location=[45.05, 3.05], zoom_start=14)

# Generate overlapping markers
markers = [
    folium.Marker(
        location=[45.05, 3.05], 
        options={'desc': f'Marker {i+1}'}
    ) for i in range(10)
]

# Add the plugin to the map
oms = OverlappingMarkerSpiderfier(
    markers=markers,
    options={'keepSpiderfied': True, 'nearbyDistance': 20}
)
oms.add_to(m)

# Save the map
m.save("example_map.html")

Addressed Issue

This PR resolves Issue #901, which requested a solution for handling overlapping markers on maps. The proposed solution integrates the OverlappingMarkerSpiderfier Leaflet plugin into Folium.

Testing

• Ran all existing and new tests to ensure the changes do not break existing functionality.

(venv) C:\Users\swtor\Desktop\python\folium>python -m pytest tests --ignore=tests/selenium
================================================================== test session starts ===================================================================
platform win32 -- Python 3.12.3, pytest-8.3.3, pluggy-1.5.0
rootdir: C:\Users\swtor\Desktop\python\folium
configfile: pyproject.toml
plugins: anyio-4.6.2.post1, nbval-0.11.0
collected 205 items

tests\plugins\test_antpath.py .                                                                                                                     [  0%]
tests\plugins\test_beautify_icon.py .                                                                                                               [  0%]
tests\plugins\test_boat_marker.py ..                                                                                                                [  1%]
tests\plugins\test_dual_map.py .                                                                                                                    [  2%]
tests\plugins\test_encoded.py ..                                                                                                                    [  3%]
tests\plugins\test_fast_marker_cluster.py ....                                                                                                      [  5%]
tests\plugins\test_feature_group_sub_group.py .                                                                                                     [  5%]
tests\plugins\test_float_image.py .                                                                                                                 [  6%]
tests\plugins\test_fullscreen.py .                                                                                                                  [  6%]
tests\plugins\test_grouped_layer_control.py .                                                                                                       [  7%]
tests\plugins\test_heat_map.py ...                                                                                                                  [  8%]
tests\plugins\test_heat_map_withtime.py .                                                                                                           [  9%]
tests\plugins\test_marker_cluster.py .                                                                                                              [  9%]
tests\plugins\test_minimap.py .                                                                                                                     [ 10%]
tests\plugins\test_overlapping_marker_spiderfier.py .                                                                                               [ 10%]
tests\plugins\test_pattern.py .                                                                                                                     [ 11%]
tests\plugins\test_polyline_offset.py ....                                                                                                          [ 13%]
tests\plugins\test_polyline_text_path.py .                                                                                                          [ 13%]
tests\plugins\test_realtime.py .                                                                                                                    [ 14%]
tests\plugins\test_scroll_zoom_toggler.py .                                                                                                         [ 14%] 
tests\plugins\test_semicircle.py .                                                                                                                  [ 15%]
tests\plugins\test_tag_filter_button.py .                                                                                                           [ 15%]
tests\plugins\test_terminator.py .                                                                                                                  [ 16%]
tests\plugins\test_time_slider_choropleth.py .                                                                                                      [ 16%]
tests\plugins\test_timeline.py .                                                                                                                    [ 17%]
tests\plugins\test_timestamped_geo_json.py .                                                                                                        [ 17%] 
tests\plugins\test_vectorgrid_protobuf.py ...                                                                                                       [ 19%]
tests\test_features.py ....................                                                                                                         [ 28%]
tests\test_folium.py ....................                                                                                                           [ 38%]
tests\test_jinja.py ..............                                                                                                                  [ 45%] 
tests\test_map.py ................                                                                                                                  [ 53%]
tests\test_raster_layers.py ......                                                                                                                  [ 56%]
tests\test_repr.py ......                                                                                                                           [ 59%]
tests\test_template.py .........                                                                                                                    [ 63%] 
tests\test_utilities.py ....................................................................                                                        [ 96%]
tests\test_vector_layers.py .......                                                                                                                 [100%]

================================================================== 205 passed in 48.18s ==================================================================

(venv) C:\Users\swtor\Desktop\python\folium>python -m pytest tests/selenium
================================================================== test session starts =================================================================== 
================================================================== test session starts =================================================================== 
platform win32 -- Python 3.12.3, pytest-8.3.3, pluggy-1.5.0
platform win32 -- Python 3.12.3, pytest-8.3.3, pluggy-1.5.0
rootdir: C:\Users\swtor\Desktop\python\folium
configfile: pyproject.toml
plugins: anyio-4.6.2.post1, nbval-0.11.0
collected 70 items                                                                                                                                         

tests\selenium\test_geojson_selenium.py
DevTools listening on ws://127.0.0.1:52531/devtools/browser/699f8db3-b007-48b7-866d-dc8f09834ce4
.                                                                                                           [  1%]
tests\selenium\test_heat_map_selenium.py .                                                                                                          [  2%] 
tests\selenium\test_selenium.py ....Created TensorFlow Lite XNNPACK delegate for CPU.
...................................................x............                                                [100%]

======================================================= 69 passed, 1 xfailed in 206.61s (0:03:26) ========================================================

• Successfully tested map visualizations with overlapping markers.

PS: Feedback is welcome! If there are additional improvements or refinements needed, I am happy to make the necessary adjustments.

[hansthen]

Looks good to me. Nice to have this ability.

@Conengmo Do you also want to review?

[hansthen]

I really like how you used an IFFE here. I think we should use these for all our templates.

[swtormy]

Hi! I looked up this method in FastMarkerCluster, so at least one plugin already uses IFFE)

[Conengmo]

Thanks for taking this up @swtormy! I love how complete your PR is!

One thing that tripped me up a bit though is whether marker objects should be added to the map or a feature group before being added to the OverlappingMarkerSpiderfier object. Turns out, looking at your implementation, they should not. Because the OMS class does create the markers in JS in its template. But in the docs example the markers are also added to the map.

That issue also led me to question whether we could simplify the template in this plugin class. Currently it contains code to create markers and popups. But we already have the Marker and Popup class. Ideally, we'd reuse more of those! That way, all the existing stuff around markers like tooltips, colors and icons works as well. It doesn't right now.

Suggestion, and I don't know if this is possible or reasonable: create Marker and Popup as usual, add them to a map or feature group as usual, and let the OMS class only handle the spiderfying and the click events. So in the OMS template overwrite the existing popup click events and bind them to the oms.

In short:

• in the current implementation, should markers be added to the map? I guess no, so documentation should be clear on that limitation.
• Or, can we take the marker and popup creation out of this plugin and use the existing marker and popup classes?
• if so, should this plugin actually be a Layer? Or should it just be a collector of existing markers?

Apologies if that's a bit much. I'd love to hear what you think about this. And let me know if I understood it correctly! Feel free to be open about what kind of changes you feel okay with.

[swtormy]

Thanks so much for the thoughtful review, @Conengmo! Your comments are incredibly valuable, and you've picked up on several important points that I missed - both in the code and in the documentation. Looks like my documentation decided to freelance its own narrative. Clearly, it needs a little more supervision. 😄

I completely agree that using the existing Marker and Popup classes is a much better approach. Additionally, your suggestion to use oms solely for managing the spiderfying and click events makes perfect sense.

Here's an overview of the changes:

• I changed the base class to MacroElement because the plugin no longer acts as a layer but rather as a utility to enhance existing markers on the map.

• I removed the logic of creating markers and popups from template. Now OverlappingMarkerSpiderfier only processes existing markers added to the map.

  {{ this._parent.get_name() }}.eachLayer(function(layer) {
      if (layer instanceof L.Marker) {
          oms.addMarker(layer);
      }
  });

• I have significantly simplified the template, but I had to keep the logic for closing popups in the spiderfy event. Without this, when clicking on a cluster of markers and triggering the spiderfy action, the popup of the first marker would open unintentionally.

  oms.addListener('spiderfy', function() {
                      {{ this._parent.get_name() }}.closePopup();
                  });

• I additionally decomposed the tests. In the previous version there were several checks in one test, now I have written a different checks for each test.

I've reworked the code to reflect this approach - please take a look at my latest implementation and let me know what you think.

[Conengmo]

Great work @swtormy, thanks so much for making these changes. I love how the code became simpler but everything, like the click events, still works.

I have basically two comments still. One about how we deal with class arguments in Folium that I should have mentioned the previous review, sorry about that. The other about a new missing use case I'd like to hear your opinion on.

I think with these out of the way we can get this merged!

[swtormy]

I decided to try a recursive approach to finding markers, but not in the js template. In doing so, I wanted to keep the approach of minimal user interaction with the plugin so that the user doesn't have to add each token separately.

I tried modifying add_to by collecting all markers in it.

def add_to(
        self, parent: Element, name: Optional[str] = None, index: Optional[int] = None
    ) -> Element:
      self._parent = parent
      self.markers = self._get_all_markers(parent)
      super().add_to(parent, name=name, index=index)

 def _get_all_markers(self, element: Element) -> list:
      markers = []
      for child in element._children.values():
          if isinstance(child, Marker):
              markers.append(child)
          elif hasattr(child, "_children"):
              markers.extend(self._get_all_markers(child))
      return markers

In recursion I am looking for markers within all children, it seems that this approach should help to find markers not only in FeatureGroup but also in FeatureGroupSubGroup.

I completely agree that adding more functionality increases the code and therefore the possibility of getting bugs. Therefore, if you still think that there is no need to complicate things here, I will gladly remove the recursive traversal and leave only manual addition of markers.

[Conengmo]

Lovely! This works great, good idea to do it this way. I have one tiny comment about a default value in the docstring that I'll merge myself, after that if there's nothing more from your end I'll go ahead and merge this one!

[swtormy]

@hansthen, @Conengmo thank you for reviewing and approving the changes! 👍

I’ve tested the plugin locally, including Selenium tests, and everything passed successfully. However, if I understand correctly, there is an issue with the Selenium environment for python3.9 and 3.13 in the CI pipeline that doesn't bother with the plugin. Maybe, it might be resolved by simply restarting the failing runner? Or is the problem in my code?

[Conengmo]

I’ve restarted the tests but they still fail… Ill take a closer look tomorrow.

[swtormy]

I’ve restarted the tests but they still fail… Ill take a closer look tomorrow.

Ok, I tried to get Micromamba up on my windows, with an Ubuntu container to run selenium tests on python 3.9 and 3.13. It took longer than I thought. I'll take a closer look tomorrow as well.

I want to try to rule out my tests first, something tells me that it might be because of them, even though there is no clear indication of them in the logs.

[hansthen]

I also have problems running the tests. It would be cool if there were some instructions on how to setup the tests to run locally as part of the developer instructions.

[Conengmo]

@hansthen theres this list in the contributors guide: https://github.com/python-visualization/folium/blob/main/.github/CONTRIBUTING.md#contributing-code. Is there anything missing there? If I’m not mistaken Selenium downloads drivers itself these days.

[swtormy]

@hansthen theres this list in the contributors guide: https://github.com/python-visualization/folium/blob/main/.github/CONTRIBUTING.md#contributing-code. Is there anything missing there? If I’m not mistaken Selenium downloads drivers itself these days.

Perhaps @hansthen means that this instruction doesn't have information on how to model python 3.9 and 3.13 in ubuntu if the user is using other OS.

Perhaps I'm missing something, but this instruction describes the process in which the selenium driver is loaded for the OS the user is running on. In my case it is windows and macos. But on github the test actions are performed in ubuntu.

I haven't yet had time to recreate the conditions from Selenium Tests locally, it took longer than I thought.

[Conengmo]

I reran the same test on the main branch and it fails there as well, so it's not because of your change: https://github.com/python-visualization/folium/actions/runs/11880066659/job/33603469550

[Conengmo]

Perhaps @hansthen means that this instruction doesn't have information on how to model python 3.9 and 3.13 in ubuntu if the user is using other OS.

That makes sense, but that goes a bit to far for a contributor guide IMO to go into containerizing different OS'es :)

[Conengmo]

I made a temporary fix for the Selenium test failure in #2034. Could one of you review it?

I'll go ahead and merge this PR since the test failure is unrelated.

[Conengmo]

Thanks for your excellent work on this @swtormy!