[Feature]: Visualizing light mechanics separately from color

[kwvanderlinde]

Feature Request

Lights (actual lights, and personal lights, but I'm not talking about auras here) are being pulled in a few different directions, especially:

1. As a way to represent / communicate game mechanics (e.g., dim light vs bright light)
2. As a part of the environment, aesthetically illuminating a map (so should blend nicely, etc)

The first was supported in 1.11.5 and earlier, as the colour of a light acted like a tag in addition to determining the colour drawn on the map. Lights of the same colour would have their lit areas merged together and wouldn't blend.

In 1.12.2 and later, the colour of a light only determines which colour is drawn on the map. Lights of the same colour would blend together rather than being merged together.

Trying to support both in one overlay, with both behaviours being based mainly on one property (color) is a bit much since they have inherently different characteristics. Additionally, it could be useful to have both environmental lighting that blend while also getting support for game mechanics.

The Solution you'd like

I would like to split the current light overlay into two separate overlays.

1. The first overlay (the "light overlay") would render light as part of the environment, i.e., by drawing the light's colour in an area and by blending lights together. This way, lights can be added to the map and add some aesthetic quality (especially if we also improve how lights are rendered!).
2. The second overlay (the "lumens overlay") would show a sort of contour map / elevation map that communications the strongest lumens level at each point on the map. The higher the lumens value, the brighter the colour that is drawn to the map, and darkness would be drawn in black. This would look much like how the default 5e lights look in 1.11.5 and earlier, just merging by lumens instead of merging by colour.

Each overlay could be enabled/disabled independently. Those who just want the mechanics given by lumens can use the lumens overlay, those who just want the ambience of lighting can use the light overlay. And they can be comboed, e.g., by temporarily enabling the lumens overlay to check the mechanical situation before going back to just lighting.

On top of that, lumens would be defined for each range of a light source, instead of having one lumens value for the whole thing. This would not only allow more direct modeling of game mechanics, it would also serve as a replacement for the ability to set one colour (or no colour) for the "bright range" of a light and another colour for the "dim range".

The default lights provided by MapTool should also be updated so that they can make use of the new features. E.g., the 5e lights would set high and low lumens values for the lights instead of setting colours for that purpose.

Alternatives that you've considered.

• Reverting to 1.11.5 behaviour to allow users of the old lighting system to upgrade.
• Defining new types of lights that individually have the described behaviours. Environmental lights would be rendered to the map like "real" lights, with additive blending and no colour merging. Mechanical light would be merged according to colour and then rendered overtop of the map like lights in 1.11.5 and earlier.
• Have each light control how it blends. If not configured to blend, a light acts like a 1.11.5 light and merges with other non-blending lights according to colour. If configured to blend, will act like a "real" light and blend addtitively with other blending lights. regardless of colour. Or, to go the extra mile, allow blending lights to even control the effects used to blend them together.

Additional Context

This feature request deliberately intersects with a bunch of other issues.

• It could be the means to addressing [Feature]: Improve sight rendering when map vision is set to Day or Off #3748 and [Bug]: Multiple light sources actually darken each other due to their dim light range when the lumens should be, instead, added to overlapping light or at least not darken. #3755 for those that only need a dim/bright distinction or similar gradation of brightness.
• It could also open the door to restoring additive blending after [Bug]: Performance regression for lights/sights performance on 1.12.0 #3651. An optional lighting feature with borderline performance is better than it being the only option when showing lights. Those willing to pay the price can do so.
• It could open a door to solving Support for DND 5e additive darkvision #1895, though via a lumens-boosting approach instead of a colour-boosting approach.
• Not directly so, but it can enable features like Aesthetic Light Options - Gradients/Fades, Blending modes #1908 as well. If the light overlay isn't tied to mechanics, but only visual representation, we can be a bit more free in experimenting with alternative lighting options.

It should also be noted that there are a gagillion other ideas that people have for where lights can go. E.g., user-defined light types that can group lights behaviourly. E.g., having overlapping lumens regions add their lumens together instead of merging. My hope is that this feature (or whatever we hash out in the end) can serve as a baseline that frees lighting up a little to be able to support the various use cases folks have.

And regarding auras, I left them out of here because they don't have anything to do with lights except that we use some of the same UI for the two. There's no lumens attached to auras (at least, they don't do anything), they don't interact with lights or darkness nor do they affect sight, and it's not clear that they should blend like lights. Basically, they don't mean anything, so they can be used for all sorts of unrelated purposes, like a spell's AoE, targetting ranges, field of view, measurements, states, you name it. So let's leave auras be in the context of this feature request.

---

I made a proof of concept a while ago which shows roughly what I'm thinking in terms of the new overlays. The details aren't right (look how dark the light overlay makes the map 😂) but the general idea is there.

First up, this is what lights could look like, if they had a gradient falloff and were allowed to blend nicely:

And here is what the new lumens overlay could look like, using lumens=100 for bright light and lumens=50 for dim light:

And the combo version:

[Azhrei]

I love the lighting features you've been working on — full steam ahead! 😉 (Nothing constructive to add here, just emotional support!)

[kwvanderlinde]

For the lumens overlay, there's some details in the concept that still need to be figured out.

First of all, how do we deal with the fact that different folks can use different lumens scales. I.e., one group may only need lumens up to 100, while others might use a vastly greater range of values if it's useful. So how dark do we make the lumens overlay for a given value? I.e., what shade of grey should we use for a lumens of, say, 50? How about 100? Or 1,000?

There's also the question of daylight. It's currently modeled as an implicit light source with lumens=1, which was a design choice so that any existing darkness wouldn't be trumped by the new daylight. But the above scheme would make it so that such daylight would be represented by a very dark color in the lumens overlay, which is probably not what most people would expect or want to see.

One idea to address both of these points is to introduce a configurable daylight lumens level, either per campaign or per map. That way daylight can be set much brighter than 1 if desired and wouldn't be locked to such a faint value. Then we can also use that daylight level as a threshold for bright light. Any lumens at least as high as the daylight level can be rendered as either transparent or very faint white in the lumens overlay. Any lumens less than the daylight level would have a shade of grey proportional to how bright it is compared to daylight.

[Ahnoold]

For the issue of how strongly to shade the lumens overlay, could a user modifiable setting be added to indicate the maximum lumens level that they intend to use? Perhaps defaulting to 100? MapTool could then divide up the range according to however many discrete shades of grey are being used, and automatically assign each shade according to the calculated divisions. For example, let's say 5 shades of grey are being used. At a default maximum lumens of 100, grey shade 1 would be assigned to lumens in the range of 1 to 20, shade 2 to the range of 21 to 40, etc. If the maximum lumens setting is changed to 1,000, then grey shade 1 is assigned to the lumens range of 1 to 200, shade 2 to the range of 201 to 400, etc. It might even be possible to not require users to specifically enter the maximum lumens value, but rather simply base it off the maximum lumens of all the lights defined for the campaign. If you wanted to get even fancier (and more complicated), the number of discrete shades of grey could also be altered according the range of lumens being used (fewer shades for smaller lumens ranges and vice versa).

As for daylight, if its value is allowed to be changed, darkness goes back to not working when its abs(lumens) is less than what daylight lumens is set to. Is that desired behavior? Instead, what if the lumens overlay simply isn't applied to the ambient daylight at all? Also, on maps with Vision set to Day, the lumens overlay of lights are only applied where they overlap areas of darkness, and are ignored where lights overlap daylight without any overlapping darkness. Colors of lights would still be shown, but the lumens shading depicting the extent of the area they reveal are usually irrelevant anyway unless overlapping with darkness?

[thelsing]

How about making lumens a double value and fix max and daylight to 100.0. With floating point values people can still use as much levels as they want without having yet another config value.

[kwvanderlinde]

For the issue of how strongly to shade the lumens overlay, could a user modifiable setting be added to indicate the maximum lumens level that they intend to use? Perhaps defaulting to 100? MapTool could then divide up the range according to however many discrete shades of grey are being used, and automatically assign each shade according to the calculated divisions. For example, let's say 5 shades of grey are being used. At a default maximum lumens of 100, grey shade 1 would be assigned to lumens in the range of 1 to 20, shade 2 to the range of 21 to 40, etc. If the maximum lumens setting is changed to 1,000, then grey shade 1 is assigned to the lumens range of 1 to 200, shade 2 to the range of 201 to 400, etc. It might even be possible to not require users to specifically enter the maximum lumens value, but rather simply base it off the maximum lumens of all the lights defined for the campaign. If you wanted to get even fancier (and more complicated), the number of discrete shades of grey could also be altered according the range of lumens being used (fewer shades for smaller lumens ranges and vice versa).

This illustrates a really good point that not every lumens value necessarily needs to be displayed separately. E.g., as a D&D 5e player, what I basically care about is the categories of dark, dim and bright. But the lumens values can be more fine-grained than that, e.g., to establish priority between different lights and darknesses. So I think it is desirable to have some sort of ranges like you describe, and I expect it would need to be configurable to account for different systems. Going on your idea, it could be a simple string that determines things. E.g., this would be sufficient to define the three 5e categories where <=0 is darkness, 0-99 is dim, >=100 is bright:

#00 0#77 100#ff

syntax is just to show that it can be simple. this isn't meant as a proposal

As for daylight, if its value is allowed to be changed, darkness goes back to not working when its abs(lumens) is less than what daylight lumens is set to. Is that desired behavior?

There is a difference between foisting it on a user in an update vs letting the user choose to brighten their daylight.

Instead, what if the lumens overlay simply isn't applied to the ambient daylight at all? Also, on maps with Vision set to Day, the lumens overlay of lights are only applied where they overlap areas of darkness, and are ignored where lights overlap daylight without any overlapping darkness. Colors of lights would still be shown, but the lumens shading depicting the extent of the area they reveal are usually irrelevant anyway unless overlapping with darkness?

I don't have a coherent response at the moment, but I'm thinking about these points too 🙂

[kwvanderlinde]

How about making lumens a double value and fix max and daylight to 100.0. With floating point values people can still use as much levels as they want without having yet another config value.

In theory I am a fan of this idea, generally in favour of trying to be more specific about what things mean. But how do we treat a campaign that currently use "excessive" lumens values beyond 100, where users might care about the distinction between, say, 200 and 1000?

I also wonder what you think of the idea that different lumens values could belong to the same game-mechanical category? E.g., 50, 51, 52 might all be "dim light", where the little differences just establish a priority. If that is actually desirable, we would probably be looking at it needing a configuration as well? Maybe we could associate categories with integers, and let the fractional portion be used just for priority?

---

On the topic of lumens=100, it looks like kind of a historical accident that lights use that value by default. I haven't tried to look into the history yet, but just from reading the related code it seems like it was first used for personal lights (which would otherwise have lumens=0), and naturally became the default for lights as a result of how that was implemented. So maybe we should feel more free to pick other thresholds for "bright" that could give more breathing room, even if we stick with integers? ... I don't know, just throwing it out there, I don't think much would actually be gained trying to change any of that.

[Ahnoold]

Could it be worth the extra effort to further separate brightness (lumens) from ranking (which light's lumens is displayed where multiple overlap) when rendering the lumens overlay? Maybe ranking defaults to being equal to lumens, but can be changed if desired by the user. This could be another way to address the daylight issue by defaulting daylight to maximum lumens and minimum ranking, thus retaining brightly rendered daylight while still allowing darkness (lights currently defined with negative lumens) to function on maps with Vision set to Day. This would also allow a dimmer light to take precedence over a brighter light if other situations warrant it.

[kwvanderlinde]

That ranking is exactly what lumens already are. So what you're describing would have ranking replace lumens, then lumens would become only a rendering hint. So I would turn the phrasing around: leave lumens alone, and introduce some other attribute that indicates a conceptual brightness level or category.

But this is going beyond the scope of this FR. There are many different ways we could have different lights interacting and overriding one another, and even other ways lights might be usefully modify by the environment, and that all deserves its own conversation. In this context of this FR, I think we should aim for exposing what we already have, and only offering tweaks to the existing system as needed to support it (e.g., the proposed per-range lumens values).

---

This part is where I try to get the FR back on track after I derailed it 🙂

For daylight, the only reason I even started thinking about it is because the lumens overlay will suddenly make it very obvious that it doesn't necessarily align with mechanical expectations. And then I also had the problem of how to pick lumens colours and the two seemed to fit. But I now think that was mistaken and I should make a separate FR for changes to daylight. And then - assuming we get a lumens overlay prior to any changes to daylight - we should just go with the option to exclude it from the lumens overlay as long as that is feasible. The main point of the overlay is to communicate game mechanics, so until daylight models game mechanics it makes sense not to include it.

Now how do we pick colours for the lumens overlay? The most straightforward is to just pick min/max values. Going off thelsing's idea of fixing the max to 100 lumens, we can just clamp any lumens to the range [0, 100], then use that to linearly (or otherwise) interpolate from black to white, possibly adjusting opacity as well. We can always expand this in the future to either make rendering more configurable, or to switch to floating point lumens, or clump lumens together into categories, or whatever. But I don't think there is a downside to just picking 100 to start and otherwise leaving the lumens system be for now, and leaving them for the future means there can be dedicate discussion for the pros/cons of each. Setting the max to 100 also works well for the current default lights provided by MapTool as they all default to 100 lumens anyways.

Let me know what you think. Not trying to discount any ideas that have been brought up, just hoping to refocus this FR.

[Ahnoold]

Establishing the limits of what to work on now vs. what to leave for later makes sense. My comments were offered for consideration in case they might help, nothing more. I think that what you're proposing will be a big improvement on what is in the program now.

The only risk I see for clamping lumens to a 0 to 100 range (or -100 to 100?) is that there could be existing campaigns that have defined lights outside that range. Not sure how many people there might be out there that would be affected. I'm sure that backwards compatibility is always a consideration for any change made to the program, though.

Something else that I think will influence colour selection for the lumens overlay is how many different levels of lumens are going to be visually distinguished from each other. As you have pointed out, D&D and Pathfinder games really only care about three levels: bright, dim, and dark. People playing other systems may want more, however. 5? 10? Don't want too many or it will likely lose usefulness as people can't actually distinguish between them. (Is that a lumens 42 or a lumens 43? I can't tell unless I see them next to each other.)

Another consideration is whether the lumens overlay needs to be visually different from how soft fog is rendered, and if so, how to keep it looking different?

I think that ultimately, you're just going to have to select the colour(s) etc. by picking something, seeing how it looks, and then tweaking from there.

[kwvanderlinde]

The only risk I see for clamping lumens to a 0 to 100 range (or -100 to 100?) is that there could be existing campaigns that have defined lights outside that range. Not sure how many people there might be out there that would be affected. I'm sure that backwards compatibility is always a consideration for any change made to the program, though.

BC is definitely is a consideration. So this clamping would only apply during rendering, and wouldn't mess with the actual lumens defined for a light. That way, light/dark areas still play off each other as they do today, and would leave the door open for any future improvements.

Something else that I think will influence colour selection for the lumens overlay is how many different levels of lumens are going to be visually distinguished from each other. As you have pointed out, D&D and Pathfinder games really only care about three levels: bright, dim, and dark. People playing other systems may want more, however. 5? 10? Don't want too many or it will likely lose usefulness as people can't actually distinguish between them. (Is that a lumens 42 or a lumens 43? I can't tell unless I see them next to each other.)

Yep. This lumens overlay is most compelling when few lumens levels are in play. I'd love to hear from someone who might actually want to use it with many levels, but I expect that in such a case it would only be used for a general sense of the situation rather than nitpicky differences between lumens levels. At least until (if) we had more fun supportive feature to it 😄 But that would all need to depend on some actual use cases.

Another consideration is whether the lumens overlay needs to be visually different from how soft fog is rendered, and if so, how to keep it looking different?

Very good point. In my tinkerings I haven't noticed this as something that matters, but there's quite a difference between a dev focused on implementing/testing something, and someone who's actually trying to run a game. If it is an issue, I'm open to suggestions about how we might keep them visually distinct. Some thoughts off the top of my head:

• Don't use plain grey for the lumens overaly; instead use shades of a different colour.
• Give the lumens overlay a texture.
• Make the lumens overlay borders something different to distinguish those areas.
• Changing how soft FoW looks instead of the lumens overlay? That textured options look kind of interesting for FoW ...

[FullBleed]

On the idea of giving the lumens overlay a texture... I'd rather see softfog get a texture. I think it would actually be nice if softfog also desaturated an area, though I'm sure that there would be a lot of maps where that effect alone would not be all that noticeable.

[Ahnoold]

I believe that it would be desirable to clearly differentiate between an area that is dim/dark vs. an area a token can't see into, especially for tokens that have Darkvision. Since users can already change the color of FoW in each map's settings, I think applying a texture to at least one of the overlays is almost necessary since no matter what color would be selected for the lumens overlay, it's possible that the FoW could be set to the same color. I would also prefer to see the soft FoW have the texture applied to it. Since that is already an available option in the map settings, perhaps setting a default texture for it wouldn't take too much effort?

[kwvanderlinde]

I just put up PR #3837, aiming to be as minimal as possible while getting the main benefits of the lumens overlay. I.e.,:

• Lumens overlay + View menu toggle
• Environmental lighting + View menu toggle
• New light syntax for per-range lumens

I deliberately left out:

• Textured light
• Soft FoW changes
• Hot key for temporarily showing the lumens overlay

All these (and more!) would be good additions, but can be tackled as improvements after the main split between lighting and lumens is done.

[Phergus]

Do you expect colored lights to be rendered as white on white background?

[kwvanderlinde]

Yes, it's a casualty of the new soft-style lighting. If it's considered undesirable, we can look at other blending functions, but most natural looking ones would have that same property. There is also way to do something that is partial blending function + partial alpha composite that I think would keep some color, but the reuslts may not be as nice for map with images.

[Phergus]

I expected that was the case. Not really seeing it as a much of an issue as I don't expect there will be a lot of folks using white maps with colored lights. Except maybe for winter scenes. Hmm.

[kwvanderlinde]

I plan to add a couple preferences yet, especially for controlling whether the lumens overlay or environmental lights are shown by default.

[adventuremagic123]

I expected that was the case. Not really seeing it as a much of an issue as I don't expect there will be a lot of folks using white maps with colored lights. Except maybe for winter scenes. Hmm.

A lot of RPG material comes with black & white maps. On those maps, people will want to use different colors for different light sources -- a torch might look different from a lamp, for example. I use different colors also to differentiate between dark vision and normal vision, as another example.

I actually have decided to redo all my B&W maps with Dungeondraft to make them more interesting for my players -- but it's best to allow people to play with B&W maps if they don't have the time to do that.

[adventuremagic123]

BTW, great thoughts on all this stuff. I can't wait to see the changes in MapTool. Very excited.

[adventuremagic123]

One more thought. I see lots of good options here. I'd love to use gradients but others might not. Maybe, make their use configurable?

If there's ever a situation where some people might want it one way while others prefer another, always provide configurable options ... if you can.

[kwvanderlinde]

A lot of RPG material comes with black & white maps. On those maps, people will want to use different colors for different light sources -- a torch might look different from a lamp, for example. I use different colors also to differentiate between dark vision and normal vision, as another example.

Hmm, yeah that's a good point. I can try avoid the white-pinning, but I doubt there's a good solution for both image-based maps and solid-colour maps. Basically I expect that alpha-compositing would still be a good fit on more plain maps like the B&W maps, while the blending like in the beta is better for fleshed out environments. So perhaps a per-map configuration to inform which style to use?

One more thought. I see lots of good options here. I'd love to use gradients but others might not. Maybe, make their use configurable?

There's some talk of that over in #1908. Love to hear some thoughts on the idea I presented at the end there.

[adventuremagic123]

Yes, seems like there should be a per-map property (configuration option) that defaults to color map but can be changed to black & white map if needed.

[kwvanderlinde]

A couple more issues now that I'm looking at this again:

1. Darkness isn't being coloured for GMs anymore.
2. The defaults for whether to show the lumens overlay and lights is backwards of what it probably should be. I.e., it makes more sense for an upgrading user to expect their lights to be rendered in colour without having to first discover the new option to enable it. Meanwhile, the lumens overlay is new and not yet depended on, so it could be left disabled by default.

[kwvanderlinde]

I also need to update the help text for the new lighting syntax.

[kwvanderlinde]

I'll close this off as it's been through an entire release cycle without too many complaints 😅. Any problems can be addressed in new bug reports or FRs.