/explore: Available data is misleading based on zoom level

[pdille]

I pulled up some channels to see when data was available and the way the bar graphs are drawn depending upon zoom level is quite misleading. In the first image you'll see that it appears there is data available up to and a little past 6am on the 8th. But then when you zoom in, you see that there is not data for the 8th and only up to about 22:00 on the 7th.

In my personal opinion, I find these new bars very confusing and very much preferred the line charts of before.

[dognotdog]

Getting the bars right and consistent is, apparently, hard.

/browse looked like so:

https://esdr.cmucreatelab.org/browse/#channels=26151.tVOC,,30.PM25_UG_M3,&time=1606975659.5904672,1606978274.156305&zoom=3&center=49.35720091782239,-92.91969375610346&cursor=1607105203.169&search=pm2

[chrisbartley]

Possible solutions (not intended to be mutually exclusive...an ideal implementation would employ them all):

1. For sensors where the sample rate is fixed, make use of a feed's channelSpecs field, which stores (or could store) the sample interval. E.g., we know Airnow's sample interval is 1 hour, there's also some fixed interval at which we're pulling PurpleAir data, etc. If the sample interval isn't in the feed's channelSpecs already, it wouldn't be hard to insert sample intervals into the majority of feeds people care about (EPA, PurpleAir, etc).
2. As a fallback for the above, the grapher could try to "sense" the sample interval by just looking at the gaps between samples. This would also be a reasonable solution for devices where sample rate isn't fixed (e.g. a Speck's sample interval is user configurable).
3. As a last resort, decide that bars can never be wider than N seconds. E.g perhaps 3600s would be a reasonable start, even for sensors that sample at a faster rate. We could just decide that, after N/2 seconds (assuming bars are centered on the sample timestamp), a sample is "stale" and the value should no longer be considered accurate.

[rsargent]

Bumping this. We keep getting complaints that we're misrepresenting RAMP data from Albert.

[dognotdog]

@rsargent @chrisbartley as far as I can tell, this is a limitation of using tiles, as at some increased zoom tiles simply don't contain many samples any more, and the grapher actually tries to widen the bars, to fill the tile, instead of showing a single blip. Eventually, it'll just show nothing when the zoom level is high enough and the tiles are all empty. Limiting zoom isn't simple when some channels only have one sample every 24h.

What do you suggest should happen in these cases? I imagine if we get empty tiles, we could fall back to the raw data API, though there is no allowance for that, currently, and going further, when no samples exist within the shown window, what would be the mechanism to get the nearest samples outside the window?

[rsargent]

If it's difficult or impossible to reliably determine a bar width, let's go back to the previous style of display -- lines between dots, plus optional single-pixel-wide bars.

…

On Mon, Oct 11, 2021 at 1:38 PM Dömötör Gulyás ***@***.***> wrote: @rsargent <https://github.com/rsargent> @chrisbartley <https://github.com/chrisbartley> as far as I can tell, this is a limitation of using tiles, as at some increased zoom tiles simply don't contain many samples any more, and the grapher actually tries to widen the bars, to fill the tile, instead of showing a single blip. Eventually, it'll just show nothing when the zoom level is high enough and the tiles are all empty. Limiting zoom isn't simple when some channels only have one sample every 24h. What do you suggest should happen in these cases? I imagine if we get empty tiles, we could fall back to the raw data API, though there is no allowance for that, currently, and going further, when no samples exist within the shown window, what would be the mechanism to get the nearest samples outside the window? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#24 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACPJ46MR3LQ43LNOJ3ZARTUGMOJXANCNFSM4X7TRG5A> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[dognotdog]

That does bring up the whole UI question again... it is possible to switch to lines instead of bars, but that option is currently hidden in the interface, see #15 and this view.

In either case it doesn't solve the problem of zooming in too far eventually resulting in zero samples being visible.

[rsargent]

Cool. Let's then switch to lines and not give the option for bars, since a number of people feel it's often misleading. In my belief, that will address the problem as reported. Afterwards, if you feel there's a problem about zooming in too far until zero samples are visible, report it as a separate issue with screenshot so we can look together and triage.

…

On Mon, Oct 11, 2021 at 2:34 PM Dömötör Gulyás ***@***.***> wrote: That does bring up the whole UI question again... it is possible to switch to lines instead of bars, but that option is currently hidden in the interface, see #15 <#15> and this view <https://environmentaldata.org/#channels=26151.tVOC,30.PM25_UG_M3,3841.AVG_PM2_5_24HR_UG_M3_AQI&time=1563759254.639,1609616649.387&plotHeight=5.000&plotAreaHeight=40.000&showSparklines=true&showFilters=true&showSettings=true&showExperimentalSettings=true&showResults=true&center=40.34094297768818,-79.89267890799294&zoom=12> . In either case it doesn't solve the problem of zooming in too far eventually resulting in zero samples being visible. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#24 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACPJ44RDRICCSKJJ5NV563UGMU4TANCNFSM4X7TRG5A> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.