Differences between Waze Data Feed and BigQuery

[g4brielvs]

Through the Development Data Partnership, I’m supporting several projects leveraging Waze for Cities and few questions arose. Specifically, differences between information available via the Waze Data Feed and via BigQuery.

In particular, we are trying to understand how the “duration” of a jam could potentially be inferred. On jams, the delay column informs the delay of the jam (in seconds) compared to the free flow speed. In other words, Waze doesn’t seem to provide the duration, but it provides how much longer it would take for a user to drive through the segment.

On the other hand, if let’s say we retrieve from Waze Data Feed every minute and identify a segment for 10 minutes; would it be accurate to assume the approximate “duration” of the report to be 10 minutes?

As you say, there is no way to determine the END time of the jam from the current fields available in BQ and thus the duration of a jam is uncertain. The timestamp associated with the jam records is the start of the jam and it is the same timestamp for all of the jams with a common uuid, regardless of when they were recorded.

My team has used an indicator of jam duration by counting the number of records with the same uuid in the BQ jams table (i.e. higher jam count → longer lasting jam). In some cases, we have estimated the jam time as the count of records times 90 seconds, which we estimate to be about the average time between reports. However, this isn’t robust because some jam records are coming in more or less frequently (probably related to how many Waze users are in the jam). Based on a comparison we did of the BQ records to the 10 min feed that we are saving separately, the reports in BQ table seem to largely come in anywhere from 30 s to 2 min in our cities of interest. In our 10 min feed we have a timestamp for every time we save the feed, so we know about when the jam disappears. Using the feed, we get a duration to the nearest 10 minutes, just like the approach you describe. This will overestimate the short jams.

I have contacted the Waze engineering team about this (Andrew and Tommy) to see if they could add a field for the jam END time (or even just the report time) to the BQ table. Hopefully they can respond here!

Best, Lauren

Originally posted on Waze for Cities Discuss.