Initial loading of feeds requests more metadata than is necessary

[chrisbartley]

The getFeedsFromOffset() function has no filtering on the fields, so it ends up loading much more data than is necessary and will likely slow down initial page load since it currently has to make the request ~80 times.

The only query string param is the offset:

	getFeedsFromOffset(feedOffset, callback) {
	  let request = new XMLHttpRequest();
	  if (this.geographicBounds) {
	  	// if a geographicBounds is set, only get feeds in that area to speed up getting a list of sensors and more importantly their coordinates
	  	// this is for live data display in a fixed window, eg. plumeviz
	  	// https://esdr.cmucreatelab.org/api/v1/feeds?whereAnd=productId=69,latitude%3E=39.420978,latitude%3C=40.756547,longitude%3E=-81.451293,longitude%3C=-79.677010

	  	request.open('GET', `${this.apiUrl}/feeds?offset=${feedOffset}&whereAnd=latitude%3E=${this.geographicBounds.min.lat},latitude%3C=${this.geographicBounds.max.lat},longitude%3E=${this.geographicBounds.min.lng},longitude%3C=${this.geographicBounds.max.lng}`, true);

	  } else {
	  	request.open('GET', `${this.apiUrl}/feeds?offset=${feedOffset}`, true);

	  }

You should be able to speed up loading by using the fields param. For example:

https://esdr.cmucreatelab.org/api/v1/feeds?fields=id,name,latitude,longitude,channelBounds,lastUpload

There might be other fields you need, but the above should be a good start in order to filter on feeds which have data in the last 30 days, show all of a feed's channel names, etc.

You could also load most recently created feeds first, by sorting on ID in descending order:

https://esdr.cmucreatelab.org/api/v1/feeds?fields=id,name,latitude,longitude,channelBounds,lastUpload&orderBy=-id

We should also talk about other options to make initial page load more efficient such as:

1. Relaxing/upping ESDR's 1000-feed hard limit on requests
2. Client side caching, e.g. caching the metadata for the 80K+ feeds in the browser's local storage. On load, environmentaldata.org could first fetch any new feeds (i.e. IDs greater than the largest ID in the cache), and then fetch any with a lastModified date newer than the last visit to the site.