Add ForkDigest Filter
#206
Conversation
| func (s mongoStore) GetHistory(ctx context.Context, start int64, end int64, peerFilter *model.PeerFilter) ([]*models.HistoryCount, error) { | ||
| var filter primitive.D | ||
| if peerFilter != nil && | ||
| peerFilter.ForkDigest != nil { | ||
| forkDigest := *(peerFilter.ForkDigest) | ||
| if forkDigest[0:2] == "0x" { | ||
| forkDigest = forkDigest[2:] | ||
| } | ||
| forkDigestBytes, err := hex.DecodeString(forkDigest) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| filter = bson.D{ | ||
| { | ||
| Key: "$match", Value: bson.D{{Key: "fork_digest", Value: forkDigestBytes}}, | ||
| }, | ||
| { | ||
| Key: "$and", Value: bson.A{ | ||
| bson.D{{Key: "time", Value: bson.D{{Key: "$gt", Value: start}}}}, | ||
| bson.D{{Key: "time", Value: bson.D{{Key: "$lt", Value: end}}}}, | ||
| }, | ||
| }, | ||
| } | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| } else { | ||
| filter = bson.D{ | ||
| { | ||
| Key: "$and", Value: bson.A{ | ||
| bson.D{{Key: "time", Value: bson.D{{Key: "$gt", Value: start}}}}, | ||
| bson.D{{Key: "time", Value: bson.D{{Key: "$lt", Value: end}}}}, | ||
| }, | ||
| }, | ||
| } | ||
| } | ||
|
|
||
| cursor, err := s.coll.Find(ctx, filter) |
There was a problem hiding this comment.
I'm not sure if there's a better way to write this given that the Find on line 106
(cursor, err := s.coll.Find(ctx, filter)) expects filter to be of type bson.D
| aggregateByClientVersion(peerFilter: PeerFilter): [ClientVersionAggregation!]! | ||
| getHeatmapData(peerFilter: PeerFilter): [HeatmapData!]! | ||
| getNodeStats(peerFilter: PeerFilter): NodeStats! | ||
| getNodeStatsOverTime(start: Float!, end: Float!, peerFilter: PeerFilter): [NodeStatsOverTime!]! |
There was a problem hiding this comment.
I've tested all the other methods using peerFilter locally, but getNodeStatsOverTime always returned:
{
"data": {
"getNodeStatsOverTime": []
}
}before and after the peerFilter additions, so I'm not 100% sure this is working as expected
There was a problem hiding this comment.
@spacesailor24, That's totally fine. getNodeStatsOverTime returns the count of nodes each day from the History table. A scheduler pushes data to the history table each day. So, ideally, you won't see anything with that query unless you run the crawler for at least a day. Change this to second and you will see data.
nodewatch-api/crawler/crawl/service.go
Line 76 in 3f407e6
Also, I have tested the APIs. Working fine with or without peerFilter.
So, with this implementation, if no filter is provided it returns all data.
That looks good to me. UI can filter based on the mainnet.
store/peerstore/mongo/mongo.go
Outdated
| } | ||
|
|
||
| var err error | ||
| if peerFilter != nil && |
There was a problem hiding this comment.
Can we add the nil checking inside AddForkDigestFilterToQueryPipeline so we don't need to check it every places.
There was a problem hiding this comment.
@sadiq1971 Should AddForkDigestFilterToQueryPipeline error if peerFilter or peerFilter.forkDigest is nil?
Instead, I could refactor the method to AddPeerFilterToQueryPipeline and it would return an unmodified pipeline if peerFilter or peerFilter.forkDigest was nil or empty
This commit does the above
There was a problem hiding this comment.
Yes, that exactly what I said. Will just add to pipeline if not nil
Adds
peerFilter:to the following GraphQL APIs:
aggregateByAgentName(peerFilter: PeerFilter)aggregateByCountry(peerFilter: PeerFilter)aggregateByOperatingSystem(peerFilter: PeerFilter)aggregateByNetwork(peerFilter: PeerFilter)aggregateByHardforkSchedule(peerFilter: PeerFilter)aggregateByClientVersion(peerFilter: PeerFilter)getHeatmapData(peerFilter: PeerFilter)getNodeStats(peerFilter: PeerFilter)getNodeStatsOverTime(start: Float!, end: Float!, peerFilter: PeerFilter)getRegionalStats(peerFilter: PeerFilter)getAltairUpgradePercentage(peerFilter: PeerFilter)Example Usage
closes #205