[Perf] Test hypothesis 3: Whisper network overhead

[oskarth]

Identified Behavior

@churik observed that Status takes up a lot of bandwidth when doing common operations, in total two orders of magnitude more than Telegram.

Specifically, creating an account takes ~30mb vs <1mb for Telegram.

• User story part of MVP scope
• Identified end user behavior

Hypothesis

Whisper overhead is cause of x30+ data for creating an account when using Ropsten RPC.

(Hypothesis 2: Blockchain is still being synced when using Ropsten RPC.)

UPDATE 2018-01-20: Hypothesis 3: p2p layer is causing network overhead.

Assumptions

1. Assuming this data usage is repeatable and corresponds roughly 1-1 to specified action
2. Assuming this is a critical problem for end user
3. Assuming this is reasonably fixable (order of magnitude of TG)

Experiment

• Repeat action and confirm data usage looks the same
• Profile data usage and see where 80% of it comes from

Acceptance criteria

• Numbers on where majority of data usage comes from
• Creation of issue with proposal on fix to get within order of magnitude of TG
• Basic understanding of relation to findings in this issue and Chu's other findings

Notes

• [Perf] User story 2: Minimize data consumption  #2931

[oskarth]

@divan do you want to look into this? I know you mentioned it a bit.

[oskarth]

@annadanchenko pointed out it might also be a good idea to test most common user action, which is probably:

chatting for 1 min (emoji/text) | ~0,2 MB | ~3-10 MB | messages contains emojis, text, long messages pasted from clipboard

It'd also be OK to replace specific hypothesis with one related to this. The general idea is to use a specific scenario and spike test, as there's likely to be overlap in solution. If there isn't it'd be good to have a hunch as to why for future experiments.

[divan]

@oskarth I'm working on the code that will allow testing this hypothesis. If I'm going to do the verification myself — cannot answer right now.

[oskarth]

@divan Gotcha, thanks. Probably worth checking with @JekaMas so there is no overlap in effort, he is doing something similar I believe.

[JekaMas]

@divan @oskarth I've done with whisper network overhead. I've tested network overhead of clean geth without status app to get a bias.

Test env: Geth/v1.8.0-unstable-3f40b22d/linux-amd64/go1.9.2
Metodology: each case ran for 10 minutes on two independent nodes at the same time
Script: https://gist.github.com/JekaMas/bd726dbc563864998d1df191b6976e1e

IDLE

With whisper

Stats (bytes/s):

	Receive	Transmit	Total
mean	370813	370085	740899
std	218636	211305	429757
min	506	132	638
25%	179029	186036	365065
50%	374748	377393	752141
75%	552994	550450	1103444
max	799617	739831	1539448

Traffic sum(bytes):

Receive	222487946
Transmit	222051570
Total	444539516

Without whisper

Stats (bytes/s):

	Receive	Transmit	Total
mean	231629	354676	586305
std	128071	201885	329736
min	506	3226	3732
25%	120942	179424	300366
50%	250381	364511	614892
75%	334190	525168	859358
max	473914	706034	1179948

Traffic sum(bytes):

Receive	138977652
Transmit	212805692
Total	351783344

Create account

With whisper

Stats (bytes/s):

	Receive	Transmit	Total
mean	27126	39463	66589
std	19041	22432	41296
min	506	3226	3732
25%	8793	19411	28205
50%	27531	40407	67938
75%	36440	56470	92911
max	69192	80766	149958

Traffic sum(bytes):

Receive	1627560
Transmit	2367788
Total	3995348

Without whisper

Stats (bytes/s):

	Receive	Transmit	Total
mean	25840	38924	64765
std	17559	24042	41487
min	506	330	836
25%	9923	18095	28019
50%	24173	37437	61610
75%	34852	56276	91128
max	61951	82216	144167

Traffic sum(bytes):

Receive	1550436
Transmit	2335474
Total	3885910

1-to-1 chat

With whisper

Stats (bytes/s):

	Receive	Transmit	Total
mean	419388	386979	806367
std	276086	245251	521201
min	426	3652	4078
25%	157804	165446	323250
50%	408736	374733	783469
75%	638422	594332	1232754
max	907451	819841	1727292

Traffic sum(bytes):

Receive	251633287
Transmit	232187419
Total	483820706

Without whisper

Stats (bytes/s):

	Receive	Transmit	Total
mean	268071	364106	632177
std	201343	235094	436076
min	426	330	756
25%	85817	150765	236583
50%	244263	348307	592570
75%	422895	561594	984490
max	639356	781904	1421260

Traffic sum(bytes):

Receive	160843101
Transmit	218463610
Total	379306711

Network overhead

	Whisper	W/o Whisper	Diff
IDLE	444539516	351783344	+20.87%
Create account	3995348	3885910	+2.74%
1-to-1 chat	483820706	379306711	+21.6%

Result: Whisper adds 20% of traffic in idle and chatting however this overhead relates to geth themself and we can't quickly fix this issue. So i suggest to monitor this metrics from time to time and consentrace on performance issues.

[oskarth]

@JekaMas Would you mind making the main metrics on the same format as @churik's investigation? So it is more clear what the difference is in terms of magnitude.

[feuGeneA]

@oskarth , I took a quick stab at trying to correlate @JekaMas ' results above with @churik 's results in #2931 . In short, I didn't find much of a correlation.

I made the following assumptions:

• churik had whisper enabled, implying her data should be compared to Jeka's "With whisper" data.
• churik's data consumption numbers are a sum of both receive and transmit totals.
• churik's "chatting for 1 min" observation was in a 1:1 chat scenario.

The 1:1 chat action is the most relatable. churik observed between 3 and 10 MB of traffic for 1 minute of chatting, which equates to between 52,429 and 174,763 bytes per second. Compare this to Jeka's mean bytes per second of 806,637. The vast difference in those measurements may be due to the volume of chat messages being transmitted (churik said "messages contains emojis, text, long messages pasted from clipboard" ... not sure what sort of data Jeka was using). Or it may be that churik's sample size (1 minute) was not enough to infer a realistic mean (compared to Jeka's 10 minutes). Or, of course, it may be something else entirely.

As for account creation, Jeka observed a total of 3 MB bytes of data consumption (reported as 3,995,348 bytes), while churik observed 30 MB. Of potential importance here is Jarrad's comment suggesting that the data flows for this action are for blockchain synchronization, not whisper overhead.

And as for idle time, it's not clear whether the numbers are comparable, as churik's data was recorded while the app was in the background, and Jeka's was (presumably) recorded while idle in foreground. In any case, churik observed between 0.01 and 0.1 MB/s, and Jeka observed an average of 0.7 MB (reported as 740,899 bytes) per second.

What do these imply about the next steps for the Experiment at hand?

[dshulyak]

How is it relevant to measure network traffic when account is being created?
As far as I understand it won't have any impact on it. Neither on whisper, nor on les/eth. This is just a coincidence that when the account is being created new blocks or envelopes were received by a node.

[oskarth]

@JekaMas Who is currently owning this issue? If it is deemed to no longer be relevantly / badly specified, where is the continuation?

[ghost]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ghost]

This issue has been automatically closed. Please re-open if this issue is important to you.