Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[leo_gateway][nfs] The total file size can be wrong when writing the same file in parallel #596

Closed
mocchira opened this issue Feb 2, 2017 · 0 comments

Comments

@mocchira
Copy link
Member

mocchira commented Feb 2, 2017

How to reproduce

There is a case that writing the same file in parallel may happen.
(Implicitly caused by NFS client at least on linux)
We still don't know the exact conditions,
However from what we observed log files given from a user facing this issue,
Writing the same file in parallel may happen in case

  • the target file size is relatively large (around 300MB
  • the bandwidth between a NFS client and leo_gateway is relatively narrow (across the web
  • on ubuntu14.04LTS with LeoFS1.3.1

How it works

Say there are two threads(A, B) spawned by NFS client to write a same file which size is 300MB in parallel and they will work like the following

  1. A start to write the file at offset 0 ### writing a former part of a file
  2. B start to write the file at offset 181542912 ### writing a latter parf of a file
  3. A and B write the file while keep increasing each offset by 1048576(1024 * 1024
  4. B finish to write the last latter part of the file
  5. A finish to write the last former part of the file (in this case offset: 181403648 byte count:139264

Since the current implementation has not assumed such behavior,
the final total size calculated by A can be wrong.

How to fix

https://github.com/leo-project/leofs/blob/1.3.2/apps/leo_gateway/src/leo_nfs_file_handler.erl#L339 must be called only in case SrcMetadata#?METADATA.cnumber >= IndexEnd

@yosukehara yosukehara added this to the 1.3.3 milestone Feb 2, 2017
@mocchira mocchira changed the title [leo_gateway][nfs] The total file size can be wrong when writing one file in parallel [leo_gateway][nfs] The total file size can be wrong when writing the same file in parallel Feb 3, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants