Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Access Log format is wrong when writing a large object #130

Closed
firejun opened this issue Jan 28, 2014 · 7 comments
Closed

Access Log format is wrong when writing a large object #130

firejun opened this issue Jan 28, 2014 · 7 comments
Assignees
@mocchira
Labels
Bug Improve _leo_logger
Milestone
1.0.0-pre3

Comments

@firejun
Copy link

firejun commented Jan 28, 2014

Access Log is wrong when I put over 5MB file.

    [PUT]   test     test/1/2/7593b8d15bee259f58f726559646f8f11ebebbdc.68.2.2.2.jpg
    1       5242880 2014-01-27 23:48:38.978861 +0900        1390834118978966        200
    [PUT]   test     test/1/2/7593b8d15bee259f58f726559646f8f11ebebbdc.68.2.2.2.jpg
    2       5242880 2014-01-27 23:48:39.241534 +0900        1390834119241834        200
    [PUT]   test     test/1/2/7593b8d15bee259f58f726559646f8f11ebebbdc.68.2.2.2.jpg
    3       5242880 2014-01-27 23:48:39.455905 +0900        1390834119456055        200
    [PUT]   test     test/1/2/7593b8d15bee259f58f726559646f8f11ebebbdc.68.2.2.2.jpg
    4       1069056 2014-01-27 23:48:39.500824 +0900        1390834119500858        200

I think good that rename when file is chunk.
ex)
test.txt 1 --> test.txt_1
@yosukehara
Copy link
Member

Thank you for your report. I'll fix this issue that a character will be replaced from "new-line" to "hyphen" in the log.

@ghost ghost assigned yosukehara Jan 28, 2014
@mocchira
Copy link
Member

Since the url specification allowed including "hyphen" , "underscore" , there is some possibility of existing files which path suffix is "_1" , "_2" etc...
So replacing new-line with \n would be more better.

@yosukehara
Copy link
Member

I think "\n" is not good because it is not easily recognize key-name. So I'll consider it, again.

@yosukehara
Copy link
Member

I'm going to add a new element which is "chunk number" in the log as the follows:

--------+-------+--------------------+---------+-------+---------------------------------------+-----------------------+----------
Method  | Bucket| Path               |Chunk num| Size  | Timestamp                             | Unix Time             | Response
--------+-------+--------------------+---------+-------+---------------------------------------+-----------------------+----------
[PUT]    photo   photo/large.png      1         102400  2014-01-31 13:28:56.148269 +0900        1381206536148320        200
[PUT]    photo   photo/large.png      2         102400  2014-01-31 13:28:56.148269 +0900        1381206536148320        200
[PUT]    photo   photo/large.png      3         256     2014-01-31 13:28:56.148269 +0900        1381206536148320        200
[PUT]    photo   photo/small.png                1024    2014-01-31 13:28:56.148269 +0900        1381206536148320        200

@mocchira
Copy link
Member

Gooood to me

@yosukehara yosukehara added this to the 1.0.0-pre3 milestone Feb 13, 2014
@yosukehara yosukehara assigned mocchira and unassigned yosukehara Feb 13, 2014
yosukehara added a commit to leo-project/leo_gateway that referenced this issue Feb 20, 2014
@yosukehara
Fix the issue of the access Log format - leo-project/leofs/issues/130
5b127f0
@yosukehara
Copy link
Member

Sharing a sample log

  • Small object:
[GET]   test    test/8813   0   128 2014-02-20 16:40:27.946886 +0900    1392882027946913    200
[GET]   test    test/4547   0   128 2014-02-20 16:40:27.946964 +0900    1392882027946993    200
[GET]   test    test/1618   0   128 2014-02-20 16:40:27.947520 +0900    1392882027947549    200
[GET]   test    test/3318   0   128 2014-02-20 16:40:27.947585 +0900    1392882027947617    200
[GET]   test    test/29047  0   128 2014-02-20 16:40:27.948277 +0900    1392882027948310    200
[GET]   test    test/13476  0   128 2014-02-20 16:40:27.948323 +0900    1392882027948348    200
[PUT]   test    test/1781   0   223230  2014-02-20 16:40:27.948656 +0900    1392882027948680    200
[GET]   test    test/761    0   128 2014-02-20 16:40:27.948765 +0900    1392882027948785    200
  • Large object:
[PUT]   photo   photo/smartos-latest.vmwarevm.tar.bz2   1   15728640    2014-02-20 16:48:06.29162 +0900 1392882486029200    0
[PUT]   photo   photo/smartos-latest.vmwarevm.tar.bz2   2   15728640    2014-02-20 16:48:06.600076 +0900    1392882486600104    0
[PUT]   photo   photo/smartos-latest.vmwarevm.tar.bz2   3   15728640    2014-02-20 16:48:07.146381 +0900    1392882487146437    0
[PUT]   photo   photo/smartos-latest.vmwarevm.tar.bz2   4   15728640    2014-02-20 16:48:07.667068 +0900    1392882487667097    0
[PUT]   photo   photo/smartos-latest.vmwarevm.tar.bz2   5   15728640    2014-02-20 16:48:08.222807 +0900    1392882488222836    0
[PUT]   photo   photo/smartos-latest.vmwarevm.tar.bz2   6   15728640    2014-02-20 16:48:08.771490 +0900    1392882488771519    0
[PUT]   photo   photo/smartos-latest.vmwarevm.tar.bz2   7   15728640    2014-02-20 16:48:09.276005 +0900    1392882489276037    0
[PUT]   photo   photo/smartos-latest.vmwarevm.tar.bz2   8   15728640    2014-02-20 16:48:09.809428 +0900    1392882489809457    0
[PUT]   photo   photo/smartos-latest.vmwarevm.tar.bz2   9   15728640    2014-02-20 16:48:10.310396 +0900    1392882490310425    0
[PUT]   photo   photo/smartos-latest.vmwarevm.tar.bz2   10  15728640    2014-02-20 16:48:10.872610 +0900    1392882490872639    0
[PUT]   photo   photo/smartos-latest.vmwarevm.tar.bz2   11  5649889 2014-02-20 16:48:11.63416 +0900 1392882491064001    200

@firejun
Copy link
Author

firejun commented Feb 20, 2014

Thank you very much!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mocchira mocchira

Labels
Bug Improve _leo_logger
Projects
None yet
Milestone
1.0.0-pre3
Development

No branches or pull requests
3 participants
@yosukehara @mocchira @firejun