Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #2479 - Refactors the form to be more testable #2528

Merged
merged 7 commits into from
Jul 4, 2018

Conversation

karlcow
Copy link
Member

@karlcow karlcow commented Jun 27, 2018

This PR fixes issue #2479

Proposed PR background

The intent is to

  • provide a more testable form
  • add tests for HTTP POST (success and failure)
  • add validation checks for form parameters

r? @miketaylr

karlcow added 4 commits June 28, 2018 07:38
This adds a function which can be augmented in the future to check form parameters
This adds the test for this function.
By starting the logger early up, we can use it to track failures here and there.
This creates an helper to check if the domain is blacklisted.
Probably this can evolve into something more refined where the blacklist is maintained in the config file and not in the code.
@karlcow karlcow requested a review from miketaylr June 27, 2018 22:43
@TravisBuddy
Copy link

TravisBuddy commented Jun 27, 2018

Travis tests have failed

Hey @karlcow,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

nosetests
..............................................................................E..............
======================================================================
ERROR: Test that post is working on /issues/new.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "/home/travis/build/webcompat/webcompat.com/tests/unit/test_urls.py", line 77, in test_successful_post_new_issue
    username='yeeha'))
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/werkzeug/test.py", line 840, in post
    return self.open(*args, **kw)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/testing.py", line 200, in open
    follow_redirects=follow_redirects
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/werkzeug/test.py", line 803, in open
    response = self.run_wsgi_app(environ, buffered=buffered)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/werkzeug/test.py", line 716, in run_wsgi_app
    rv = run_wsgi_app(self.application, environ, buffered=buffered)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/werkzeug/test.py", line 923, in run_wsgi_app
    app_rv = app(environ, start_response)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 2309, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 2295, in wsgi_app
    response = self.handle_exception(e)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 1741, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/travis/build/webcompat/webcompat.com/webcompat/views.py", line 237, in create_issue
    response = report_issue(form, proxy=True).json()
AttributeError: 'dict' object has no attribute 'json'
-------------------- >> begin captured logging << --------------------
flask.app: INFO: None http://testing.example.org
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 93 tests in 1.404s

FAILED (errors=1)
sleep 5
Writing logs to: /home/travis/build/webcompat/webcompat.com/tmp
Statuses Initialization…
Oooops.
We can't find /home/travis/build/webcompat/webcompat.com/data/milestones.json
Double check that everything is configured properly
in config/secrets.py and try again. Good luck!

Fetching milestones from Github…
Milestones saved in data/
Milestones in memory
Starting server in ~*TEST MODE*~
 * Serving Flask app "webcompat" (lazy loading)
 * Environment: production
   WARNING: Do not use the development server in a production environment.
   Use a production WSGI server instead.
 * Debug mode: on
 * Running on http://localhost:5000/ (Press CTRL+C to quit)
 * Restarting with stat
Writing logs to: /home/travis/build/webcompat/webcompat.com/tmp
Statuses Initialization…
Milestones in memory
Starting server in ~*TEST MODE*~
 * Debugger is active!
 * Debugger PIN: 295-160-707

$ npm run lint

> webcompat@ lint /home/travis/build/webcompat/webcompat.com
> npm run lint:js && npm run lint:css


> webcompat@ lint:js /home/travis/build/webcompat/webcompat.com
> eslint ./Gruntfile.js ./tests ./grunt-tasks ./webcompat/static/js/lib


> webcompat@ lint:css /home/travis/build/webcompat/webcompat.com
> stylelint './webcompat/static/css/src/**/*.css' './webcompat/static/css/webcompat.dev.css'


$ npm run build

> webcompat@ build /home/travis/build/webcompat/webcompat.com
> grunt

Running "checkDependencies:default" (checkDependencies) task

Running "jst:compile" (jst) task
File webcompat/static/js/dist/templates.js created.

Running "concat:dist" (concat) task

Running "concat:diagnose" (concat) task

Running "concat:issues" (concat) task

Running "concat:issueList" (concat) task

Running "concat:userActivity" (concat) task

Running "uglify:dist" (uglify) task
>> 1 file created 546.15 kB → 385.13 kB

Running "uglify:ga" (uglify) task
>> 1 file created 780 B → 585 B

Running "uglify:issues" (uglify) task
>> 1 file created 48.78 kB → 27.11 kB

Running "uglify:issueList" (uglify) task
>> 1 file created 37.69 kB → 20.14 kB

Running "uglify:userActivity" (uglify) task
>> 1 file created 21.91 kB → 11.36 kB

Running "uglify:diagnose" (uglify) task
>> 1 file created 12.9 kB → 6.44 kB

Running "uglify:contributors" (uglify) task
>> 1 file created 1.1 kB → 737 B

Running "postcss:dist" (postcss) task
>> 1 processed stylesheet created.

Running "cssmin:combine" (cssmin) task
>> 1 file created. 1.8 kB → 36.74 kB

Running "purifycss:target" (purifycss) task
Source Files:  [ 'webcompat/templates/about.html',
  'webcompat/templates/contact.html',
  'webcompat/templates/contributors.html',
  'webcompat/templates/error.html',
  'webcompat/templates/index.html',
  'webcompat/templates/issue.html',
  'webcompat/templates/layout.html',
  'webcompat/templates/list-issue.html',
  'webcompat/templates/new-issue.html',
  'webcompat/templates/privacy.html',
  'webcompat/templates/user-activity.html' ]
Source Files:  [ 'webcompat/templates/about.html',
  'webcompat/templates/contact.html',
  'webcompat/templates/contributors.html',
  'webcompat/templates/contributors/build-tools.html',
  'webcompat/templates/contributors/diagnose-bug.html',
  'webcompat/templates/contributors/organize-webcompat-events.html',
  'webcompat/templates/contributors/report-bug.html',
  'webcompat/templates/contributors/reproduce-bug.html',
  'webcompat/templates/contributors/site-outreach.html',
  'webcompat/templates/contributors/sub-nav.html',
  'webcompat/templates/contributors/web-platform-research.html',
  'webcompat/templates/dashboard/footer.html',
  'webcompat/templates/dashboard/layout.html',
  'webcompat/templates/dashboard/triage.html',
  'webcompat/templates/dashboard/triage/activity-indicator.html',
  'webcompat/templates/dashboard/triage/filters.html',
  'webcompat/templates/dashboard/triage/needstriage-list.html',
  'webcompat/templates/dashboard/triage/no-result.html',
  'webcompat/templates/dashboard/triage/stats.html',
  'webcompat/templates/dashboard/triage/top-bar.html',
  'webcompat/templates/error.html',
  'webcompat/templates/home-page/browse-issues.html',
  'webcompat/templates/home-page/form.html',
  'webcompat/templates/home-page/hero.html',
  'webcompat/templates/home-page/how-it-works.html',
  'webcompat/templates/home-page/join-us.html',
  'webcompat/templates/index.html',
  'webcompat/templates/issue.html',
  'webcompat/templates/issue/issue-aside.html',
  'webcompat/templates/issue/issue-comment-submit.html',
  'webcompat/templates/issue/issue-information.html',
  'webcompat/templates/layout.html',
  'webcompat/templates/list-issue.html',
  'webcompat/templates/list-issue/search-issue-filter.html',
  'webcompat/templates/nav/addon-links.html',
  'webcompat/templates/nav/nav.html',
  'webcompat/templates/nav/shared-nav-links.html',
  'webcompat/templates/nav/topbar.html',
  'webcompat/templates/new-issue.html',
  'webcompat/templates/privacy.html',
  'webcompat/templates/shared/footer.html',
  'webcompat/templates/shared/svg-icons.html',
  'webcompat/templates/user-activity.html',
  'webcompat/templates/user-activity/reported-by-user.html',
  'webcompat/templates/user-activity/user-mentions.html',
  'webcompat/templates/web_modules/pagination.html',
  'webcompat/templates/web_modules/tag.html' ]
Source Files:  [ 'webcompat/templates/issue/aside.jst',
  'webcompat/templates/issue/issue-comment-list.jst',
  'webcompat/templates/issue/issue-labels-sub.jst',
  'webcompat/templates/issue/issue-labels.jst',
  'webcompat/templates/issue/issue-milestones-sub.jst',
  'webcompat/templates/issue/issue-milestones.jst',
  'webcompat/templates/issue/metadata.jst',
  'webcompat/templates/issue/thanks.jst',
  'webcompat/templates/issue/upload-image.jst',
  'webcompat/templates/list-issue/dropdown.jst',
  'webcompat/templates/list-issue/issuelist-issue.jst',
  'webcompat/templates/list-issue/issuelist-search.jst',
  'webcompat/templates/list-issue/issuelist-sorting.jst',
  'webcompat/templates/web_modules/issue-list.jst',
  'webcompat/templates/web_modules/label-editor.jst',
  'webcompat/templates/web_modules/milestone-editor.jst' ]
Source Files:  []
Style Files:  [ 'webcompat/static/css/webcompat.dev.css' ]
Style Files:  [ 'webcompat/static/css/src/animations.css',
  'webcompat/static/css/src/bio.css',
  'webcompat/static/css/src/button.css',
  'webcompat/static/css/src/dropdown.css',
  'webcompat/static/css/src/filter-bar.css',
  'webcompat/static/css/src/filter.css',
  'webcompat/static/css/src/footer.css',
  'webcompat/static/css/src/form.css',
  'webcompat/static/css/src/grid.css',
  'webcompat/static/css/src/hero.css',
  'webcompat/static/css/src/icons.css',
  'webcompat/static/css/src/issue.css',
  'webcompat/static/css/src/issues-list.css',
  'webcompat/static/css/src/label-box.css',
  'webcompat/static/css/src/label-editor.css',
  'webcompat/static/css/src/labels.css',
  'webcompat/static/css/src/nav.css',
  'webcompat/static/css/src/notification-bar.css',
  'webcompat/static/css/src/pagination.css',
  'webcompat/static/css/src/sub-nav.css',
  'webcompat/static/css/src/tiles.css',
  'webcompat/static/css/src/typo.css',
  'webcompat/static/css/src/variables.css' ]

    ________________________________________________
    |
    |   PurifyCSS has reduced the file size by ~ 10.5%  
    |
    ________________________________________________
    

    ________________________________________________
    |
    |   PurifyCSS - Rejected selectors:  
    |   .visually-hidden
    |	.filter-bar-dropdown-listbox ul
    |	.filter-bar-dropdown-listbox a
    |	.filter-bar-dropdown-listbox a:focus
    |	:checked + label .collapsed-text
    |	.filter-bar-dropdown-input:not(:checked) + label .collapsed-text
    |	.filter-bar-dropdown-listbox
    |	:checked ~ .filter-bar-dropdown-listbox
    |	.filter-label .needscontact
    |	.filter-label .needscontact:hover
    |	.filter-label .needscontact.is-active
    |	.filter-label .sitewait
    |	.filter-label .sitewait:hover
    |	.filter-label .sitewait.is-active
    |	.form-upload:hover .visually-hidden
    |	.is-validated .check::after
    |	.is-validated.no-js-error .label-upload
    |	.is-validated.no-js-error:hover .label-upload
    |	.is-validated.no-js-error:focus-within .label-upload
    |	.is-validated .label-remove
    |	.is-validated:hover .icon-remove
    |	.is-validated:focus-within .icon-remove
    |	.is-validated:hover .label-remove .label-icon-message
    |	.is-validated:focus-within .label-remove .label-icon-message
    |	.label-editor-list-item.focused
    |	.label-editorLauncher
    |	.label-editorLauncher.is-active + .label-editor::after
    |	.label-editorLauncher.is-active + .label-editor
    |	.label-editor-launcher:hover
    |	.label-editor-launcher::before
    |	.label-editor-launcher.is-active::before
    |	.wc-Issue-categoryEditor-button .label-editorLauncher
    |	.issue.label-needscontact .x2
    |	.issue.label-sitewait .x2
    |	.label-box-header.label-needscontact
    |	.label-box-editor .label-needscontact
    |	.label-box-header.label-sitewait
    |	.label-box-editor .label-sitewait
    |	.label-box .label-editor-list .label-needscontact
    |	.label-box .label-editor-list .label-sitewait
    |	.is-offscreen
    |	.notification-bar
    |	.notification-information
    |	.notification-alert
    |	.notification-thanks
    |	.is-disabled
    |	.italic
    |	blockquote
    |
    ________________________________________________
    
File "tmp/purestyles.css" created.

Done.

$ nosetests
..............................................................................E..............
======================================================================
ERROR: Test that post is working on /issues/new.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "/home/travis/build/webcompat/webcompat.com/tests/unit/test_urls.py", line 77, in test_successful_post_new_issue
    username='yeeha'))
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/werkzeug/test.py", line 840, in post
    return self.open(*args, **kw)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/testing.py", line 200, in open
    follow_redirects=follow_redirects
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/werkzeug/test.py", line 803, in open
    response = self.run_wsgi_app(environ, buffered=buffered)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/werkzeug/test.py", line 716, in run_wsgi_app
    rv = run_wsgi_app(self.application, environ, buffered=buffered)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/werkzeug/test.py", line 923, in run_wsgi_app
    app_rv = app(environ, start_response)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 2309, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 2295, in wsgi_app
    response = self.handle_exception(e)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 1741, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/travis/build/webcompat/webcompat.com/webcompat/views.py", line 237, in create_issue
    response = report_issue(form, proxy=True).json()
AttributeError: 'dict' object has no attribute 'json'
-------------------- >> begin captured logging << --------------------
flask.app: INFO: None http://testing.example.org
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 93 tests in 1.404s

FAILED (errors=1)

@karlcow
Copy link
Member Author

karlcow commented Jun 27, 2018

@miketaylr if I didn't mess up. I have rebased this pull request on today's version of the master.

@karlcow
Copy link
Member Author

karlcow commented Jun 27, 2018

→ nosetests
.............................................................................................
----------------------------------------------------------------------
Ran 93 tests in 2.358s

OK

@karlcow
Copy link
Member Author

karlcow commented Jun 27, 2018

Oops Sorry Travis.

Copy link
Member

@miketaylr miketaylr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Very nice commits, and easy to review. Thanks for all the new tests.

Just one question about removing .json()

@@ -213,6 +213,20 @@ def test_is_valid_issue_form(self):
('username', u''),
])
self.assertTrue(helpers.is_valid_issue_form(valid_form))
# The value for submit-Type can be only:

This comment was marked as abuse.

@@ -42,6 +61,43 @@ def test_new_issue(self):
rv = self.app.get('/issues/new', environ_base=headers)
self.assertEqual(rv.status_code, 200)

@patch('webcompat.views.report_issue')
def test_successful_post_new_issue(self, mock_proxy):

This comment was marked as abuse.

@@ -232,7 +235,7 @@ def create_issue():
session['form_data'] = form
return redirect(url_for('login'))
elif form.get('submit_type') == PROXY_REPORT:
response = report_issue(form, proxy=True).json()
response = report_issue(form, proxy=True)

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

@@ -568,8 +568,7 @@ def is_valid_issue_form(form):
'problem_category',
'submit_type',
'url',
'username',
]
'username', ]

This comment was marked as abuse.

@miketaylr
Copy link
Member

making current status more visible: #2528 (comment)

@karlcow
Copy link
Member Author

karlcow commented Jul 4, 2018

def report_issue(form, proxy=False):
'''Report an issue, as a logged in user or anonymously.'''
# /repos/:owner/:repo/issues
path = 'repos/{0}'.format(REPO_URI)
if proxy:
return proxy_request('post', path,
data=json.dumps(build_formdata(form)))
else:
return github.post(path, build_formdata(form))

report_issue is used in 3 places in the code all in views.py:

One time in /file route to extract the issue number number=response.get('number')

# This route won't ever be viewed by a human being--there's not
# a corresponding template. It exists just to submit an issue after
# a user auths with GitHub.
@app.route('/file')
def file_issue():
"""File an issue on behalf of the user that just gave us authorization."""
form_data = session.get('form_data', None)
if not session:
abort(401)
if session and (form_data is None):
abort(403)
response = report_issue(session['form_data'])
# Get rid of stashed form data
session.pop('form_data', None)
session['show_thanks'] = True
return redirect(url_for('show_issue', number=response.get('number')))

Twice in issues/new route to extract the exact same thing.

# form submission for 3 scenarios: authed, to be authed, anonymous
if form.get('submit_type') == AUTH_REPORT:
if g.user: # If you're already authed, submit the bug.
response = report_issue(form)
session['show_thanks'] = True
return redirect(url_for('show_issue',
number=response.get('number')))
else: # Stash form data into session, go do GitHub auth
session['form_data'] = form
return redirect(url_for('login'))
elif form.get('submit_type') == PROXY_REPORT:
response = report_issue(form, proxy=True).json()
session['show_thanks'] = True
return redirect(url_for('show_issue', number=response.get('number')))
else:
# if anything wrong, we assume it is a bad forged request
abort(400)

The response is not used for anything else

Let's make things easier, then and return just what we need. :)

@karlcow
Copy link
Member Author

karlcow commented Jul 4, 2018

→ nosetests
.............................................................................................
----------------------------------------------------------------------
Ran 93 tests in 1.985s

OK

@karlcow
Copy link
Member Author

karlcow commented Jul 4, 2018

@karlcow
Copy link
Member Author

karlcow commented Jul 4, 2018

I could have simplified more report_issue. but it really reports an issue. :)

Before report_issue was reporting either a JSON as dict, either a response object, this makes it uniform so it will only report JSON data as a dict. It simplifies testing and make it less… surprising. :)

If in the future we need the access to status, we can revisit this and makes it return response but uniformly.
@karlcow
Copy link
Member Author

karlcow commented Jul 4, 2018

OK Let's see if travis is happy :) then @miketaylr it's all yours. :)

Basically report_issue was non uniform, it was returning a Response object when anonymous, and a JSON dict when logged. I changed the logic. so it always returns JSON as dict.

@miketaylr
Copy link
Member

Basically report_issue was non uniform, it was returning a Response object when anonymous, and a Response object when anonymous. I changed the logic. so it always returns JSON as dict.

thanks for cleaning this up!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants