Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DC-3635] Remove portion of query checking primary consent #1834

Merged
merged 2 commits into from
Dec 21, 2023
Merged

[DC-3635] Remove portion of query checking primary consent #1834

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
18499e2
[DC-3635] Remove portion of query checking primary consent
Dec 15, 2023
15e1485
[DC-3635] Remove fitbit check in CT
Dec 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 2 additions & 22 deletions data_steward/analytics/cdr_ops/controlled_tier_qc/check_controlled_tier_part2.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1300,7 +1300,6 @@ def query_template(table_era):
# **If check fails:**<br>
# * The issue `participant with multiple records` means that those participants have multiple rows in the wear_study table, which should not be possible. Investigate the issue. Start with the CR that creates the wear_study table. <br>
# * The issue `not in person table` means that participants exist in the wear_study table that aren't in the person table, which should not be possible. Investigate the issue. Start with the CR that creates the wear_study table.<br>
# * The issue `no primary consent` means that participants exist in the wear_study table that do not have proper primary consent. Investigate the issue. It is possible that there is another way to determine primary consent. <br>

# +
query = JINJA_ENV.from_string("""
Expand Down Expand Up @@ -1329,26 +1328,6 @@ def query_template(table_era):
SELECT person_id
FROM `{{project_id}}.{{ct_dataset}}.person` o
)

UNION ALL

SELECT
'no primary consent' as issue,
COUNT(person_id) as bad_rows
FROM `{{project_id}}.{{ct_dataset}}.wear_study` ws
WHERE person_id not in ( -- aou consenting participants --
SELECT cte.person_id
FROM latest_primary_consent_records cte
LEFT JOIN ( -- any positive primary consent --
SELECT *
FROM `{{project_id}}.{{ct_dataset}}.observation`
WHERE REGEXP_CONTAINS(observation_source_value, '(?i)extraconsent_agreetoconsent')
AND value_as_concept_id = 45877994) o
ON cte.person_id = o.person_id
AND cte.latest_date = o.observation_date
WHERE o.person_id IS NOT NULL
)

""")
q = query.render(project_id=project_id, ct_dataset=ct_dataset)
df1 = execute(client, q)
Expand All @@ -1371,6 +1350,7 @@ def query_template(table_era):
ignore_index=True)
# -


df1

# +
Expand Down Expand Up @@ -1530,4 +1510,4 @@ def highlight_cells(val):
return f'background-color: {color}'


df.style.applymap(highlight_cells).set_properties(**{'text-align': 'left'})
df.style.applymap(highlight_cells).set_properties(**{'text-align': 'left'})
20 changes: 0 additions & 20 deletions data_steward/analytics/cdr_ops/rt_cdr_qc/cdr_deid_qa_report10_extra.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -866,26 +866,6 @@ def my_sql(table_name, column_name):
SELECT person_id
FROM `{{project_id}}.{{rt_cdr_deid}}.person` o
)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MichaelSchmidt82 you would also need to update the comment for this check to remove primary_consent related info.

UNION ALL

SELECT
'no primary consent' as issue,
COUNT(person_id) as bad_rows
FROM `{{project_id}}.{{rt_cdr_deid}}.wear_study` ws
WHERE person_id not in ( -- aou consenting participants --
SELECT cte.person_id
FROM latest_primary_consent_records cte
LEFT JOIN ( -- any positive primary consent --
SELECT *
FROM `{{project_id}}.{{rt_cdr_deid}}.observation`
WHERE REGEXP_CONTAINS(observation_source_value, '(?i)extraconsent_agreetoconsent')
AND value_as_concept_id = 45877994) o
ON cte.person_id = o.person_id
AND cte.latest_date = o.observation_date
WHERE o.person_id IS NOT NULL
)

""")
q = query.render(project_id=project_id,
rt_cdr_deid=rt_cdr_deid)
Expand Down