Skip to content

Commit 4db8276

Browse files
allison-code-dot-orgallison-code-dot-org
authored
Feature/ambassadors (#141)
2 parents 5078a25 + 81020b5 commit 4db8276

20 files changed

+739
-7
lines changed

‎dbt/models/marts/misc/dim_forms.sql renamed to ‎archive/dim_forms.sql

File renamed without changes.

‎dbt/data/seed_cs_ambassador_app.csv

Lines changed: 260 additions & 0 deletions
Large diffs are not rendered by default.

‎dbt/models/marts/misc/_misc_models.yml

Lines changed: 64 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,68 @@
11
version: 2
22

33
models:
4-
- name: dim_forms
5-
description: all forms data with form geos fields
4+
# - name: dim_forms
5+
# description: all forms data with form geos fields
6+
# columns:
7+
# - name: form_id
8+
# description: unique id for table
9+
# tests:
10+
# - unique
11+
# - not_null
12+
13+
- name: dim_foorms
14+
description: |
15+
This model contains the formatted survey responses for all foorm surveys (e.g. YWinCS ambassador application)
616
columns:
7-
- name: form_id
8-
description: unique id for table
9-
tests:
10-
- unique
11-
- not_null
17+
- name: form_name
18+
19+
- name: user_id
20+
tests:
21+
- not_null
22+
23+
- name: code_studio_name
24+
description: the name of the person filling out the foorm, as it appears in code studio
25+
26+
- name: email
27+
description: email associated with the account filling out the foorm
28+
29+
- name: created_at
30+
description: the date on which the foorm was submitted
31+
32+
- name: item_name
33+
34+
- name: item_text
35+
36+
- name: item_type
37+
38+
- name: response_text
39+
40+
- name: full_response_text
41+
description: includes the text associated with "other"
42+
43+
- name: school_year
44+
description: the school year in which the foorm was submitted
45+
46+
- name: school_name
47+
description: if the school name is in all-caps and there is a school_id associated with it, it was found in the NCES dropdown. If not, it was typed in manually by the user.
48+
49+
- name: school_type
50+
51+
- name: school_state
52+
53+
- name: school_id
54+
description: if the ambassador found their school in the NCES dropdown, this will be populated. Null otherwise.
55+
56+
- name: dim_ambassador_activity
57+
description: |
58+
this model contains all the student activity within ambassador's sections in order to answer questions like:
59+
- how many students were associated with each ambassador?
60+
- how many sections did each ambassador have?
61+
- what course(s) was the student activity associated with?
62+
columns:
63+
- name: school_year
64+
description: the school year in which students were added to the section
65+
66+
- name: dim_ambassador_apps
67+
description: |
68+
this model contains information from the ambassador applications (imported periodically from google form output). There is one row per application submitted.

‎dbt/models/marts/misc/dim_ambassador_activity.sql

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
with
2+
3+
foorm_survey as (
4+
select *
5+
from {{ ref('dim_foorms') }}
6+
where form_name = 'surveys/teachers/young_women_in_cs'
7+
),
8+
9+
sections as (
10+
select *
11+
from {{ ref('stg_dashboard__sections') }}
12+
),
13+
14+
followers as (
15+
select *
16+
from {{ ref('stg_dashboard__followers') }}
17+
),
18+
19+
school_years as (
20+
select *
21+
from {{ ref('int_school_years') }}
22+
),
23+
24+
user_levels as (
25+
select *
26+
from {{ ref('stg_dashboard__user_levels') }}
27+
),
28+
29+
course_structure as (
30+
select *
31+
from {{ ref('dim_course_structure') }}
32+
)
33+
34+
select distinct
35+
fs.user_id
36+
, fs.code_studio_name
37+
, fs.email
38+
, s.section_id
39+
, s.section_name
40+
, s.created_at as section_created_dt
41+
, sy.school_year
42+
, f.student_id
43+
, cs.course_name_true as course_name
44+
, cs.script_name
45+
from foorm_survey as fs
46+
47+
left join sections as s
48+
on fs.user_id = s.teacher_id
49+
50+
left join followers as f
51+
on s.section_id = f.section_id
52+
53+
left join school_years as sy
54+
on f.student_added_at between sy.started_at and sy.ended_at -- school year when the followers were added to the section
55+
56+
left join user_levels as ul
57+
on f.student_id = ul.user_id
58+
and trunc(f.student_added_at) <= trunc(ul.created_at)
59+
and ul.created_at between sy.started_at and sy.ended_at -- user activity for section participants (followers) after they were added to the section and in the same school year when the followers were added to the section
60+
61+
left join course_structure as cs
62+
on ul.script_id = cs.script_id
63+
and ul.level_id = cs.level_id
64+
65+
where fs.item_name = 'teacher_account'
66+
and fs.response_value = 'true'

‎dbt/models/marts/misc/dim_ambassador_apps.sql

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
with ambassador_apps as (
2+
select *
3+
from {{ ref('stg_external_datasets__ambassador_apps') }}
4+
)
5+
6+
select *
7+
from ambassador_apps

‎dbt/models/marts/misc/dim_foorms.sql

Lines changed: 195 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,195 @@
1+
with
2+
3+
foorm_simple_survey_forms as (
4+
select *
5+
from {{ ref('stg_dashboard_pii__foorm_simple_survey_forms') }}
6+
),
7+
8+
foorm_simple_survey_submissions as (
9+
select *
10+
from {{ ref('stg_dashboard_pii__foorm_simple_survey_submissions') }}
11+
),
12+
13+
foorm_submissions_reshaped as (
14+
select *
15+
from {{ ref('stg_analysis__foorm_submissions_reshaped') }}
16+
),
17+
18+
foorm_forms_reshaped as (
19+
select *
20+
from {{ ref('stg_analysis__foorm_forms_reshaped') }}
21+
),
22+
23+
foorm_surveys as (
24+
select
25+
fssf.path,
26+
fssf.form_name,
27+
fsss.simple_survey_form_id,
28+
fsss.foorm_submission_id,
29+
fsss.user_id,
30+
fsss.created_at,
31+
fsr.item_name,
32+
fsr.matrix_item_name,
33+
fsr.response_value,
34+
fsr.response_text,
35+
ffr.item_type,
36+
ffr.item_text,
37+
ffr.matrix_item_header,
38+
ffr.response_options,
39+
ffr.num_response_options
40+
41+
from foorm_simple_survey_forms as fssf
42+
43+
join foorm_simple_survey_submissions as fsss
44+
on fsss.simple_survey_form_id = fssf.foorm_simple_survey_form_id
45+
46+
left join foorm_submissions_reshaped as fsr
47+
on fsr.submission_id = fsss.foorm_submission_id
48+
49+
left join foorm_forms_reshaped as ffr
50+
on ffr.form_name = fssf.form_name
51+
and ffr.form_version = fssf.form_version
52+
and fsr.item_name = ffr.item_name
53+
),
54+
55+
comments as (
56+
select distinct
57+
foorm_submission_id
58+
, item_name
59+
, response_text
60+
, left(item_name, (position ('-Comment' in item_name)-1)) as item_name_parent
61+
from foorm_surveys
62+
where item_name like '%-Comment'
63+
),
64+
65+
questions_multiselect as (
66+
select
67+
f.*
68+
, replace(replace(split_part(f.response_text, ', ',1),'[',''),']','') sp1
69+
, replace(replace(split_part(f.response_text, ', ',2),'[',''),']','') sp2
70+
, replace(replace(split_part(f.response_text, ', ',3),'[',''),']','') sp3
71+
, replace(replace(split_part(f.response_text, ', ',4),'[',''),']','') sp4
72+
, replace(replace(split_part(f.response_text, ', ',5),'[',''),']','') sp5
73+
, replace(replace(split_part(f.response_text, ', ',6),'[',''),']','') sp6
74+
, replace(replace(split_part(f.response_text, ', ',7),'[',''),']','') sp7
75+
, replace(replace(split_part(f.response_text, ', ',8),'[',''),']','') sp8
76+
, replace(replace(split_part(f.response_text, ', ',9),'[',''),']','') sp9
77+
, replace(replace(split_part(f.response_text, ', ',10),'[',''),']','') sp10
78+
, replace(replace(split_part(f.response_text, ', ',11),'[',''),']','') sp11
79+
, replace(replace(split_part(f.response_text, ', ',12),'[',''),']','') sp12
80+
, replace(replace(split_part(f.response_text, ', ',13),'[',''),']','') sp13
81+
, replace(replace(split_part(f.response_text, ', ',14),'[',''),']','') sp14
82+
, replace(replace(split_part(f.response_text, ', ',15),'[',''),']','') sp15
83+
from foorm_surveys f
84+
where item_type = 'multiSelect'
85+
),
86+
87+
multiselect_long as (
88+
select *
89+
from questions_multiselect
90+
unpivot (
91+
multiple_choice for response in (
92+
sp1
93+
, sp2
94+
, sp3
95+
, sp4
96+
, sp5
97+
, sp6
98+
, sp7
99+
, sp8
100+
, sp9
101+
, sp10
102+
, sp11
103+
, sp12
104+
, sp13
105+
, sp14
106+
, sp15
107+
)
108+
)
109+
where len(multiple_choice)>=1
110+
),
111+
112+
user_school_infos as (
113+
select *
114+
from {{ ref('stg_dashboard_pii__user_school_infos') }}
115+
),
116+
117+
school_years as (
118+
select *
119+
from {{ ref('int_school_years') }}
120+
),
121+
122+
school_infos as (
123+
select *
124+
from {{ ref('stg_dashboard__school_infos') }}
125+
),
126+
127+
schools as (
128+
select *
129+
from {{ ref('dim_schools') }}
130+
),
131+
132+
users as (
133+
select *
134+
from {{ ref('stg_dashboard_pii__users') }}
135+
)
136+
137+
select distinct
138+
s.form_name
139+
, s.foorm_submission_id
140+
, s.user_id
141+
, u.name as code_studio_name
142+
, u.teacher_email as email
143+
, trunc(s.created_at) as created_at
144+
, s.matrix_item_name
145+
, s.matrix_item_header
146+
, s.item_name
147+
, trim(rtrim(s.item_text,'.')) as item_text
148+
, s.item_type
149+
, s.response_value
150+
, case
151+
when s.item_type = 'multiSelect' then fsl.multiple_choice
152+
when s.response_value = 'other' then 'other'
153+
when s.response_value = 'true' then 'Y'
154+
else trim(s.response_text)
155+
end as response_text
156+
, case
157+
when s.response_value = 'other'
158+
then (
159+
select c.response_text
160+
from comments c
161+
where s.foorm_submission_id = c.foorm_submission_id
162+
and s.item_name = c.item_name_parent
163+
)
164+
when s.response_value = 'true' then 'Y'
165+
else trim(s.response_text)
166+
end as full_response_text
167+
, sy.school_year
168+
, coalesce (ss.school_name, si.school_name) as school_name
169+
, coalesce (ss.state, si.state) as school_state
170+
, coalesce (ss.school_type, si.school_type) as school_type
171+
, si.school_id
172+
173+
from foorm_surveys as s
174+
175+
join users as u
176+
on s.user_id = u.user_id
177+
178+
left join multiselect_long as fsl
179+
on s.foorm_submission_id = fsl.foorm_submission_id
180+
and s.item_name = fsl.item_name
181+
182+
left join school_years sy
183+
on s.created_at between sy.started_at and sy.ended_at
184+
185+
left join user_school_infos as usi
186+
on s.user_id = usi.user_id
187+
and s.created_at between usi.started_at and coalesce (usi.ended_at, sysdate)
188+
189+
left join school_infos as si
190+
on usi.school_info_id = si.school_info_id
191+
192+
left join schools as ss
193+
on si.school_id = ss.school_id
194+
195+
where s.item_name not like '%-Comment'

‎dbt/models/marts/students/_students__models.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,7 @@ models:
108108
config:
109109
tags: ['released']
110110

111+
111112
# - name: dim_student_status
112113
# description: This model categorizes students based on their activity status across different school years. It provides insights into user engagement by assigning a status that reflects their activity in the current, previous, and any earlier school years.
113114
# columns:

‎dbt/models/staging/analysis/_analysis__models.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
version: 2

‎dbt/models/staging/analysis/_analysis__sources.yml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
version: 2
2+
3+
sources:
4+
- name: analysis
5+
database: dashboard
6+
schema: analysis
7+
tables:
8+
- name: foorm_forms_reshaped

‎dbt/models/staging/analysis/base/base_analysis__foorm_forms_reshaped.sql

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
with
2+
source as (
3+
select *
4+
from {{ source('analysis', 'foorm_forms_reshaped') }}
5+
)
6+
7+
select *
8+
from source

‎dbt/models/staging/analysis/stg_analysis__foorm_forms_reshaped.sql

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
with
2+
3+
foorm_forms_reshaped as (
4+
select *
5+
from {{ ref('base_analysis__foorm_forms_reshaped') }}
6+
)
7+
8+
select *
9+
from foorm_forms_reshaped
10+
11+

‎dbt/models/staging/analysis_pii/_analysis_pii__models.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
version: 2

0 commit comments

Comments
 (0)