Proper setup with rollover access rules

[davis68]

I recently changed my class' grading structure to allow for early completion of assignments.

The desired behavior is for students to be able to start an assignment early (tag early) for full credit; after the official release date they can continue to complete it under regular rules. I am using the roll_over​ setting to handle access before the official release date so that they can continue to work.

Some students at least are reporting that after the official release (regular tag) they are no longer able to work on early​ sessions.

Can you tell me if these rules are wrong for this use case?

{# {{{ lesson submission header #}

{% macro lesson_header(nr, title="", release_offset=-2, due_offset=7, due_late_days=0) %}

title: "{{title}} lesson {{"{0}".format(nr).rjust(2)}}"
description: |
    # {{title}} lesson {{"{0}".format(nr).rjust(2)}}
rules:
    tags:
    -   regular
    -   approved
    -   notforcredit
    -   early
    -   late

    start:
    -
        if_has_role: [instructor, ta, course_aide]
        may_start_new_session: True
        may_list_existing_sessions: True
        tag_session: notforcredit
    -
        if_before: lesson {{nr}} - {{release_offset}} days @ 00:00
        if_has_role: [student]
        if_has_fewer_sessions_than: 1
        may_start_new_session: True
        may_list_existing_sessions: True
        default_expiration_mode: roll_over
        tag_session: early
    -
        if_after: lesson {{nr}} - {{release_offset}} days @ 00:00
        if_before: lesson {{nr}} + {{due_offset}} days @ 23:59
        if_has_role: [student]
        if_has_fewer_sessions_than: 1
        may_start_new_session: True
        may_list_existing_sessions: True
        default_expiration_mode: roll_over
        tag_session: regular
    -
        if_after: lesson {{nr}} + {{due_offset}} days @ 23:59
        if_before: lesson {{nr}}  + {{due_offset+due_late_days}} days @ 23:59
        if_has_role: [student]
        if_has_fewer_sessions_than: 2
        may_start_new_session: True
        may_list_existing_sessions: True
        default_expiration_mode: roll_over
        tag_session: late
    -
        if_after: lesson {{nr}} + {{due_offset+due_late_days}} days @ 23:59
        if_before: course_end @ 23:59
        if_has_role: [student]
        if_has_fewer_sessions_than: 3
        may_start_new_session: True
        may_list_existing_sessions: True
        default_expiration_mode: end
        tag_session: notforcredit
    -
        may_start_new_session: False
        may_list_existing_sessions: True

    access:
    # Instructor view
    -
        if_has_role: [instructor, ta, course_aide]
        permissions: [view, submit_answer, change_answer, see_answer_before_submission, end_session, see_correctness]
    # Unenrolled view
    -
        if_has_role: [unenrolled]
        message: |
            You do not seem to be enrolled in the course website, which is why
            you cannot make changes below. Please verify that you are signed in,
            and then return to the class web page and find the big blue "Enroll"
            button near the top.
        permissions: [view]
    # Unfinished full-credit session marked 'roll_over' before due date.
    -
        if_has_tag: early
        if_in_progress: True
        if_expiration_mode: roll_over
        message: |
          You have started an early session.  It will roll over to a regular
          session when the official lesson release date happens.
        permissions: [view, submit_answer, end_session, see_correctness, change_answer, set_roll_over_expiration_mode]
    # Unfinished full-credit session marked 'end' after due date.
    -
        if_has_tag: regular
        if_in_progress: True
        if_after: lesson {{nr}} - {{release_offset}} days @ 00:00
        if_expiration_mode: end
        message: |
          The due date has passed. If you have marked your session to end at the
          deadline, it will receive full credit, but it will end automatically
          fairly soon. If you would like to continue working, please mark your
          session to roll over into 50% credit by selecting 'Keep session and
          apply new rules'.
        permissions: [view, submit_answer, end_session, see_correctness, change_answer, set_roll_over_expiration_mode]
    # Unfinished full-credit session marked 'roll_over' before due date.
    -
        if_has_tag: regular
        if_in_progress: True
        if_expiration_mode: roll_over
        message: |
          You have marked your session to roll over to 50% credit at the due
          date. If you would like to have your current answers graded as-is
          (and receive full credit for them), please select 'End session
          and grade'.
        permissions: [view, submit_answer, end_session, see_correctness, change_answer, set_roll_over_expiration_mode]
    # Finished full-credit session before due date.  Don't show answer.
    -
        if_has_tag: regular
        if_after: lesson {{nr}} @ 00:00
        if_before: lesson {{nr}} + {{due_offset}} days @ 23:59
        permissions: [view, see_correctness]
    # Practice session after due date.
    -
        if_has_tag: notforcredit
        if_after: lesson {{nr}} + {{due_offset}} days @ 23:59
        permissions: [view, submit_answer, see_correctness, change_answer, see_answer_after_submission]
    # Any other fallback case.
    -
        permissions: [view, see_correctness]

    grade_identifier: lesson{{"{0}".format(nr).zfill(2)}}
    grade_aggregation_strategy: max_grade

    grading:
    # Full-credit deadlines
    -
        if_has_tag: early
        description: "Full Credit (Early Start)"
        due: lesson {{nr}} + {{due_offset}} days @ 23:59
        generates_grade: True
        credit_percent: 100
    -
        if_has_tag: regular
        description: "Full Credit (Regular)"
        due: lesson {{nr}} + {{due_offset}} days @ 23:59
        generates_grade: True
        credit_percent: 100
    # Late deadline
    -
        if_has_tag: late
        description: "50% Credit (Late Start)"
        due: lesson {{nr}} + {{due_offset+due_late_days}} days @ 23:59
        generates_grade: True
        credit_percent: 50
    # No-credit viewing
    -
        if_has_tag: notforcredit
        description: "Study Only (No Credit Available)"
        due: course_end @ 23:59
        generates_grade: true
        credit_percent: 0
    # Approved late take
    -
        if_has_tag: approved
        description: "Full Credit (Approved Exception)"
        generates_grade: true
        credit_percent: 100
    -
        credit_percent: 0

{% endmacro %}

{# }}} #}

They are invoked in a flow file as:

{{ lesson_header(0, title="lesson 00 computation/intro", release_offset=0, due_offset=16) }}

[davis68]

• All lesson numbers are two-digit (lesson00, lesson01, etc.).
• release_offset was originally intended to handle M/W shifts in due dates, but is always 0. I can remove it but haven't.
• The late tag is not currently in use.

[davis68]

Here are two that are reported as unanswerable on returning:

• early: https://relate.cs.illinois.edu/course/cs101-sp22/grading/single-grade/20794/2147/
• regular: https://relate.cs.illinois.edu/course/cs101-sp22/grading/single-grade/21134/2150/

Based on my understanding of the flow docs, roll_over should move to the next access rule set at the time period. E.g. early should become subject to regular. Is there more handling setup than that?

[inducer]

Rollover only happens when someone clicks "impose deadline" on the grading interface.

[davis68]

500s for me when I apply, e.g. on

https://relate.cs.illinois.edu/course/cs101-sp22/grading/by-opportunity/2147/

Edit: n/m on a later try it worked.

[davis68]

Hmm, "Expire sessions" simply ended all of the early sessions rather than rolling them over to regular rules.

[inducer]

500s for me when I apply

Yeah, that was a transient thing with the queuing system, apparently.

  File "/home/andreas/relate/.venv/lib/python3.9/site-packages/kombu/connection.py", line 896, in default_channel
    self._ensure_connection(**conn_opts)
  File "/home/andreas/relate/.venv/lib/python3.9/site-packages/kombu/connection.py", line 434, in _ensure_connection
    return retry_over_time(
  File "/usr/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/andreas/relate/.venv/lib/python3.9/site-packages/kombu/connection.py", line 451, in _reraise_as_library_errors
    raise ConnectionError(str(exc)) from exc

Exception Type: OperationalError at /course/cs101-sp22/grading/by-opportunity/2147/
Exception Value: timed out

Unrelated, AFAICT.

n/m on a later try it worked.

As in, it ran, or it did what you expected?

Hmm, "Expire sessions" simply ended all of the early sessions rather than rolling them over to regular rules.

Sorry about that! I hope it's a manageable disaster. The code for this is a wee bit weird:

relate/course/flow.py

Lines 1033 to 1064 in 206689b

	session_start_rule = get_session_start_rule(
	flow_session.course, flow_session.participation,
	flow_session.flow_id, fctx.flow_desc, now_datetime,
	for_rollover=True)
	if not session_start_rule.may_start_new_session:
	# No new session allowed: finish.
	finish_flow_session(fctx, flow_session, grading_rule,
	now_datetime=now_datetime, respect_preview=False)
	return True
	else:
	flow_session.access_rules_tag = session_start_rule.tag_session
	# {{{ FIXME: This is weird and should probably not exist.
	access_rule = get_session_access_rule(
	flow_session, fctx.flow_desc, now_datetime)
	if session_start_rule.default_expiration_mode is not None:
	flow_session.expiration_mode = \
	session_start_rule.default_expiration_mode
	elif not is_expiration_mode_allowed(
	flow_session.expiration_mode, access_rule.permissions):
	flow_session.expiration_mode = flow_session_expiration_mode.end
	# }}}
	flow_session.save()
	return True

relate/course/utils.py

Lines 310 to 354 in 206689b

	if not _eval_participation_tags_conditions(rule, participation):
	continue
	if not for_rollover and hasattr(rule, "if_in_facility"):
	if rule.if_in_facility not in facilities:
	continue
	if not for_rollover and hasattr(rule, "if_has_in_progress_session"):
	session_count = FlowSession.objects.filter(
	participation=participation,
	course=course,
	flow_id=flow_id,
	in_progress=True).count()
	if bool(session_count) != rule.if_has_in_progress_session:
	continue
	if not for_rollover and hasattr(rule, "if_has_session_tagged"):
	tagged_session_count = FlowSession.objects.filter(
	participation=participation,
	course=course,
	access_rules_tag=rule.if_has_session_tagged,
	flow_id=flow_id).count()
	if not tagged_session_count:
	continue
	if not for_rollover and hasattr(rule, "if_has_fewer_sessions_than"):
	session_count = FlowSession.objects.filter(
	participation=participation,
	course=course,
	flow_id=flow_id).count()
	if session_count >= rule.if_has_fewer_sessions_than:
	continue
	if not for_rollover and hasattr(rule, "if_has_fewer_tagged_sessions_than"):
	tagged_session_count = FlowSession.objects.filter(
	participation=participation,
	course=course,
	access_rules_tag__isnull=False,
	flow_id=flow_id).count()
	if tagged_session_count >= rule.if_has_fewer_tagged_sessions_than:
	continue

What it does is ask "would a student be able to open a new session at this point, if we ignore a number of conditionals related to session existence". Maybe looking at this you can figure out what happened?