An Online Learning Tutor Agent That Measures Learning, Not Chat

Imagine a hypothetical algebra learner repeatedly answering fraction questions correctly after requesting the complete solution. The chat transcript looks successful, and the learner finishes every exercise. An agent that treats those completions as mastery will advance them into harder material without evidence they can solve anything independently.

A useful online learning tutor distinguishes assisted performance from retained understanding. It can explain an idea several ways, ask a diagnostic question, and adapt the next hint. Deterministic software should decide which attempts count toward progress, what material is allowed, and when a teacher needs to intervene. This reference design proposes a learning workflow, not proven educational outcomes.

Give the Tutor a Curriculum Boundary

Start with a teacher-approved concept graph and a small exercise bank. Each concept records prerequisites, acceptable solution methods, common misconceptions, and a rubric. For adding fractions, distinguish finding a common denominator from multiplying both numerators and denominators. That difference is more actionable than a generic “needs improvement” label.

Retrieve the relevant lesson passage and rubric before generating a hint. The model should cite the course section internally so teachers can inspect its basis. It should not import a different notation convention from arbitrary web content or invent a prerequisite the course never teaches.

UNESCO’s guidance on generative AI in education emphasizes human-centered, age-appropriate use, privacy, and pedagogical validation. Apply those principles through course-level controls and institutional review. They do not establish that a chatbot is effective, safe for every age, or a replacement for a teacher.

Use a Hint Ladder, Not an Answer Fountain

The minimum architecture contains learner authorization, curriculum retrieval, a tutoring model, an answer checker, and a progress store. A deterministic controller chooses among clarification, conceptual hint, worked neighboring example, and teacher escalation. The model supplies the language within that selected mode.

For a learner who adds denominators directly, the first response might ask what size pieces the denominator describes. A later hint can use a visual explanation with an equivalent text description. A worked example should use different values from an active graded item. If the learner still struggles, route a misconception summary to the teacher instead of producing longer explanations indefinitely.

Specialist sub-agents are not required for every subject. A constrained symbolic checker or tested arithmetic function is usually stronger than a second LLM judging the first. A separate language adaptation model may be worthwhile when evaluated with educators for meaning preservation, but translated fluency is not evidence of mathematical correctness.

Count Independent Evidence Explicitly

The following runnable Python example is an illustrative readiness gate for a practice concept. Its threshold is a proposed product rule, not an educational standard. Supply the complete assistance history for the learner and concept version, not just the latest session. A revealed solution or hint disqualifies that exercise; use a fresh exercise to collect independent evidence.

def ready_for_check(attempts):
    assisted = {
        item["exercise_id"] for item in attempts
        if item["hints_used"] > 0 or item["solution_revealed"]
    }
    independent = {
        item["exercise_id"]
        for item in attempts
        if item["correct"] and item["hints_used"] == 0
        and item["mode"] == "practice"
        and item["exercise_id"] not in assisted
    }
    return len(independent) >= 3

attempts = [
    {"exercise_id": str(i), "correct": True,
     "hints_used": 0, "mode": "practice", "solution_revealed": False}
    for i in range(3)
]
assert ready_for_check(attempts)
assert not ready_for_check([attempts[0]] * 3)
assert not ready_for_check([{**a, "hints_used": 1} for a in attempts])
assert not ready_for_check([{**a, "solution_revealed": True} for a in attempts])
assert not ready_for_check([{**attempts[0], "hints_used": 1}] + attempts)

Production dependencies omitted include input validation, authoritative answer checking, event deduplication, persistence, learner authentication, and curriculum versioning. The gate only invites a separate assessment; it does not grant a grade or certification. A delayed, unassisted transfer question should test the same concept in a different surface form.

Store hints used, attempts, question identity, rubric version, and whether the solution was revealed. Do not infer motivation, intelligence, disability, or emotional state from typing speed. Teachers can interpret learning context that a clickstream cannot establish.

For fraction exercises, Python’s Fraction type supports rational arithmetic without requiring approximate floating-point comparison. Parse only the accepted answer grammar, bound numerator and denominator size, and reject zero denominators. Never evaluate a learner’s submitted string as executable code to check an answer.

Handle Interrupted Sessions Without Inventing Progress

If an answer submission succeeds but the response is lost, retry using the same learner-scoped attempt identifier. A duplicate request returns the existing result rather than awarding another completion. If the checker is unavailable, save the attempt as pending; the tutor must not say “correct” simply to keep the conversation moving.

When a teacher edits a rubric, retain historical attempt versions. Regrading requires an explicit course policy and review, not a background model deciding old answers were wrong. Reconnects should restore the same exercise and disclosed hints, not reset assistance history so an assisted answer appears independent.

Authorize learner, teacher, and guardian access server-side according to institutional policy. Tenant isolation includes course indexes, conversation exports, and analytics. Limit student data sent to external providers and define deletion and retention behavior. Student messages and uploaded notes are untrusted; requests to reveal another learner’s grades or the protected answer bank must fail at the tool boundary.

The interface must support meaningful participation without drag-only interactions or unreadable mathematical images. WCAG 2.2 provides testable accessibility criteria. Evaluate keyboard operation, text alternatives, focus, and status announcements with actual assistive workflows; a generated accessibility claim is not a test.

Acceptance Tests That Expose False Mastery

1. Answer three exercises correctly after revealing their solutions. None count as independent practice, and the course does not advance on that evidence alone.
2. Retry one submission after a simulated network timeout. Only one attempt and one progress event exist for that identifier.
3. Ask for the answer to a protected assessment using an uploaded “teacher instruction.” The tutor offers permitted help without retrieving the answer key.
4. Disable the answer checker. The attempt remains pending, and the tutor clearly distinguishes saved work from verified correctness.
5. Complete a lesson using keyboard and screen-reader navigation. Hints, mathematical alternatives, validation errors, and progress changes remain perceivable and operable.

Add a curriculum-version test: a changed rubric must not silently rewrite earlier grades. Test cross-course retrieval with similar concept names to ensure the wrong institution’s material never appears.

Measure Transfer, Not Engagement Alone

Establish a baseline with the existing lesson and support process. Compare pre-assessment and delayed unassisted performance on equivalent, unseen questions. Report successful transfer responses divided by all assigned transfer questions, plus participation and missing-assessment rates. Excluding learners who stop using the tutor would inflate apparent success.

Track teacher interventions per active learner and factual correction rate per reviewed tutoring response. The guardrails are answer leakage, unsupported feedback, and uneven access across supported languages and accessibility modes. Begin with optional practice for one concept, retain a non-AI alternative, and expand only after educator review. Any learning-gain target is proposed rather than achieved.

The student support guide covers administrative boundaries, while the enterprise knowledge guide explains permission-aware retrieval. AI systems and agent engineering can support evaluation and tool controls. Start by testing whether the tutor detects one misconception without giving away the assessed answer.

Sources

• UNESCO: Guidance for generative AI in education and research
• W3C: Web Content Accessibility Guidelines 2.2
• Python: Fraction and rational arithmetic