AI Can Grade the Transcript—Humans Grow the Coach with Doug Foltz

Doug Foltz explains how he used AI to solve a real coach-development bottleneck: mentor coaching doesn't scale. By building a competency rubric and an AI "agent" that evaluates coaching transcripts, Doug's team reduced hours of expert analysis to minutes—then re-centered the human work where it matters most: reflection, agency, and a short mentor-coaching conversation. The bigger idea: "communal co-intelligence"—AI not just as a personal assistant, but as a tool that helps a whole coaching community preserve culture, build consistency, and scale development without losing what makes coaching human.

Episode description

How do you scale mentor coaching when you don't have the budget—or the hours? Doug Foltz (Content Engineering & Value Alignment Lead at Gloo, DMin candidate at Asbury, and longtime church-planting coach) shares how he built an AI-supported mentor-coaching loop: a detailed competency rubric + an AI evaluator that reviews transcripts in minutes. But Doug also warns about a hidden danger: AI can bypass reflection, which is essential for adult learning. So they intentionally added "friction" back into the process—reflection first, then AI feedback, then a short human coaching conversation. Along the way, Doug introduces a powerful concept: communal co-intelligence—AI that strengthens a community's shared language, values, and coaching culture.

Key moments (timestamps)

• 0:02–1:20 – Who Doug is + why Brian calls him the "AI guy"

• 1:49–3:21 – The real problem: coaching training doesn't stick without mentor coaching

• 3:34–5:06 – Doug's solution: a rubric + AI agent that evaluates transcripts (levels 1–3)

• 6:44–8:15 – The twist: reflection is essential; AI can accidentally remove it

• 8:28–9:00 – The human loop: 15–20 minute mentor conversation after reflection + report

• 10:38–14:35 – Why AI matters: replaces 3–4 hours of expert analysis with minutes

• 15:04–16:15 – The church's role: protect what's uniquely human; set boundaries

• 16:27–19:16 – "Communal co-intelligence": AI + a coaching community's culture and standards

• 21:24–23:00 – What they observed: fast growth from Level 1 → Level 2; harder jump to Level 3

• 23:29–25:46 – Craft guild model: learn the fundamentals, then innovate without losing the core

• 28:57–31:14 – What's next: agentic systems, tools + data access, and AI as "work orchestrator"

Key ideas

• AI can scale mentor coaching by doing the transcript evaluation quickly and consistently.

• Reflection is non-negotiable in adult learning; AI can "steal" it by doing the thinking for you.

• The solution is intentional friction: reflection → AI feedback → short human mentor coaching.

• Agency matters: don't make AI the all-knowing guru; keep the learner's authority intact.

• Communal co-intelligence: AI can reinforce a shared coaching culture across many coaches.

• Early gains can be rapid (novice → intermediate), but advanced mastery takes longer.

• The future is agentic systems that combine tools + data + context to orchestrate real work.

Quotable lines (pull quotes)

• "We really can't scale coaching very well."

• "Mentor coaching is what makes the training stick."

• "My process actually bypasses [reflection] entirely."

• "We added a friction point… and we made them reflect."

• "You don't want the AI to be the all-knowing guru."

• "That's the part of the process that we said, we're going to replace." (re: 3–4 hours of evaluation)

• "Communal co-intelligence… it's the AI with our coaching community."

• "It becomes this orchestrator of work within an organization."

Discussion questions (for Learning Lab / staff meeting)

1. Where would AI help us scale without compromising what we value most?

2. What part of our development process must remain human-only?

3. Where might AI accidentally remove reflection, struggle, or ownership?

4. What would a "reflection-first" workflow look like for our coaches or trainers?

5. What are the risks of communal AI (shared culture) becoming static or overly controlling?

6. If AI becomes an "orchestrator of work," what data is off-limits—and why?

Practical takeaway

AI is best used as a leverage tool—not a replacement for learning. Let it do the heavy lift of analysis and pattern recognition, then spend your human time where it counts: reflection, discernment, presence, and coaching conversations that build ownership and growth. If you design it well, AI doesn't dilute your culture—it can actually help you scale it.