Skip to main content

StemBlock AI - Detailed Cost Breakdown & ROI Analysis

Note (Feb 2026): The costs in this document are based on December 2025 Mistral pricing. The platform has since migrated to Google Gemini via Vertex AI. See AI_TRAINING_COST_ESTIMATE.md for current costs.

Date: December 21, 2025 Purpose: Financial analysis of AI features for business decisions Audience: Executives, Product Managers, Finance Team

Executive Summary​

Current Monthly AI Costs: $7,125 Projected Annual Spend (Current Features): $85,500 Projected Annual Spend (Full Scale): $8.24M (if all tiers maximize usage) Realistic Projection (50% adoption): ~$4.12M/year

Revenue Impact: Premium AI features justify 2-3 tier upgrades per coach ROI Target: AI costs should be <10% of subscription revenue

1. Current Monthly Spend (December 2025)​

Active Features​

STEM Evaluation Engine​

Volume: 4,800 submissions/month Cost per submission: $0.077

ComponentTokensRateCost
Input (3000)14.4M$0.14/M$2,016
Output (600)2.88M$0.42/M$1,210
Subtotal17.28Mavg $0.20/M$3,226

Monthly Usage Patterns:

  • Week 1-2 (assignment due dates): 30-40% of submissions
  • Week 3-4 (coach review): 40-50%
  • Steady: 20% throughout

Cache Efficiency: ~0% (no identical submissions, cache expires 1 hour) Actual Cost: $3,226/month (no cache savings yet)

English Writing Workflow (Mistral)​

Volume: 1,900 submissions/month Cost per submission: $0.88

ComponentTokensRateCost
Moderation (mistral-small)1.9M$0.07/M$133
Feedback Gen (mistral-large, 1500)2.85M$0.27/M$770
Assessment (mistral-large, 1500)2.85M$0.27/M$770
Subtotal7.59Mavg $0.116/M$1,673

Fee Assumption: Based on TEAM tier coaches (20 coaches × 95 submissions/month) Early Stage: Only ~200 active coaches in TEAM tier using writing feature

Cache Efficiency: ~5% (some similar prompts repeated) Actual Cost: $1,673/month

Parent Insights Generation​

Volume: 1,500 generations/month (cached heavily) Cost per insight: $0.0015

ComponentTokensRateCost
Generation + Output1.5M$0.14/M$210
Subtotal1.5M$0.14/M$210

Caching Strategy:

  • Generated on dashboard load (student profile view)
  • Cached 1 week per student
  • ~500 active parent users × 3 students = 1,500 unique students
  • Dashboard views/week: ~3,000 (but 80% cache hits)
  • Actual new generations: ~600/month

Cache Efficiency: ~85% (heavy caching) Actual Cost: $210/month (after caching benefit, was ~$1,400 without cache)

Coach Feedback Generation (Optional)​

Volume: 4,800 submissions/month × 40% request = 1,920 calls Cost per call: $0.50

ComponentTokensRateCost
Input (3000)5.76M$0.14/M$806
Output (700)1.34M$0.42/M$564
Subtotal7.1Mavg $0.21/M$1,370

Usage: Advanced coaches (40% adoption) request detailed feedback Cache Efficiency: ~0% (each request is unique due to coach notes) Actual Cost: $1,370/month

Current Monthly Summary​

FeatureCost% of Total
STEM Evaluation$3,22645%
English Writing$1,67323%
Coach Feedback$1,37019%
Parent Insights$2103%
TOTAL$6,479100%

Projected Annual (Current Pattern): $6,479 × 12 = $77,748

2. Cost Per User / Cost Per Feature Use​

Pricing Context​

  • COMMUNITY Tier: $0/month (free)
  • TEAM Tier: $29/month per coach
  • ENTERPRISE Tier: $299/month per organization

Cost Allocation by Tier​

STEM Evaluation (4,800 submissions/month):

  • COMMUNITY coaches: 50% × $3,226 = $1,613

    • Avg: 5 coaches × 4,800 submissions = $1,613
    • Cost per coach: $323/month (heavily subsidized)
    • Cost per use: $0.067

  • TEAM coaches: 40% × $3,226 = $1,290

    • Avg: 20 coaches × 40 submissions = $1,290
    • Cost per coach: $65/month
    • Cost per use: $0.065

  • ENTERPRISE: 10% × $3,226 = $323

    • Avg: 10 orgs × 480 submissions = $323
    • Cost per org: $33/month
    • Cost per use: $0.067

Insight: Cost per use is constant (~$0.067), but COMMUNITY coaches use AI 10x more heavily

  • Implication: Free tier is being heavily subsidized by premium tiers

Revenue Per AI Feature​

TEAM Tier Monthly Revenue:

  • 20 coaches × $29 = $580/month
  • STEM AI cost: $1,290/month
  • Margin: -$710 (AI costs exceed subscription revenue for TEAM)

ENTERPRISE Tier Monthly Revenue:

  • 10 orgs × $299 = $2,990/month
  • STEM AI cost (allocated): $323/month
  • Margin: +$2,667 (profitable, 89% gross margin)

Current Profitability:

  • COMMUNITY tier: -$1,613 (unprofitable)
  • TEAM tier: -$710 (unprofitable)
  • ENTERPRISE tier: +$2,667 (very profitable)
  • Net: -$1,656/month (losing money on free + TEAM tiers)

Action Items:

  1. Increase TEAM tier price to $59-79/month
  2. Implement COMMUNITY tier AI quota (5 evals/month max)
  3. Focus on ENTERPRISE conversion (most profitable)

3. Projected Spend at Scale​

Assumption: 50% Adoption Rates (Realistic)​

TierUsersAdoptionMonthly UsesCost/UseMonthly Cost
STEM Evaluation
COMMUNITY500 coaches10%5,000$0.067$335
TEAM200 coaches40%8,000$0.067$536
ENTERPRISE50 orgs50%2,500$0.067$168
TOTAL15,500$0.067$1,039/month
vs. $3,226 today

Analysis​

  • Only 15% of COMMUNITY coaches use STEM AI today (rest ignore it)
  • TEAM tier adoption is 20% of available capacity
  • ENTERPRISE at 25% of available capacity
  • Total AI spend if all tiers adopted at 100%: $22M/month (vs. $7.1K today)

Profitability at Scale (50% Adoption)​

TierRevenue/monthAI Cost/monthMarginMargin %
COMMUNITY$0$335-$335-∞
TEAM$5,800$536+$5,26491%
ENTERPRISE$14,950$168+$14,78299%
NET$20,750$1,039+$19,71195%

Key Insight: At 50% adoption, AI features are extremely profitable

  • COMMUNITY: Still subsidized (needs quota limit)
  • TEAM: Highly profitable if adoption increases to 40%+
  • ENTERPRISE: Extremely profitable

4. Cost Scenarios​

Scenario A: Conservative (10% Adoption)​

  • Monthly AI Cost: $1,500
  • Annual: $18,000
  • User Count: 50 coaches actively using AI
  • Cost per active user: $30/month
  • Status: Early stage, low volume

Scenario B: Moderate (30% Adoption)​

  • Monthly AI Cost: $4,500
  • Annual: $54,000
  • User Count: 150 coaches actively using AI
  • Cost per active user: $30/month
  • Status: Growing adoption, breakeven emerging

Scenario C: Growth (50% Adoption)​

  • Monthly AI Cost: $7,500
  • Annual: $90,000
  • User Count: 250 coaches actively using AI
  • Cost per active user: $30/month
  • Status: Rapid growth, highly profitable for TEAM+

Scenario D: Mature (100% Adoption)​

  • Monthly AI Cost: $15,000
  • Annual: $180,000
  • User Count: 500 coaches actively using AI
  • Cost per active user: $30/month
  • Status: Market leader position, scale economics kick in

5. Cost Reduction Opportunities​

1. Cache Optimization (Quick Win)​

Current State:

  • STEM Evaluation: 1-hour cache, ~0% hit rate
  • Parent Insights: 1-week cache, ~85% hit rate

Opportunity: Extend STEM cache to 7 days + implement semantic similarity

Implementation:

// Cache key includes file content hash, not just submissionId
const cacheKey = hash(files.map(f => f.content).join('|'));

// If same files submitted multiple times, reuse score
const cached = cacheMap.get(cacheKey);
if (cached && isSimilar(files, cached.files)) {
  return cached.result;
}

Projected Savings:

  • Estimate: 30-50% cache hit rate across all submissions
  • Current cost: $6,479/month
  • With caching: $3,239/month
  • Monthly savings: $3,240 (50% reduction)
  • Annual savings: $38,880

Implementation Effort: 4 hours ROI: Immediate

2. Model Optimization (Medium Effort)​

Current: Using Mistral open-mistral-7b for all tasks Optimization: Use smaller, cheaper models for simple tasks

Model Options:

mistral-small (cheaper):    $0.07/$0.21  (vs open-mistral $0.14/$0.42)
mistral-tiny (cheapest):    $0.04/$0.12  (new, not tested)

Use Cases:

  • Moderation → mistral-small/tiny (simple yes/no)
  • Parent Insights → mistral-small (summarization)
  • Complex Analysis → open-mistral-7b (keep for quality)

Projected Savings:

  • 40% of tasks move to mistral-small: $2,591 × 50% = $1,296/month
  • 20% of tasks move to mistral-tiny: $1,296 × 75% = $972/month
  • Total monthly savings: $2,268 (35% reduction)
  • Annual savings: $27,216

Implementation Effort: 8 hours (testing + prompt refinement) ROI: High (35% cost reduction)

3. Batch Processing (Advanced)​

Concept: Group submissions by assignment, evaluate in single prompt

Example:

Current: 30 individual API calls (30 × 1500 tokens = 45K tokens)
Batch: 1 API call with all 30 (45K tokens, but more efficient parsing)
Potential savings: 20-30% fewer output tokens

Projected Savings:

  • Most valuable for COMMUNITY tier (high volume)
  • Estimate: 15-20% cost reduction on output tokens
  • Current output cost: $1,210/month
  • Savings: $181/month
  • Annual savings: $2,172

Implementation Effort: 20 hours (prompt redesign + batch logic) ROI: Medium (low absolute savings for effort)

4. Switch to Claude Haiku for Moderation (Provider Diversification)​

Current: Using Mistral mistral-small for moderation Alternative: Claude claude-3-haiku

Pricing:

  • Mistral small: $0.07/$0.21
  • Claude Haiku: $0.80/$4 per 1M tokens
  • Claude is 11x more expensive per token

However: Claude Haiku handles moderation with 1/3 the tokens

  • Current: 700 tokens per moderation
  • Claude: ~200 tokens per moderation
  • Net cost similar, but Claude more reliable for edge cases

Verdict: Not recommended unless moderation accuracy issues occur

5. Fine-tuning (Long-term)​

Concept: Train custom model on STEMBlock evaluation data

Process:

  1. Collect 1000+ evaluation examples (current data)
  2. Fine-tune Mistral on STEM evaluation patterns
  3. Deploy custom model (slightly higher cost initially, but better quality)

Projected Benefits:

  • Better evaluation quality (+15-20% coach satisfaction)
  • Fewer tokens needed (-20-30% due to domain-specific training)
  • Better confidence scores (more reliable)

Projected Savings:

  • Token reduction: 20% × $6,479 = $1,296/month
  • Fine-tuning cost: ~$5,000 one-time + $500/month hosting
  • Net monthly savings: $796 (after fine-tuning amortization)
  • Annual net savings: $9,552

Implementation Effort: 40+ hours (data collection, fine-tuning, testing) ROI: Medium (1-year payback) Timeline: Q2 2026 (after 6 months of production data)

6. Cost Optimization Roadmap​

Q1 2026 (Immediate)​

  • Implement 7-day cache + semantic similarity
    • Savings: $38,880/year
    • Effort: 4 hours

Q2 2026 (Quick Wins)​

  • Switch to model tiering (small/tiny for simple tasks)
    • Savings: $27,216/year
    • Effort: 8 hours
  • Implement batch processing for high-volume tasks
    • Savings: $2,172/year
    • Effort: 20 hours

Q3 2026 (Scaling)​

  • Add token-level cost tracking
    • Benefit: Visibility into spending
    • Effort: 16 hours
  • Benchmark Claude vs. Mistral quality
    • Benefit: Know when to switch
    • Effort: 8 hours

Q4 2026 (Optimization)​

  • Collect fine-tuning data from 6 months production
    • Benefit: Foundation for custom model
    • Effort: 4 hours (automated)

Q1 2027 (Advanced)​

  • Deploy fine-tuned Mistral model
    • Savings: $9,552/year
    • Effort: 40+ hours

7. Budget Projections​

Q1 2026 (Current Spend + Growth)​

  • Coaches: 50 (growing from 20)
  • Estimated cost: $7,500/month
  • Annual burn: $90,000

Q2 2026 (After Cache + Tiering Optimization)​

  • Coaches: 100 (growing)
  • Pre-optimization: $12,000/month
  • Post-optimization: $7,500/month (38% reduction)
  • Annual burn: $90,000

Q3 2026 (Continued Growth)​

  • Coaches: 150
  • Cost: $8,500/month (growth outpaces savings)
  • Annual burn: $102,000

Q4 2026 (Scale)​

  • Coaches: 200
  • Cost: $12,000/month
  • Annual burn: $144,000

Full Year 2026 Budget​

  • Q1: $22,500
  • Q2: $22,500 (post-optimization)
  • Q3: $25,500
  • Q4: $36,000
  • Total 2026: $106,500

Note: This assumes continued growth without major pricing changes

8. Revenue Impact Analysis​

AI Features Drive Premium Tier Adoption​

Survey Data (Hypothetical):

  • 30% of TEAM conversions cite "AI Evaluation" as key feature
  • 50% of ENTERPRISE conversions cite "AI Evaluation" + "English Writing"

TEAM Tier Conversion Value​

  • Price: $29/month
  • Margin (after AI costs): $20/month (after $0.067 per eval × 135 evals avg)
  • Lifetime value (3 years): $720
  • Conversion probability (with AI features): 15%
  • Incremental revenue per coach acquired: $108

ENTERPRISE Tier Conversion Value​

  • Price: $299/month
  • Margin (after AI costs): $290/month (economies of scale)
  • Lifetime value (3 years): $10,440
  • Conversion probability (with AI features): 8%
  • Incremental revenue per school acquired: $835

AI Feature ROI​

  • Current AI spend: $7,500/month
  • Incremental revenue (30% attribution): $2,250/month
  • Net AI contribution: -$5,250/month (negative)

Challenge: Not enough premium tier adoption yet to justify AI costs

Solution:

  1. Increase AI feature visibility (dashboard prominence)
  2. Require TEAM tier for advanced AI features (Coach Feedback, Batch Grading)
  3. Market "AI-Powered Evaluation" more heavily

9. Break-even Analysis​

What Adoption Rate Makes AI Profitable?​

Assumptions:

  • Average coach: $29/month (TEAM tier)
  • AI cost per coach: $30-50/month (based on usage)
  • Need: Coach tier revenue > AI costs

Formula:

Break-even coaches = (Monthly AI spend) / (Revenue per coach - AI cost per coach)
Break-even coaches = $7,500 / ($29 - $40) = Not feasible at current TEAM price!

Break-even with Pricing Adjustment:

If TEAM = $59/month:
Break-even coaches = $7,500 / ($59 - $40) = 395 coaches

If TEAM = $79/month:
Break-even coaches = $7,500 / ($79 - $40) = 192 coaches

Current coaches on TEAM: 20 (below break-even)

Implication:

  • TEAM pricing needs to increase to $59-79/month to be sustainable
  • OR implement COMMUNITY quota limits (max 5 evals/month)

Immediate (This Month)​

  1. Implement 7-day cache for STEM evaluations

    • Target: 35-50% cache hit rate
    • Expected savings: $38,880/year
    • Effort: 4 hours

  2. Set COMMUNITY tier quota to 10 submissions/month

    • Target: Reduce free tier subsidy
    • Expected savings: $1,500/month (from COMMUNITY tier)
    • Effort: 2 hours

  3. Increase TEAM tier pricing to $49/month (from $29)

    • Target: Improve margin on AI features
    • Expected revenue impact: +$400/month per active coach
    • Effort: 1 hour (change in billing system)

Short-term (Next 2 Months)​

  1. Implement model tiering (small models for simple tasks)

    • Target: 25-35% cost reduction
    • Expected savings: $27,216/year
    • Effort: 8 hours

  2. Add cost monitoring dashboard (admin view)

    • Target: Visibility into AI spending
    • Effort: 16 hours

Medium-term (Q2 2026)​

  1. Benchmark Claude vs. Mistral

    • Test on English Writing workflow
    • Decide on multi-model strategy
    • Effort: 8 hours + 2 weeks analysis

  2. Launch "AI Evaluation" marketing campaign

    • Highlight AI confidence, coach feedback, parent insights
    • Target: +50% premium tier adoption
    • Expected revenue: +$5,000/month

Summary Table​

OptimizationEffortMonthly SavingsAnnual SavingsPayback
7-day Cache4h$3,240$38,880Immediate
Model Tiering8h$2,268$27,216Immediate
COMMUNITY Quota2h$1,500$18,000Immediate
Batch Processing20h$181$2,172120 days
Fine-tuning40h$796$9,55212 months
TOTAL74h$7,985$95,8201 week avg

Combined Impact:

  • Current cost: $7,500/month
  • After all optimizations: $0 (pay for itself)
  • Annual savings: $95,820
  • ROI: 1,290% (on 74 hours of engineering time)

Document Version: 1.0 Last Updated: December 21, 2025 Status: Ready for Finance Review & Implementation Planning Next Review: Monthly (until optimizations complete)