Higher Ed Ops Consulting

← All posts

What Building a Free Interview Tool Taught Me About AI and Expertise

Three people practicing for job interviews on a phone, laptop and tablet, connected to one central tool

A few months ago, several talented people in my network were laid off, and I wanted to help in the way I know best: I built something. With Claude as my build partner, I created a free interview practice tool grounded in every interview best practice I’ve learned over my career.

Here’s what surprised me most: the build itself did not take up most of the time (shout out to Claude!); most of my hours went into two things — writing out the context so the AI understood exactly who the tool was for, and validating every output against what I know a strong interview actually looks like.

Since launching it, I’ve learned even more from the people using it. Here’s what the tool does, and four lessons from building it that apply far beyond interview prep.

What the tool does

The tool works from three inputs: your résumé, the job description, and background on the people interviewing you. With those, it creates:

  • Interviewer profiles — a cheat sheet on each interviewer, the questions they’re likely to ask, and what to emphasize given your background
  • Scored practice — a mock interview where you type or record your answers and get a score plus specific ways to strengthen them
  • A fit analysis — how well you match the role, your strengths and gaps, what to highlight, and keywords to add to your résumé

Everything stays on the user’s device. I see none of it, and that was a design requirement from day one.

Lesson 1: Context does the heavy lifting

The quality of what AI builds depends on how well you explain the problem. Before writing a single feature, I spent hours describing who the tool was for, what a great interview looks like, and what a nervous candidate needs in the moment. The clearer that picture became, the better every piece of the tool got. When people ask me how to get more out of AI, this is almost always where I start.

Lesson 2: Validation is the job

AI hands you smart-sounding, legit-looking answers. Without the expertise to validate them, you can confidently ship something that, as I’ve said before, is subpar at best and inaccurate at worst. I tried to test every interview question, every score and every piece of feedback against what I know from years of hiring and coaching. That review took longer than the build, and it’s the reason I’m comfortable sharing the tool with people who are counting on it.

Lesson 3: Design for every user, starting with the ones least like you

The most useful feedback I received was also the most humbling: the tool leaned too heavily toward executive roles. The interview questions reflected the kinds of interviews I’ve had, and they missed what candidates for front-line positions actually face. So I went back and built in a broader range of question types for different roles and career stages.

It was a good reminder that every builder carries their own experience into the work. The fix is to put the tool in front of people whose experience differs from yours, early and often.

Lesson 4: Let people choose their own path

I originally designed a guided experience that walked everyone through every step, start to finish. Then I heard from users who just wanted to jump straight into practice, or go directly to the résumé and fit analysis, without the full setup. So I added options: a quick two-minute setup, the full guided experience, and a sample walkthrough so people can see how it works before they commit.

Letting each person choose the path that fit their moment made the tool far more useful, and far more likely to be used.

Why this matters for education teams

Every one of these lessons applies to the AI work happening on campuses right now. The people best positioned to put AI to work are the ones who already know the work — just technical enough to be dangerous, and enough of a subject-matter expert to solve the right problems. Your advisors, registrars and program staff already hold that expertise.

And as lessons 3 and 4 show, a tool only creates value when the people it serves actually use it. That’s why I build with users from the start and treat adoption as part of the design.

If you want to see exactly how I approach every AI project, I broke down my method in Stop Boiling the Ocean: My 4-Step Method for Working with AI. And if you know someone preparing for an interview, the tool is free to use and share.

If your team is ready to build tools your people will actually use, let’s talk.

Want this for your team? I help education organizations put AI to work with training, workflow automation and custom tools. Book a free 30-minute call

Get new posts by email

Practical notes on AI, workflows and adoption for education teams. New posts weekly. Unsubscribe anytime.