Conversational AI demo: testing my co-host after 'AI slop'

I went back and forth for ages trying to get the whole AI co-host thing right when I first started Underway. This was a while ago now, and honestly I thought I'd cracked it at one point. Then I left it alone for a stretch, not recording any episodes with the co-host at all.

The idea is simple to say and hard to get right: it reads what you've done, your commits, your calendar, your sales, and asks you about it out loud like a real person would, one question at a time. You talk for a few minutes and an episode comes out numbered and ready. Getting the voice of that interviewer to feel like a person rather than a script took longer than I expected.

My good friend Alessandro is someone I introduced to Underway, and I had him use it the way anyone else would, cold, no guidance from me. Unfortunately it was just absolutely terrible. He ran into what he called AI slop, bad questions and over-the-top emotional responses. Oh, well done. Yeah, I see. I understand. That kind of thing.

He told me afterwards it's AI slop, it's really bad, it doesn't know anything, and it makes the product look bad.

The specific complaint that came up most in my own session with Cedar was it jumping in too quickly. I quite like where you go with the mm-hmm, I said, because it says you're listening. But you do jump over the top of me quite quickly, don't you. Cedar's own answer was I don't mean to, I'll hold back.

That interruption problem is a different fault from slop. Slop is saying the wrong thing. Talking over someone is not waiting for the right moment to say anything at all. Both were present in how the co-host behaved before this test, and both needed fixing separately.

Sitting with Cedar again, I asked it directly: what was episode 40 about? It answered correctly and specifically, that episode was on 19 September, about adding WhatsApp and Slack bots that nudge you to record, a proactive assistant that spots meaningful events, messages you, helps shape the episode, then opens the recording setup. That's a real lookup against real episode history, not a guess dressed up as confidence.

I told Cedar straight: I think the fact I'm speaking to you is the next thing. You seem to be at least improved, a bit more human-like in your responses. That's the bar I care about, not whether it sounds impressive, but whether it sounds like it's actually listening and actually knows what it's talking about.

I also asked whether it had other accents or voices. It doesn't have a list to switch between, and its voice doesn't change to other accents. That's a plain limit worth saying out loud rather than working around in the demo.

AI slop isn't one thing. In Alessandro's case it showed up as two separate failures: answers that were hollow and overly enthusiastic, and a co-host that spoke over him before he'd finished a thought. Fixing the tone of the words doesn't fix the timing of when they're said, and fixing the timing doesn't fix whether the content is specific or generic.

The test that actually surfaces this is handing the product to someone who has no reason to be kind about it and no instructions from you. I used it as anyone else would, is how I put it. If you only ever demo your own AI feature yourself, you'll talk around its gaps without noticing. Alessandro didn't have that instinct, which is exactly why his run was useful.

If you're building or evaluating something like this, there are a few specific things worth testing rather than just listening for how natural it sounds. Does it answer a factual question about your own history correctly, the way Cedar named the date and content of episode 40. Does it interrupt, or does it wait. Does it know its own limits and say so plainly, as Cedar did on accents, rather than inventing an answer.

Who this suits: founders who want a spoken record of their day without facing a camera, using a conversation instead of a monologue. Who it doesn't suit yet: anyone expecting the co-host to carry a whole interview unsupervised on a first cold run, because that's exactly where slop showed up for Alessandro.