Vibe Engines
YouTube

Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path

Learn RustChapter 10: Streaming replies

Lesson 10.5 · 8 min

Chapter review: streaming replies

Chatterbox v1.1 shows an answer as the model writes it, and Ctrl-C stops a long one without losing the conversation. This last chapter review mixes everything that made that work: futures, tasks, channels, bytes cut in awkward places and select!. Eight predictions, then one exercise that puts the whole streaming path in a single function.

Adds to ChatterboxNothing new: a mixed quiz on the whole chapter, and a bigger exercise that uses all of it.

What this chapter gave Chatterbox

Four lessons turned a program that froze while it waited into one that waits for several things at once. The ideas, one line each:

  • Lesson 10.1: an async fn hands back a future that does nothing until it is awaited; tokio runs futures, tokio::join! drives several at once, and code inside async must never block.
  • Lesson 10.2: tokio::spawn(async move { … }) starts a task that owns what it uses, and a channel carries its pieces: recv gives None once every Sender is gone, and send fails once the Receiver is gone.
  • Lesson 10.3: the network delivers chunks of bytes, not lines, so bytes wait in a Vec<u8> until a newline makes a whole line, and only then become text.
  • Lesson 10.4: tokio::select! waits for whichever comes first and drops the rest, Ctrl-C becomes a future, and dropping a Receiver is all it takes to stop the task feeding it.

Nothing new appears on this page. If a question stumps you, its answer names the lesson to reread.

Mixed quiz

Eight small programs, each run for real when the course was built. They hop between the four lessons on purpose, so the first job is working out which idea a program is about.

PredictSave on the way out
async fn save(history: &str) {
println!("saving {history}");
}
 
#[tokio::main]
async fn main() {
let _ = save("3 messages");
println!("bye, Sam!");
}

What does this program print?

  • bye, Sam!
  • saving 3 messages bye, Sam!
  • bye, Sam! saving 3 messages
Reveal the answer

The program prints: bye, Sam! Calling save only makes a future, a recipe for the work, and nothing ever awaits it, so it is thrown away uncooked and saving never prints. A forgotten .await is not an error; the work silently never happens. Lesson 10.1 has the recipe-and-meal picture.

PredictA task that reads the banner
#[tokio::main]
async fn main() {
let banner = String::from("Chatterbox v1.1");
let task = tokio::spawn(async {
println!("{banner}");
});
task.await.unwrap();
}

Does it compile?

  • It compiles and runs
  • Compile error E0373
  • Compile error E0382
Reveal the answer

It does not compile: E0373. A spawned task may run on after the code that started it is gone, so tokio will not let it borrow anything. Here the block only borrows banner, and the compiler says the async block may outlive the function. async move hands the task its own banner and it compiles (lesson 10.2).

error[E0373]: async block may outlive the current function, but it borrows `banner`, which is owned by the current function
 --> src/main.rs:4:29
  |
4 |     let task = tokio::spawn(async {
  |                             ^^^^^ may outlive borrowed value `banner`
5 |         println!("{banner}");
  |                    ------ `banner` is borrowed here
  |
  = note: async blocks are not executed immediately and must either take a reference or ownership of outside variables they use
help: to force the async block to take ownership of `banner` (and any other referenced variables), use the `move` keyword
  |
4 |     let task = tokio::spawn(async move {
  |                                   ++++
PredictCount the data lines
fn main() {
let chunks = ["data: Hi\n", "\nda", "ta: [DONE]\n"];
let mut pending: Vec<u8> = Vec::new();
let mut data_lines = 0;
for chunk in chunks {
pending.extend_from_slice(chunk.as_bytes());
while let Some(end) = pending.iter().position(|&b| b == b'\n') {
let line: Vec<u8> = pending.drain(..=end).collect();
if String::from_utf8_lossy(&line).starts_with("data:") {
data_lines += 1;
}
}
}
println!("data lines: {data_lines}, waiting: {} bytes", pending.len());
}

Three chunks arrive. What does this print?

  • data lines: 2, waiting: 0 bytes
  • data lines: 3, waiting: 0 bytes
  • data lines: 1, waiting: 12 bytes
Reveal the answer

The program prints: data lines: 2, waiting: 0 bytes Three newlines make three lines: data: Hi, a blank line, and data: [DONE], whose two halves were glued back together in pending before anyone read them. The blank line is not a data line, so the count is 2, and nothing is left waiting. Lines come from newlines, never from chunks (lesson 10.3).

PredictWait inside a helper
use std::time::Duration;
 
async fn pretend_reply(message: &str) -> String {
tokio::time::sleep(Duration::from_millis(10)).await;
format!("you said: {message}")
}
 
fn reply_twice(message: &str) -> String {
let first = pretend_reply(message).await;
format!("{first} / {first}")
}
 
#[tokio::main]
async fn main() {
println!("{}", reply_twice("hi"));
}

Does it compile?

  • It compiles and runs
  • Compile error E0728
  • Compile error E0308
Reveal the answer

It does not compile: E0728. .await means "pause here", and only async code has somewhere to pause to. reply_twice is a plain fn, so the compiler refuses, the same error as a plain main in lesson 10.1. Make reply_twice async too, and await it in main.

error[E0728]: `await` is only allowed inside `async` functions and blocks
 --> src/main.rs:9:40
  |
8 | fn reply_twice(message: &str) -> String {
  | --------------------------------------- this is not `async`
9 |     let first = pretend_reply(message).await;
  |                                        ^^^^^ only allowed inside `async` functions and blocks
PredictA stop button nobody holds
use std::time::Duration;
use tokio::sync::mpsc;
 
async fn first_event(mut pieces: mpsc::Receiver<String>, mut stop: mpsc::Receiver<()>) -> String {
tokio::select! {
piece = pieces.recv() => format!("got {piece:?}"),
_ = stop.recv() => String::from("stopped"),
}
}
 
#[tokio::main]
async fn main() {
let (tx, pieces) = mpsc::channel(8);
let (press, stop) = mpsc::channel(1);
drop(press); // oops: now nobody can press stop
tokio::spawn(async move {
tokio::time::sleep(Duration::from_millis(200)).await;
let _ = tx.send(String::from("Hello")).await;
});
println!("{}", first_event(pieces, stop).await);
}

The answer arrives after 200 ms. What does this print?

  • stopped
  • got Some("Hello")
  • got None
Reveal the answer

The program prints: stopped With its only Sender dropped, stop.recv() can never get a press, so it finishes at once with None. select! takes whichever arm finishes first, and the pattern _ accepts None too, so it prints stopped although nobody pressed anything. That is why the tests in lesson 10.4 keep a _press alive.

PredictJoined: who comes first?
use std::time::Duration;
use tokio::time::sleep;
 
async fn fetch(name: &str, ms: u64) -> String {
sleep(Duration::from_millis(ms)).await;
println!("{name} done");
format!("{name} answer")
}
 
#[tokio::main]
async fn main() {
let (a, b) = tokio::join!(fetch("slow", 300), fetch("fast", 50));
println!("{a}, then {b}");
}

The slow answer takes 300 ms, the fast one 50 ms. What is the LAST line this program prints?

  • slow answer, then fast answer
  • fast answer, then slow answer
  • slow done
Reveal the answer

The program prints: slow answer, then fast answer Both waits run at once, so fast done prints first and slow done second. But tokio::join! hands back its results in the order you wrote the futures, not the order they finished, so a is the slow answer (lesson 10.1).

PredictThe thread-style send inside async
use tokio::sync::mpsc;
 
#[tokio::main]
async fn main() {
let (tx, mut rx) = mpsc::channel(8);
tx.blocking_send(String::from("hello")).unwrap();
println!("{:?}", rx.recv().await);
}

It compiles. What happens when it runs?

  • It compiles and runs
  • It compiles, then panics
  • Compile error E0728
Reveal the answer

It compiles, then panics at run time. blocking_send waits the plain way, which is right on the keyboard thread of lesson 10.4 and wrong inside async code, where waiting the plain way would hold up every other task on that thread. The compiler cannot see the difference, so tokio checks while the program runs and panics. Inside async, use send(...).await.

PredictTwo senders, one pipe
use tokio::sync::mpsc;
 
#[tokio::main]
async fn main() {
let (tx, mut rx) = mpsc::channel(8);
let tx2 = tx.clone();
tokio::spawn(async move {
let _ = tx.send("from the model").await;
});
tokio::spawn(async move {
let _ = tx2.send("from the keyboard").await;
});
let mut count = 0;
while let Some(_piece) = rx.recv().await {
count += 1;
}
println!("{count} pieces, then the channel closed");
}

What does this program print?

  • 2 pieces, then the channel closed
  • 1 pieces, then the channel closed
  • 0 pieces, then the channel closed
Reveal the answer

The program prints: 2 pieces, then the channel closed Cloning a Sender gives the pipe a second way in; that is the multiple producer in mpsc. The channel closes only when every Sender is gone, so the first task finishing does not end the loop. Both items arrive, then recv gives None (lesson 10.2).

Milestone: the streaming path in one function

In Chatterbox the stream crosses three places: the real model's task reads chunks, turns whole lines into pieces and sends them down a channel, and answer() races those pieces against Ctrl-C. Here all of it sits in one function, relay, so a test can feed it chunks and press its stop button. parse_event from lesson 10.3 is already written; the rest is yours.

The exercise runs on the Rust Playground, which has tokio and serde. The hidden tests never wait on a clock: every step is a channel handing something over, so they give the same answer on any machine.

Try itMilestone: relay a streamed answer

Write two bodies. whole_lines takes every finished line out of the waiting bytes and leaves the unfinished rest behind. relay loops on a tokio::select! between the next chunk and a press on stop: add each chunk to its own waiting bytes, read whole lines with parse_event, send each Piece to out, and return the right Ending. The hidden tests cut a letter in half, end a stream with and without [DONE], press stop before and during an answer, and walk away from out.

Write two bodies. whole_lines takes every finished line out of the waiting bytes and leaves the unfinished rest behind. relay loops on a tokio::select! between the next chunk and a press on stop: add each chunk to its own waiting bytes, read whole lines with parse_event, send each Piece to out, and return the right Ending. The hidden tests cut a letter in half, end a stream with and without [DONE], press stop before and during an answer, and walk away from out.

use serde::Deserialize;
use tokio::sync::mpsc;
 
// One streamed piece, as the chat API sends it (lesson 10.3).
#[derive(Deserialize)]
struct Chunk {
choices: Vec<ChunkChoice>,
}
 
#[derive(Deserialize)]
struct ChunkChoice {
delta: Delta,
}
 
#[derive(Deserialize)]
struct Delta {
content: Option<String>,
}
 
/// What one line of the stream means.
#[derive(Debug, PartialEq)]
enum Event {
Piece(String),
Done,
Other,
}
 
/// Already written: the parse_event you wrote in lesson 10.3.
fn parse_event(line: &str) -> Event {
let data = match line.strip_prefix("data:") {
Some(rest) => rest.trim(),
None => return Event::Other,
};
if data == "[DONE]" {
return Event::Done;
}
let chunk: Chunk = match serde_json::from_str(data) {
Ok(chunk) => chunk,
Err(_) => return Event::Other,
};
match chunk.choices.into_iter().next() {
Some(choice) => match choice.delta.content {
Some(text) if !text.is_empty() => Event::Piece(text),
_ => Event::Other,
},
None => Event::Other,
}
}
 
/// How a relayed answer ended.
#[derive(Debug, PartialEq)]
enum Ending {
/// data: [DONE] arrived.
Done,
/// The chunks ran out without a [DONE].
Closed,
/// Someone pressed stop.
Stopped,
/// Nobody is listening for the pieces any more.
Gone,
}
 
/// Take every whole line out of `pending`, oldest first, as trimmed text.
/// Bytes after the last newline stay in `pending`, waiting for the next chunk.
fn whole_lines(pending: &mut Vec<u8>) -> Vec<String> {
todo!("split the whole lines off {pending:?}")
}
 
/// Relay one streamed answer. Chunks of bytes (cut anywhere, even inside a
/// letter) come in on `chunks`; every piece of text goes out on `out` as
/// soon as its line is whole. Keep going until data: [DONE] (Ending::Done),
/// the chunks run out (Closed), a press arrives on `stop` (Stopped), or
/// sending a piece fails because nobody is listening (Gone).
async fn relay(
mut chunks: mpsc::Receiver<Vec<u8>>,
out: mpsc::Sender<String>,
mut stop: mpsc::Receiver<()>,
) -> Ending {
todo!("wait for a chunk or a press, again and again")
}
 
#[tokio::main]
async fn main() {
let (feed, chunks) = mpsc::channel(64);
let (out, mut pieces) = mpsc::channel(16);
let (_press, stop) = mpsc::channel(1); // kept, never pressed
let stream = "data: {\"choices\":[{\"delta\":{\"content\":\"Hello\"}}]}\n\ndata: {\"choices\":[{\"delta\":{\"content\":\" from the café\"}}]}\n\ndata: [DONE]\n\n";
// A pretend server. .chunks(7) cuts the bytes into slices of 7, like an unkind network.
tokio::spawn(async move {
for part in stream.as_bytes().chunks(7) {
if feed.send(part.to_vec()).await.is_err() {
return;
}
}
});
let relaying = tokio::spawn(relay(chunks, out, stop));
while let Some(piece) = pieces.recv().await {
print!("{piece}");
}
println!();
println!("ended: {:?}", relaying.await.unwrap());
}

Hidden tests (8): whole_lines_never_breaks_a_letter whole_lines_keeps_an_unfinished_line nobody_listening_is_gone stop_before_anything_arrives a_letter_cut_in_half_arrives_whole a_stream_that_just_ends_is_closed pieces_arrive_in_order_and_done_ends_it stopping_keeps_what_was_shown_and_lets_the_feed_go

This exercise runs real Rust in your browser through the official Rust Playground, so it needs JavaScript.

The end of the course

That was the last page. You started with "what is a program" and finished with a function that reads a stream cut in random places, keeps every letter whole, and stops cleanly when told to, all of which you can now explain line by line. Pick one of the next steps at the end of lesson 10.4 (a /clear command or a timeout arm is a good first one), then keep going with the tokio tutorial and the Rust Book. Thank you for building Chatterbox.

Self-check

Say it out loud, then reveal

Self-checkRetrieval beats re-reading
What happens to an <code>async fn</code> you call but never await?Reveal
Nothing. Calling it only makes a future; its body runs when the future is awaited, so an unawaited future is dropped without doing any work.
Why does <code>tokio::spawn</code> need an <code>async move</code> block?Reveal
The task may keep running after the code that started it has finished, so it cannot borrow anything from there. move hands it its own values; without it the compiler stops with E0373.
Why does a streaming reader keep bytes in a <code>Vec&lt;u8&gt;</code> instead of turning each chunk into text?Reveal
A chunk can end in the middle of a line, or in the middle of a letter that takes several bytes. Only a whole line, cut at a newline, is sure to hold whole letters.
Two arms of a <code>tokio::select!</code>: what happens to the one that does not finish first?Reveal
It is dropped and takes no more steps. For a channel recv() nothing is lost: an item on its way stays in the channel for the next call.
How does stopping an answer stop the task that was producing it?Reveal
Returning drops the Receiver, which closes the channel. The task's next send returns an error, and it returns, dropping everything it owned.
Your Chatterbox so far: v1.1 (after lesson 10.4)
Chatterbox v1.1 after lesson 10.4: the whole program
src/main.rs
// Chatterbox: a small chat program, built one lesson at a time.
mod command;
mod message;
mod replier;
mod store;
 
use command::{Command, parse};
use message::Message;
use replier::{Echo, Model, Pretend, Replier};
use std::collections::HashMap;
use std::io::{self, Write};
use std::thread;
use tokio::sync::mpsc;
 
/// What the bot is called. A constant: fixed for the whole program.
const BOT: &str = "Chatterbox";
/// The version as a pair of numbers: (major, minor).
const VERSION: (u32, u32) = (1, 1);
/// The longest message we accept, in bytes.
const MAX_LEN: usize = 200;
/// What /help prints.
const HELP: &str = "commands: /help, /again, /history, /count, /stats, /remember <name> <fact>, /recall <name>, /find <word>, /last <n>, /model <echo|pretend|real>, /say <text>, /shout <text>, /debug <text>, /quit (or /h, /q, /exit)";
 
/// The first line Chatterbox prints.
fn banner() -> String {
format!("{} v{}.{}", BOT, VERSION.0, VERSION.1)
}
 
/// The greeting, with a name filled in.
fn greeting(name: &str) -> String {
let mut text = format!("Hi {name}! I'm {BOT}.");
text.push_str(" Ask me anything.");
text
}
 
/// The last `n` items of any slice: messages, numbers, words, anything.
/// `<T>` says "for any type T"; the body never needs to know which one.
fn last_n<T>(items: &[T], n: usize) -> &[T] {
let start = items.len().saturating_sub(n);
&items[start..]
}
 
/// Print an answer as its pieces arrive, and give back the whole of it
/// (None if it went wrong: what went wrong has been printed instead).
async fn answer(replier: &dyn Replier, history: &[Message]) -> Result<Option<String>, io::Error> {
let mut pieces = replier.reply(history);
let mut whole = String::new();
print!("bot: ");
loop {
// Wait for whichever comes first: the next piece, or Ctrl-C.
tokio::select! {
piece = pieces.recv() => match piece {
Some(Ok(text)) => {
print!("{text}");
io::stdout().flush()?; // show it now, not at the end of the line
whole.push_str(&text);
}
Some(Err(e)) => {
println!("(that went wrong: {e})");
return Ok(None);
}
None => break, // the channel closed: the answer is complete
},
_ = tokio::signal::ctrl_c() => {
// Returning drops `pieces`; the task sending them sees that and stops.
print!(" [stopped]");
break;
}
}
}
println!();
Ok(Some(whole))
}
 
/// The keyboard gets a thread of its own. Reading a line blocks until Enter,
/// and a blocked read cannot be called off, so it must not hold anything else
/// up. Each line is sent on the channel; the channel closes at end of input.
fn keyboard() -> mpsc::Receiver<String> {
let (tx, rx) = mpsc::channel(16);
thread::spawn(move || {
for line in io::stdin().lines() {
let line = match line {
Ok(line) => line,
Err(_) => break, // the keyboard could not be read
};
if tx.blocking_send(line).is_err() {
break; // the program has stopped listening
}
}
});
rx
}
 
// #[tokio::main] starts tokio's runtime, the part that runs async code, and
// then runs this async main inside it.
#[tokio::main]
async fn main() -> Result<(), io::Error> {
let mut messages_seen: u32 = 0;
let quiet = false;
let mut last: Option<String> = None; // the last thing you said, for /again
let mut history: Vec<Message> = store::load(store::FILE)?; // last time's conversation, if any
let mut facts: HashMap<String, String> = HashMap::new(); // name -> fact, for /recall
// Box<dyn Replier>: "some Replier, decided while the program runs".
// With a key in the environment, start with the real model.
let mut replier: Box<dyn Replier> = match Model::from_env() {
Ok(model) => Box::new(model),
Err(_) => Box::new(Echo { max_len: MAX_LEN }),
};
 
println!("{} (replies by: {})", banner(), replier.name());
let mut keys = keyboard();
println!("What should I call you?");
let name_line = keys.recv().await.unwrap_or_default();
let name = name_line.trim();
if !quiet {
println!("{}", greeting(name));
}
if !history.is_empty() {
println!(
"(I remember {} messages from last time. /history shows them.)",
history.len()
);
}
 
loop {
// A line from the keyboard, or Ctrl-C, which at the prompt means /quit.
let line = tokio::select! {
line = keys.recv() => line,
_ = tokio::signal::ctrl_c() => Some(String::from("/quit")),
};
let line = match line {
Some(line) => line,
None => break, // no more input: the keyboard was closed
};
let message = line.trim();
let cmd = match parse(message) {
Ok(cmd) => cmd,
Err(e) => {
println!("bot: {e}");
continue;
}
};
match cmd {
Command::Empty => {} // nothing typed: nothing to answer
Command::Quit => {
println!("bot: bye, {name}!");
break;
}
Command::Help => println!("bot: {HELP}"),
Command::Again => match &last {
Some(text) => {
answer(replier.as_ref(), &[Message::new("you", text)]).await?;
}
None => println!("bot: nothing to repeat yet"),
},
Command::History => {
if history.is_empty() {
println!("bot: nothing said yet");
} else {
for (i, m) in history.iter().enumerate() {
println!("{}. {}", i + 1, m.to_line());
}
}
}
Command::Count => {
let bytes: usize = history.iter().map(|m| m.text.len()).sum();
println!("bot: {} messages, {} bytes of text", history.len(), bytes);
}
Command::Stats => {
// One chain of steps: your messages -> their words -> a count per word.
let mut counts: HashMap<String, u32> = HashMap::new();
for word in history
.iter()
.filter(|m| m.author == "you")
.flat_map(|m| m.text.split_whitespace())
{
*counts.entry(word.to_lowercase()).or_insert(0) += 1;
}
let mut top: Vec<(String, u32)> = counts.into_iter().collect();
// most used first; ties in alphabetical order, so the answer never changes
top.sort_by(|a, b| b.1.cmp(&a.1).then(a.0.cmp(&b.0)));
let words: Vec<String> = top
.iter()
.take(3)
.map(|(word, n)| format!("{word} ({n})"))
.collect();
if words.is_empty() {
println!("bot: nothing said yet");
} else {
println!("bot: your top words: {}", words.join(", "));
}
}
Command::Remember(key, fact) => {
println!("bot: got it: {key} is {fact}");
facts.insert(key, fact);
}
Command::Recall(Some(key)) => match facts.get(&key) {
Some(fact) => println!("bot: {key} is {fact}"),
None => println!("bot: I don't know anything about {key}"),
},
Command::Recall(None) => {
if facts.is_empty() {
println!("bot: I don't remember anything yet. Try /remember");
} else {
// A HashMap has no order, so sort the names before printing them.
let mut names = Vec::new();
for name in facts.keys() {
names.push(name.as_str());
}
names.sort();
println!("bot: I remember: {}", names.join(", "));
}
}
Command::Find(word) => {
// The closure borrows `word` from here: a small function made on the spot.
let hits: Vec<&Message> = history
.iter()
.filter(|m| m.author == "you" && m.text.contains(&word))
.collect();
if hits.is_empty() {
println!("bot: you never said {word}");
} else {
for m in hits {
println!("{}", m.to_line());
}
}
}
Command::Last(n) => {
for m in last_n(&history, n) {
println!("{}", m.to_line());
}
}
Command::Model(choice) => {
let next: Result<Box<dyn Replier>, String> = match choice.as_str() {
"echo" => Ok(Box::new(Echo { max_len: MAX_LEN })),
"pretend" => Ok(Box::new(Pretend)),
"real" => match Model::from_env() {
Ok(model) => Ok(Box::new(model)),
Err(e) => Err(e.to_string()),
},
_ => Err(format!(
"I don't know a model called {choice}. Try echo, pretend or real"
)),
};
match next {
Ok(r) => {
replier = r;
println!("bot: replies now come from {}", replier.name());
}
Err(why) => println!("bot: {why}"),
}
}
Command::Say(text) => println!("bot: {text}"),
Command::Shout(text) => println!("bot: {}", Message::new("you", &text).shout()),
Command::Debug(text) => println!("bot: {:?}", parse(&text)),
Command::Chat(text) => {
messages_seen += 1;
history.push(Message::new("you", &text));
// the history holds its own copy, so `text` can move into `last`
last = Some(text);
if let Some(reply) = answer(replier.as_ref(), &history).await? {
history.push(Message::new("bot", &reply));
}
}
}
}
println!("messages so far: {messages_seen}");
store::save(store::FILE, &history)?;
let s = if history.len() == 1 { "" } else { "s" };
println!("saved {} message{s} to {}", history.len(), store::FILE);
Ok(())
}
 
// Tests: `cargo test` builds and runs them; the program itself never sees them.
#[cfg(test)]
mod tests {
use super::*;
 
#[test]
fn last_n_takes_from_the_end() {
assert_eq!(last_n(&[1, 2, 3, 4], 2), &[3, 4]);
assert_eq!(last_n(&["a", "b"], 5), &["a", "b"]);
assert!(last_n(&[1, 2, 3], 0).is_empty());
}
}
src/command.rs
//! Commands: what a line of input means, and what can be wrong with it.
use std::fmt;
 
/// Everything a line of input can mean. One enum, so a `match` on it must
/// handle every kind of line: the compiler checks that nothing is forgotten.
#[derive(Debug)]
pub enum Command {
Empty,
Help,
Quit,
Again,
History,
Count,
Stats,
Remember(String, String),
Recall(Option<String>),
Find(String),
Last(usize),
Model(String),
Say(String),
Shout(String),
Debug(String),
Chat(String),
}
 
/// Everything that can be wrong with a line of input. parse() returns one of
/// these instead of a Command, and main prints it.
#[derive(Debug)]
pub enum CommandError {
Unknown(String),
Missing(String, String),
Usage(String),
NotACount(String),
}
 
/// How a CommandError reads when printed with {}.
impl fmt::Display for CommandError {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
CommandError::Unknown(word) => {
write!(f, "I don't know the command {word}. Try /help.")
}
CommandError::Missing(verb, example) => {
write!(f, "{verb} what? Try /{verb} {example}")
}
CommandError::Usage(form) => write!(f, "try {form}"),
CommandError::NotACount(text) => write!(f, "'{text}' is not a count. Try /last 3"),
}
}
}
 
/// The text after a command word, or the error that says it is missing.
fn need(line: &str, verb: &str, example: &str) -> Result<String, CommandError> {
match arg(line) {
Some(text) => Ok(String::from(text)),
None => Err(CommandError::Missing(
String::from(verb),
String::from(example),
)),
}
}
 
/// How many messages /last should show. "two" or "-1" is an error.
fn parse_count(text: &str) -> Result<usize, CommandError> {
match text.parse::<usize>() {
Ok(n) => Ok(n),
Err(_) => Err(CommandError::NotACount(String::from(text))),
}
}
 
/// Turn one line of input into a Command, or say what is wrong with it.
/// Each `?` means: if this step failed, return its error right here.
pub fn parse(line: &str) -> Result<Command, CommandError> {
let cmd = match first_word(line) {
"" => Command::Empty,
"/help" | "/h" => Command::Help,
"/quit" | "/q" | "/exit" => Command::Quit,
"/again" => Command::Again,
"/history" => Command::History,
"/count" => Command::Count,
"/stats" => Command::Stats,
"/remember" => {
let usage = CommandError::Usage(String::from("/remember <name> <fact>"));
let rest = match arg(line) {
Some(rest) => rest,
None => return Err(usage),
};
match arg(rest) {
Some(fact) => Command::Remember(String::from(first_word(rest)), String::from(fact)),
None => return Err(usage),
}
}
"/recall" => match arg(line) {
Some(name) => Command::Recall(Some(String::from(name))),
None => Command::Recall(None),
},
"/find" => Command::Find(need(line, "find", "hello")?),
"/last" => Command::Last(parse_count(&need(line, "last", "3")?)?),
"/model" => Command::Model(need(line, "model", "pretend")?),
"/debug" => Command::Debug(need(line, "debug", "/say hi")?),
"/say" => Command::Say(need(line, "say", "hello")?),
"/shout" => Command::Shout(need(line, "shout", "hello")?),
word => {
if is_command(word) {
return Err(CommandError::Unknown(String::from(word)));
}
Command::Chat(String::from(line))
}
};
Ok(cmd)
}
 
/// True when the text is a command: it starts with a slash.
fn is_command(text: &str) -> bool {
text.starts_with('/')
}
 
/// The first word of the text: a slice (&str) pointing into it, not a copy.
fn first_word(text: &str) -> &str {
match text.find(' ') {
Some(i) => &text[..i],
None => text,
}
}
 
/// The text after the command word, if there is any: Some("hello") for
/// "/say hello", None for a bare "/say". No more "" standing in for nothing.
fn arg(line: &str) -> Option<&str> {
match line.find(' ') {
Some(i) => {
let rest = line[i + 1..].trim();
if rest.is_empty() { None } else { Some(rest) }
}
None => None,
}
}
 
#[cfg(test)]
mod tests {
use super::*;
 
#[test]
fn plain_text_is_chat() {
assert!(matches!(parse("hello there"), Ok(Command::Chat(t)) if t == "hello there"));
}
 
#[test]
fn help_has_a_short_form() {
assert!(matches!(parse("/help"), Ok(Command::Help)));
assert!(matches!(parse("/h"), Ok(Command::Help)));
}
 
#[test]
fn say_needs_text() {
assert_eq!(
parse("/say").unwrap_err().to_string(),
"say what? Try /say hello"
);
}
 
#[test]
fn last_needs_a_number() {
assert!(matches!(parse("/last 3"), Ok(Command::Last(3))));
assert_eq!(
parse("/last two").unwrap_err().to_string(),
"'two' is not a count. Try /last 3"
);
}
 
#[test]
fn unknown_commands_are_errors() {
assert_eq!(
parse("/dance").unwrap_err().to_string(),
"I don't know the command /dance. Try /help."
);
}
 
#[test]
fn remember_takes_a_name_and_a_fact() {
assert!(
matches!(parse("/remember pizza extra cheese"), Ok(Command::Remember(k, v)) if k == "pizza" && v == "extra cheese")
);
assert!(parse("/remember pizza").is_err());
}
}
src/message.rs
//! A Message: one line of the conversation.
use serde::{Deserialize, Serialize};
 
/// One line of the conversation: who said it, and what they said.
#[derive(Serialize, Deserialize)]
pub struct Message {
pub author: String,
pub text: String,
}
 
impl Message {
/// Make a Message from two pieces of borrowed text.
pub fn new(author: &str, text: &str) -> Message {
Message {
author: String::from(author),
text: String::from(text),
}
}
 
/// The message as one transcript line: "you: hello".
pub fn to_line(&self) -> String {
format!("{}: {}", self.author, self.text)
}
 
/// The text in capitals, for /shout.
pub fn shout(&self) -> String {
self.text.to_uppercase()
}
}
src/replier.rs
//! Who answers: anything that can reply, behind one trait.
use crate::message::Message;
use serde::{Deserialize, Serialize};
use std::env;
use std::fmt;
use std::time::Duration;
use tokio::sync::mpsc;
 
/// What can go wrong while getting a reply.
#[derive(Debug)]
pub enum ChatError {
/// The request did not get a good answer (no network, bad key, server error).
Http(reqwest::Error),
/// The model answered, but with nothing in it.
Empty,
/// /model real was asked for, but a setting is missing.
NotSet(&'static str),
}
 
impl fmt::Display for ChatError {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
ChatError::Http(e) => write!(f, "the model call failed: {e}"),
ChatError::Empty => write!(
f,
"there was nothing to reply with (an empty conversation or an empty answer)"
),
ChatError::NotSet(name) => write!(f, "set {name} first (see lesson 9.2)"),
}
}
}
 
/// Lets `?` turn a reqwest error into a ChatError on its own.
impl From<reqwest::Error> for ChatError {
fn from(e: reqwest::Error) -> Self {
ChatError::Http(e)
}
}
 
/// The receiving end of an answer: its pieces arrive in order, as they are
/// ready, and the channel closes when the answer is complete.
pub type Pieces = mpsc::Receiver<Result<String, ChatError>>;
 
/// A promise with a name: any type that implements Replier can answer a
/// conversation. `reply` returns at once; the answer arrives on the channel.
pub trait Replier {
/// A short name, for "/model" and the banner.
fn name(&self) -> &str;
/// Start answering the conversation so far (the last message is the newest).
fn reply(&self, history: &[Message]) -> Pieces;
}
 
/// A channel that already holds a whole answer, or an error, and is closed.
fn all_at_once(answer: Result<String, ChatError>) -> Pieces {
let (tx, rx) = mpsc::channel(1);
// A new channel with room for one cannot be full, so this send always works.
tx.try_send(answer).expect("a new channel has room for one");
rx
}
 
/// The echo Chatterbox has had since chapter 3. It answers all at once.
pub struct Echo {
pub max_len: usize,
}
 
impl Replier for Echo {
fn name(&self) -> &str {
"echo"
}
 
fn reply(&self, history: &[Message]) -> Pieces {
let message = match history.last() {
Some(m) => &m.text,
None => return all_at_once(Err(ChatError::Empty)),
};
let text = if message.len() > self.max_len {
format!("that is too long (the limit is {} bytes)", self.max_len)
} else if message.ends_with('?') {
String::from("Good question! I can only echo. Try /model real to ask a real model.")
} else {
format!("you said: {message}")
};
all_at_once(Ok(text))
}
}
 
/// A stand-in for a real model. It types its answer a word at a time, the way
/// a real model does, so you can watch an answer arrive in pieces.
pub struct Pretend;
 
impl Replier for Pretend {
fn name(&self) -> &str {
"pretend"
}
 
fn reply(&self, history: &[Message]) -> Pieces {
let last = match history.last() {
Some(m) => &m.text,
None => return all_at_once(Err(ChatError::Empty)),
};
let text = format!(
"(pretend model) I read all {} message{}. You last said: {last}",
history.len(),
if history.len() == 1 { "" } else { "s" },
);
let (tx, rx) = mpsc::channel(8);
// The task runs on after this function has returned, so it cannot
// borrow anything from here: `move` hands `text` and `tx` over to it.
tokio::spawn(async move {
for (i, word) in text.split(' ').enumerate() {
let piece = if i == 0 {
word.to_string()
} else {
format!(" {word}")
};
if tx.send(Ok(piece)).await.is_err() {
return; // nobody is listening any more
}
tokio::time::sleep(Duration::from_millis(40)).await;
}
});
rx
}
}
 
/// One message as a chat API expects it: a role and some content.
#[derive(Serialize)]
struct ApiMessage {
role: String,
content: String,
}
 
/// The body we send: which model, the whole conversation, and a request to
/// send the answer as it is written instead of all at the end.
#[derive(Serialize)]
struct Request {
model: String,
messages: Vec<ApiMessage>,
stream: bool,
}
 
/// One piece of a streamed answer. serde ignores every other field.
#[derive(Deserialize)]
struct Chunk {
choices: Vec<ChunkChoice>,
}
 
#[derive(Deserialize)]
struct ChunkChoice {
delta: Delta,
}
 
/// The new text in this piece. Some pieces carry none (the first one often
/// only says who is speaking), so it is an Option.
#[derive(Deserialize)]
struct Delta {
content: Option<String>,
}
 
/// What one line of the stream means.
#[derive(Debug, PartialEq)]
enum Event {
/// More of the answer.
Piece(String),
/// The answer is complete.
Done,
/// Anything else: a blank line between events, a comment, a piece with no text.
Other,
}
 
/// Read one line of a server-sent event stream as the chat API sends it:
/// `data: {json}` for each piece, then `data: [DONE]`.
fn parse_event(line: &str) -> Event {
let data = match line.strip_prefix("data:") {
Some(rest) => rest.trim(),
None => return Event::Other,
};
if data == "[DONE]" {
return Event::Done;
}
let chunk: Chunk = match serde_json::from_str(data) {
Ok(chunk) => chunk,
Err(_) => return Event::Other,
};
match chunk.choices.into_iter().next() {
Some(choice) => match choice.delta.content {
Some(text) if !text.is_empty() => Event::Piece(text),
_ => Event::Other,
},
None => Event::Other,
}
}
 
/// A real model behind an OpenAI-style chat endpoint, configured from
/// environment variables so the key never sits in the code.
pub struct Model {
url: String,
key: String,
model: String,
}
 
impl Model {
/// Read the three settings, or say which one is missing.
pub fn from_env() -> Result<Model, ChatError> {
let key =
env::var("CHATTERBOX_API_KEY").map_err(|_| ChatError::NotSet("CHATTERBOX_API_KEY"))?;
let url = env::var("CHATTERBOX_URL").map_err(|_| ChatError::NotSet("CHATTERBOX_URL"))?;
let model =
env::var("CHATTERBOX_MODEL").map_err(|_| ChatError::NotSet("CHATTERBOX_MODEL"))?;
Ok(Model { url, key, model })
}
}
 
/// Send the conversation and pass each piece of the answer to `tx` as it
/// arrives over the network.
async fn stream(
url: &str,
key: &str,
body: &Request,
tx: &mpsc::Sender<Result<String, ChatError>>,
) -> Result<(), ChatError> {
let mut response = reqwest::Client::new()
.post(url)
.bearer_auth(key)
.json(body)
.send()
.await?
.error_for_status()?;
// The network delivers bytes in chunks of any size: a chunk can end in the
// middle of a line, or in the middle of a letter that takes several bytes.
// So keep the bytes until a whole line has arrived, and only then read it.
let mut pending: Vec<u8> = Vec::new();
let mut sent_any = false;
while let Some(bytes) = response.chunk().await? {
pending.extend_from_slice(&bytes);
while let Some(end) = pending.iter().position(|&b| b == b'\n') {
let line: Vec<u8> = pending.drain(..=end).collect();
match parse_event(String::from_utf8_lossy(&line).trim()) {
Event::Piece(text) => {
sent_any = true;
if tx.send(Ok(text)).await.is_err() {
return Ok(()); // nobody is listening any more
}
}
Event::Done => return finished(sent_any),
Event::Other => {}
}
}
}
finished(sent_any)
}
 
/// An answer that ended without a single piece of text is an empty answer.
fn finished(sent_any: bool) -> Result<(), ChatError> {
if sent_any {
Ok(())
} else {
Err(ChatError::Empty)
}
}
 
impl Replier for Model {
fn name(&self) -> &str {
"real"
}
 
fn reply(&self, history: &[Message]) -> Pieces {
// Our names ("you", "bot") become the API's ("user", "assistant").
let mut messages = Vec::new();
for m in history {
let role = if m.author == "you" {
"user"
} else {
"assistant"
};
messages.push(ApiMessage {
role: String::from(role),
content: m.text.clone(),
});
}
let body = Request {
model: self.model.clone(),
messages,
stream: true,
};
// Owned copies: the task may outlive `self`, so it cannot borrow from it.
let url = self.url.clone();
let key = self.key.clone();
let (tx, rx) = mpsc::channel(16);
tokio::spawn(async move {
if let Err(e) = stream(&url, &key, &body, &tx).await {
// If nobody is listening any more, there is no one to tell.
let _ = tx.send(Err(e)).await;
}
});
rx
}
}
 
#[cfg(test)]
mod tests {
use super::*;
 
fn said(text: &str) -> Vec<Message> {
vec![Message::new("you", text)]
}
 
/// Wait for every piece of an answer and join them.
async fn whole(mut pieces: Pieces) -> Result<String, ChatError> {
let mut text = String::new();
while let Some(piece) = pieces.recv().await {
text.push_str(&piece?);
}
Ok(text)
}
 
#[tokio::test]
async fn echo_repeats_you() {
let answer = whole(Echo { max_len: 200 }.reply(&said("hi"))).await;
assert_eq!(answer.unwrap(), "you said: hi");
}
 
#[tokio::test]
async fn echo_refuses_long_messages() {
let long = "x".repeat(10);
let answer = whole(Echo { max_len: 5 }.reply(&said(&long))).await;
assert!(answer.unwrap().starts_with("that is too long"));
}
 
#[tokio::test]
async fn pretend_reads_the_whole_history() {
let history = vec![
Message::new("you", "a"),
Message::new("bot", "b"),
Message::new("you", "c"),
];
assert_eq!(
whole(Pretend.reply(&history)).await.unwrap(),
"(pretend model) I read all 3 messages. You last said: c"
);
}
 
#[tokio::test]
async fn pretend_answers_a_word_at_a_time() {
let mut pieces = Pretend.reply(&said("one two"));
let mut count = 0;
while let Some(piece) = pieces.recv().await {
piece.unwrap();
count += 1;
}
assert!(count > 1, "the answer came in {count} piece(s)");
}
 
#[test]
fn a_data_line_is_a_piece() {
let line = r#"data: {"choices":[{"delta":{"content":"Hel"}}]}"#;
assert_eq!(parse_event(line), Event::Piece(String::from("Hel")));
}
 
#[test]
fn done_ends_the_answer() {
assert_eq!(parse_event("data: [DONE]"), Event::Done);
}
 
#[test]
fn lines_without_text_are_skipped() {
assert_eq!(parse_event(""), Event::Other);
assert_eq!(parse_event(": keep-alive"), Event::Other);
let role_only = r#"data: {"choices":[{"delta":{"role":"assistant"}}]}"#;
assert_eq!(parse_event(role_only), Event::Other);
}
 
#[tokio::test]
async fn nothing_to_reply_to_is_an_error() {
let answer = whole(Echo { max_len: 200 }.reply(&[])).await;
assert!(matches!(answer, Err(ChatError::Empty)));
}
}
src/store.rs
//! Saving the conversation to a file and reading it back, as JSON.
use crate::message::Message;
use std::fs;
use std::io;
 
/// Where the history is kept, next to wherever you run Chatterbox.
pub const FILE: &str = "chatterbox.json";
 
/// The saved history, or an empty one the very first time (no file yet).
pub fn load(path: &str) -> Result<Vec<Message>, io::Error> {
match fs::read_to_string(path) {
// `?` turns a JSON error into an io::Error for us (serde_json says how).
Ok(text) => Ok(serde_json::from_str(&text)?),
Err(e) if e.kind() == io::ErrorKind::NotFound => Ok(Vec::new()),
Err(e) => Err(e),
}
}
 
/// Write the whole history to the file as pretty-printed JSON.
pub fn save(path: &str, history: &[Message]) -> Result<(), io::Error> {
let text = serde_json::to_string_pretty(history)?;
fs::write(path, text)
}
Cargo.toml
[package]
name = "chatterbox"
version = "1.1.0"
edition = "2024"
 
[dependencies]
reqwest = { version = "0.13", features = ["json"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1", features = ["macros", "rt-multi-thread", "signal", "sync", "time"] }

Download this version (zip)Unzip, then cargo run inside the folder.

That is lesson 10.5. Tick it off so the course remembers where you are. That is also the end of chapter 10: a few minutes with the course flashcards now makes it stick, and the chapter’s cheat sheet has every idea on one screen.

Questions people ask

Why does nothing happen when I call an async function in Rust?

Calling an async function only creates a future. The work runs when you await it with .await inside async code, or hand it to a runtime such as tokio. A future that is never awaited is dropped without running.

Can I use blocking_send inside async code?

No. tokio panics if blocking_send is called from async code, because waiting the plain way would hold up every other task on that thread. Use send with .await inside async code, and keep blocking_send for an ordinary thread.

Why did my select! take the stop branch when nobody pressed stop?

If every Sender of the stop channel has been dropped, recv returns None at once, and an arm whose pattern is an underscore accepts that. Keep a Sender alive for as long as the stop button should work.

What should I learn after this Rust course?

Extend Chatterbox first: a clear command, a system message, more tests or a timeout. Then the tokio tutorial goes deeper into tasks, channels and select, and the free Rust Book covers the language from another side.

Learn Rust by building a chat app · Lesson 10.5 · every compiler message above is real output from rustc 1.99.0 (b940084d7 2026-09-28) (edition 2024) · Vibe Engines · 2026