WEBVTT

1
00:00:00.000 --> 00:00:03.560
Every message you send Avriz could
be answered by five different models,

2
00:00:04.180 --> 00:00:06.220
and they differ in
price by ten times.

3
00:00:07.120 --> 00:00:08.980
So, which one should think for you?

4
00:00:09.840 --> 00:00:13.040
First, a tiny judge reads your
request and grades its difficulty.

5
00:00:13.760 --> 00:00:16.760
Trivial goes to the cheapest
rung, hard goes to the frontier.

6
00:00:17.280 --> 00:00:20.180
That's the ladder.
Real turns are messier.

7
00:00:20.640 --> 00:00:24.160
If a task loops too long, it
escalates up a rung mid-turn.

8
00:00:24.680 --> 00:00:28.120
And if a provider runs out of capacity,
the ladder rolls to the next one

9
00:00:28.120 --> 00:00:31.080
automatically. Here's
the interesting part.

10
00:00:32.020 --> 00:00:35.140
A reinforcement learning
bandit watches every turn.

11
00:00:35.680 --> 00:00:39.920
What was picked, what it cost,
whether it succeeded, and learns

12
00:00:39.920 --> 00:00:42.300
its own routing policy
from real outcomes.

13
00:00:43.520 --> 00:00:47.060
Ours discovered that the expensive
models weren't succeeding more often.

14
00:00:47.560 --> 00:00:48.860
The judge was over-routing.

15
00:00:49.800 --> 00:00:54.000
So now the learned policy serves a
small slice of live traffic capped

16
00:00:54.000 --> 00:00:57.960
by a safety ceiling and measured
head-to-head against the judge.

17
00:00:58.740 --> 00:01:00.420
If it wins, it earns more traffic.

18
00:01:01.080 --> 00:01:04.660
If a model behind a rung changes,
that arm resets and relearns.

19
00:01:05.640 --> 00:01:09.000
Routing that learns. Avriz,
your commute is build time.