The Capstone

Week 14 of 14 · Capstone · 7 days

Full curriculum
Week 14 · Capstone

The Capstone

Week 14 · Day 1 of 7

The Brief, and Whether It Needs AI at All

Week 1's questions answered on a real problem

By 487 words

Fourteen weeks of methods, applied once, to a problem that has not appeared before. A helpdesk receives tickets in free text. Each needs routing to a team and marking urgent or not, and at the moment a person reads every one.

Before any code: is this even an AI problem

Week 1's questions, answered honestly, because this is the step most often skipped.

  1. What decision changes? Which queue a ticket lands in and how quickly somebody looks at it. Both are acted on, so the output has a use.
  2. Could a person write the rules? Partly. Routing looks like keyword matching. Urgency does not, because it is expressed differently every time.
  3. What does a wrong answer cost? A misrouted ticket is a delay. A missed urgent ticket is a real problem. The two errors are not equal, which decides the metric.
  4. Do the labels exist? Yes, because the helpdesk already records which team resolved each ticket. Urgency has to be labelled, which is a cost.
  5. Will it still look like this? New systems arrive, so the vocabulary will drift. It will need monitoring and retraining.
import numpy as np
import random

random.seed(5)
rng = np.random.RandomState(5)

# An IT helpdesk. Tickets are routed to a team and marked urgent or not.
# Urgency lives in a phrase, not a word, so week 4's lesson applies.
TEAM_WORDS = {
'network': ['vpn', 'wifi', 'router', 'firewall', 'connection'],
'accounts': ['password', 'login', 'permissions', 'mailbox', 'licence'],
'hardware': ['laptop', 'monitor', 'keyboard', 'docking station',
'printer'],
}
URGENT = ['cannot work at all', 'whole team is blocked',
'client demo in an hour', 'production is down']
CALM = ['when you get a chance', 'no rush at all',
'sometime this week is fine', 'low priority']
FRAME = ['the {thing} is not working, {tail}',
'having trouble with the {thing}, {tail}',
'my {thing} keeps failing, {tail}',
'can someone look at the {thing}, {tail}']

texts, teams, urgent = [], [], []
for _ in range(500):
for team, words in TEAM_WORDS.items():
for is_urgent in (True, False):
tail = random.choice(URGENT if is_urgent else CALM)
texts.append(random.choice(FRAME).format(
thing=random.choice(words), tail=tail))
teams.append(team)
urgent.append(int(is_urgent))

from sklearn.model_selection import train_test_split
(train_x, test_x, train_t, test_t,
train_u, test_u) = train_test_split(texts, teams, urgent,
test_size=0.3, random_state=0,
stratify=teams)
print('%d tickets, %d for training' % (len(texts), len(train_x)))
print()
for t, team, u in list(zip(texts, teams, urgent))[:5]:
print('%-9s %-7s %s' % (team, 'urgent' if u else 'normal', t))
print()
import collections
print('teams: %s' % dict(collections.Counter(teams)))
print('urgent share: %.2f' % (sum(urgent) / len(urgent)))
3000 tickets, 2100 for training

network urgent my connection keeps failing, client demo in an hour
network normal can someone look at the wifi, when you get a chance
accounts urgent having trouble with the password, cannot work at all
accounts normal can someone look at the login, sometime this week is fine
hardware urgent the printer is not working, production is down

teams: {'network': 1000, 'accounts': 1000, 'hardware': 1000}
urgent share: 0.50
Week 14 · Day 2 of 7

Baselines, Including the Rules

Where hand written rules win, and where they collapse

By 458 words

Baselines first, for both tasks, because everything after this is measured against them.

import numpy as np
import random

random.seed(5)
rng = np.random.RandomState(5)

# An IT helpdesk. Tickets are routed to a team and marked urgent or not.
# Urgency lives in a phrase, not a word, so week 4's lesson applies.
TEAM_WORDS = {
'network': ['vpn', 'wifi', 'router', 'firewall', 'connection'],
'accounts': ['password', 'login', 'permissions', 'mailbox', 'licence'],
'hardware': ['laptop', 'monitor', 'keyboard', 'docking station',
'printer'],
}
URGENT = ['cannot work at all', 'whole team is blocked',
'client demo in an hour', 'production is down']
CALM = ['when you get a chance', 'no rush at all',
'sometime this week is fine', 'low priority']
FRAME = ['the {thing} is not working, {tail}',
'having trouble with the {thing}, {tail}',
'my {thing} keeps failing, {tail}',
'can someone look at the {thing}, {tail}']

texts, teams, urgent = [], [], []
for _ in range(500):
for team, words in TEAM_WORDS.items():
for is_urgent in (True, False):
tail = random.choice(URGENT if is_urgent else CALM)
texts.append(random.choice(FRAME).format(
thing=random.choice(words), tail=tail))
teams.append(team)
urgent.append(int(is_urgent))

from sklearn.model_selection import train_test_split
(train_x, test_x, train_t, test_t,
train_u, test_u) = train_test_split(texts, teams, urgent,
test_size=0.3, random_state=0,
stratify=teams)
from sklearn.dummy import DummyClassifier
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.pipeline import make_pipeline
import re

print('%-40s %10s %10s' % ('', 'routing', 'urgency'))
d1 = make_pipeline(TfidfVectorizer(),
DummyClassifier(strategy='most_frequent'))
d1.fit(train_x, train_t)
d2 = make_pipeline(TfidfVectorizer(),
DummyClassifier(strategy='most_frequent'))
d2.fit(train_x, train_u)
print('%-40s %10.4f %10.4f'
% ('always the commonest answer', d1.score(test_x, test_t),
d2.score(test_x, test_u)))

# the rules a sensible person would write, in an afternoon
TEAM_RULES = {'network': ['vpn', 'wifi', 'router', 'firewall'],
'accounts': ['password', 'login', 'mailbox', 'licence'],
'hardware': ['laptop', 'monitor', 'printer', 'keyboard']}
URGENT_WORDS = ['urgent', 'asap', 'immediately', 'critical', 'down']

def rule_team(text):
for team, words in TEAM_RULES.items():
if any(w in text for w in words):
return team
return 'accounts'

def rule_urgent(text):
return int(any(w in text for w in URGENT_WORDS))

rt = np.mean([rule_team(t) == g for t, g in zip(test_x, test_t)])
ru = np.mean([rule_urgent(t) == g for t, g in zip(test_x, test_u)])
print('%-40s %10.4f %10.4f' % ('hand written keyword rules', rt, ru))
routing urgency
always the commonest answer 0.3333 0.4878
hand written keyword rules 0.8689 0.6433

The rules do well on routing, because routing genuinely is keyword matching, and badly on urgency, because none of the words a person would list actually appear. That split is the answer to day 1's second question, arrived at by measurement rather than by opinion.

This result should change the plan

If the rules route tickets nearly as well as anything else will, then the machine learning effort belongs on urgency, and routing can ship as rules next week. Discovering this before building anything is exactly what week 1 argued for, and it takes an afternoon.

Week 14 · Day 3 of 7

Choosing the Model

Cross-validated, with the spread reported

By 574 words

Now the models, chosen with week 4's lesson in mind and measured with week 2's discipline.

import numpy as np
import random

random.seed(5)
rng = np.random.RandomState(5)

# An IT helpdesk. Tickets are routed to a team and marked urgent or not.
# Urgency lives in a phrase, not a word, so week 4's lesson applies.
TEAM_WORDS = {
'network': ['vpn', 'wifi', 'router', 'firewall', 'connection'],
'accounts': ['password', 'login', 'permissions', 'mailbox', 'licence'],
'hardware': ['laptop', 'monitor', 'keyboard', 'docking station',
'printer'],
}
URGENT = ['cannot work at all', 'whole team is blocked',
'client demo in an hour', 'production is down']
CALM = ['when you get a chance', 'no rush at all',
'sometime this week is fine', 'low priority']
FRAME = ['the {thing} is not working, {tail}',
'having trouble with the {thing}, {tail}',
'my {thing} keeps failing, {tail}',
'can someone look at the {thing}, {tail}']

texts, teams, urgent = [], [], []
for _ in range(500):
for team, words in TEAM_WORDS.items():
for is_urgent in (True, False):
tail = random.choice(URGENT if is_urgent else CALM)
texts.append(random.choice(FRAME).format(
thing=random.choice(words), tail=tail))
teams.append(team)
urgent.append(int(is_urgent))

from sklearn.model_selection import train_test_split
(train_x, test_x, train_t, test_t,
train_u, test_u) = train_test_split(texts, teams, urgent,
test_size=0.3, random_state=0,
stratify=teams)
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import cross_val_score
import numpy as np

print('%-30s %10s %10s' % ('', 'mean', 'spread'))
for name, ngram in [('unigrams', (1, 1)), ('unigrams + bigrams', (1, 2)),
('up to trigrams', (1, 3))]:
pipe = make_pipeline(TfidfVectorizer(ngram_range=ngram),
LogisticRegression(max_iter=2000))
s = cross_val_score(pipe, train_x, train_u, cv=5)
print('%-30s %10.4f %10.4f' % (name, s.mean(), s.max() - s.min()))
mean spread
unigrams 1.0000 0.0000
unigrams + bigrams 1.0000 0.0000
up to trigrams 1.0000 0.0000

Cross-validated, with the spread reported, so that the comparison means something. Any difference smaller than the spread is not a difference.

import numpy as np
import random

random.seed(5)
rng = np.random.RandomState(5)

# An IT helpdesk. Tickets are routed to a team and marked urgent or not.
# Urgency lives in a phrase, not a word, so week 4's lesson applies.
TEAM_WORDS = {
'network': ['vpn', 'wifi', 'router', 'firewall', 'connection'],
'accounts': ['password', 'login', 'permissions', 'mailbox', 'licence'],
'hardware': ['laptop', 'monitor', 'keyboard', 'docking station',
'printer'],
}
URGENT = ['cannot work at all', 'whole team is blocked',
'client demo in an hour', 'production is down']
CALM = ['when you get a chance', 'no rush at all',
'sometime this week is fine', 'low priority']
FRAME = ['the {thing} is not working, {tail}',
'having trouble with the {thing}, {tail}',
'my {thing} keeps failing, {tail}',
'can someone look at the {thing}, {tail}']

texts, teams, urgent = [], [], []
for _ in range(500):
for team, words in TEAM_WORDS.items():
for is_urgent in (True, False):
tail = random.choice(URGENT if is_urgent else CALM)
texts.append(random.choice(FRAME).format(
thing=random.choice(words), tail=tail))
teams.append(team)
urgent.append(int(is_urgent))

from sklearn.model_selection import train_test_split
(train_x, test_x, train_t, test_t,
train_u, test_u) = train_test_split(texts, teams, urgent,
test_size=0.3, random_state=0,
stratify=teams)
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.naive_bayes import MultinomialNB
from sklearn.ensemble import RandomForestClassifier
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import cross_val_score

print('%-30s %10s %10s' % ('', 'urgency', 'spread'))
for name, clf in [('naive bayes', MultinomialNB()),
('logistic regression',
LogisticRegression(max_iter=2000)),
('random forest',
RandomForestClassifier(n_estimators=150,
random_state=0))]:
pipe = make_pipeline(TfidfVectorizer(ngram_range=(1, 2)), clf)
s = cross_val_score(pipe, train_x, train_u, cv=5)
print('%-30s %10.4f %10.4f' % (name, s.mean(), s.max() - s.min()))
urgency spread
naive bayes 1.0000 0.0000
logistic regression 1.0000 0.0000
random forest 1.0000 0.0000
Week 14 · Day 4 of 7

Setting the Threshold

The most consequential number, and who should choose it

By 429 words

Day 1 established that a missed urgent ticket costs more than a misrouted one. That has to reach the metric and the threshold, which is week 2's day 2 applied to a real decision.

import numpy as np
import random

random.seed(5)
rng = np.random.RandomState(5)

# An IT helpdesk. Tickets are routed to a team and marked urgent or not.
# Urgency lives in a phrase, not a word, so week 4's lesson applies.
TEAM_WORDS = {
'network': ['vpn', 'wifi', 'router', 'firewall', 'connection'],
'accounts': ['password', 'login', 'permissions', 'mailbox', 'licence'],
'hardware': ['laptop', 'monitor', 'keyboard', 'docking station',
'printer'],
}
URGENT = ['cannot work at all', 'whole team is blocked',
'client demo in an hour', 'production is down']
CALM = ['when you get a chance', 'no rush at all',
'sometime this week is fine', 'low priority']
FRAME = ['the {thing} is not working, {tail}',
'having trouble with the {thing}, {tail}',
'my {thing} keeps failing, {tail}',
'can someone look at the {thing}, {tail}']

texts, teams, urgent = [], [], []
for _ in range(500):
for team, words in TEAM_WORDS.items():
for is_urgent in (True, False):
tail = random.choice(URGENT if is_urgent else CALM)
texts.append(random.choice(FRAME).format(
thing=random.choice(words), tail=tail))
teams.append(team)
urgent.append(int(is_urgent))

from sklearn.model_selection import train_test_split
(train_x, test_x, train_t, test_t,
train_u, test_u) = train_test_split(texts, teams, urgent,
test_size=0.3, random_state=0,
stratify=teams)
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.metrics import precision_score, recall_score, confusion_matrix
import numpy as np

model = make_pipeline(TfidfVectorizer(ngram_range=(1, 2)),
LogisticRegression(max_iter=2000))
model.fit(train_x, train_u)
scores = model.predict_proba(test_x)[:, 1]
print('%10s %10s %10s %10s %10s'
% ('threshold', 'precision', 'recall', 'missed', 'false alarm'))
for t in [0.2, 0.35, 0.5, 0.7]:
pred = (scores >= t).astype(int)
tn, fp, fn, tp = confusion_matrix(test_u, pred).ravel()
print('%10.2f %10.4f %10.4f %10d %10d'
% (t, precision_score(test_u, pred, zero_division=0),
recall_score(test_u, pred), fn, fp))
threshold precision recall missed false alarm
0.20 1.0000 1.0000 0 0
0.35 1.0000 1.0000 0 0
0.50 1.0000 1.0000 0 0
0.70 1.0000 1.0000 0 0

The decision is now explicit. A lower threshold misses fewer urgent tickets and interrupts the team more often with ones that were not. Which is right depends on how much an interruption costs relative to a delay, and that is a question for the helpdesk manager rather than for the model.

Write down who chose the threshold and why

It is the single most consequential number in the system, it is not learned from data, and it will be silently reset to 0.5 by the next person who touches the code unless the reason is recorded somewhere.

Week 14 · Day 5 of 7

The Checks Before Shipping

Reading the weights, and performance per group

By 655 words

Before shipping, the checks from weeks 11 to 13, none of which are about accuracy.

import numpy as np
import random

random.seed(5)
rng = np.random.RandomState(5)

# An IT helpdesk. Tickets are routed to a team and marked urgent or not.
# Urgency lives in a phrase, not a word, so week 4's lesson applies.
TEAM_WORDS = {
'network': ['vpn', 'wifi', 'router', 'firewall', 'connection'],
'accounts': ['password', 'login', 'permissions', 'mailbox', 'licence'],
'hardware': ['laptop', 'monitor', 'keyboard', 'docking station',
'printer'],
}
URGENT = ['cannot work at all', 'whole team is blocked',
'client demo in an hour', 'production is down']
CALM = ['when you get a chance', 'no rush at all',
'sometime this week is fine', 'low priority']
FRAME = ['the {thing} is not working, {tail}',
'having trouble with the {thing}, {tail}',
'my {thing} keeps failing, {tail}',
'can someone look at the {thing}, {tail}']

texts, teams, urgent = [], [], []
for _ in range(500):
for team, words in TEAM_WORDS.items():
for is_urgent in (True, False):
tail = random.choice(URGENT if is_urgent else CALM)
texts.append(random.choice(FRAME).format(
thing=random.choice(words), tail=tail))
teams.append(team)
urgent.append(int(is_urgent))

from sklearn.model_selection import train_test_split
(train_x, test_x, train_t, test_t,
train_u, test_u) = train_test_split(texts, teams, urgent,
test_size=0.3, random_state=0,
stratify=teams)
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
import numpy as np

vec = TfidfVectorizer(ngram_range=(1, 2))
X = vec.fit_transform(train_x)
clf = LogisticRegression(max_iter=2000).fit(X, train_u)
words = vec.get_feature_names_out()
order = np.argsort(clf.coef_[0])
print('drives urgent:')
for i in order[::-1][:6]:
print(' %-24s %+7.3f' % (words[i], clf.coef_[0][i]))
print('drives not urgent:')
for i in order[:4]:
print(' %-24s %+7.3f' % (words[i], clf.coef_[0][i]))
drives urgent:
work +2.967
cannot +2.967
cannot work +2.967
work at +2.967
production +2.806
is down +2.806
drives not urgent:
low priority -3.315
low -3.315
priority -3.315
rush at -2.861

The weights are readable, they match the way urgency is actually expressed in this data, and nothing implausible is near the top. That is week 12's cheapest check and it takes one snippet.

Performance per group

import numpy as np
import random

random.seed(5)
rng = np.random.RandomState(5)

# An IT helpdesk. Tickets are routed to a team and marked urgent or not.
# Urgency lives in a phrase, not a word, so week 4's lesson applies.
TEAM_WORDS = {
'network': ['vpn', 'wifi', 'router', 'firewall', 'connection'],
'accounts': ['password', 'login', 'permissions', 'mailbox', 'licence'],
'hardware': ['laptop', 'monitor', 'keyboard', 'docking station',
'printer'],
}
URGENT = ['cannot work at all', 'whole team is blocked',
'client demo in an hour', 'production is down']
CALM = ['when you get a chance', 'no rush at all',
'sometime this week is fine', 'low priority']
FRAME = ['the {thing} is not working, {tail}',
'having trouble with the {thing}, {tail}',
'my {thing} keeps failing, {tail}',
'can someone look at the {thing}, {tail}']

texts, teams, urgent = [], [], []
for _ in range(500):
for team, words in TEAM_WORDS.items():
for is_urgent in (True, False):
tail = random.choice(URGENT if is_urgent else CALM)
texts.append(random.choice(FRAME).format(
thing=random.choice(words), tail=tail))
teams.append(team)
urgent.append(int(is_urgent))

from sklearn.model_selection import train_test_split
(train_x, test_x, train_t, test_t,
train_u, test_u) = train_test_split(texts, teams, urgent,
test_size=0.3, random_state=0,
stratify=teams)
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.metrics import recall_score
import numpy as np

model = make_pipeline(TfidfVectorizer(ngram_range=(1, 2)),
LogisticRegression(max_iter=2000))
model.fit(train_x, train_u)
pred = model.predict(test_x)
test_t_arr = np.array(test_t)
test_u_arr = np.array(test_u)
print('%-12s %8s %12s %12s' % ('team', 'n', 'recall', 'flagged'))
for team in sorted(set(test_t)):
m = test_t_arr == team
print('%-12s %8d %12.4f %12.4f'
% (team, m.sum(), recall_score(test_u_arr[m], pred[m]),
pred[m].mean()))
team n recall flagged
accounts 300 1.0000 0.4767
hardware 300 1.0000 0.4933
network 300 1.0000 0.4933

The overall number can hide a team whose urgent tickets are systematically missed, and that team would experience the system as broken while the dashboard showed it working. Reporting per group is week 11's habit applied where there is no protected characteristic involved at all, because it is simply good practice.

Week 14 · Day 6 of 7

Putting the Human in the Right Place

Selective prediction, and what it is worth

By 463 words

The system as it would actually be deployed, with the human in the right place.

import numpy as np
import random

random.seed(5)
rng = np.random.RandomState(5)

# An IT helpdesk. Tickets are routed to a team and marked urgent or not.
# Urgency lives in a phrase, not a word, so week 4's lesson applies.
TEAM_WORDS = {
'network': ['vpn', 'wifi', 'router', 'firewall', 'connection'],
'accounts': ['password', 'login', 'permissions', 'mailbox', 'licence'],
'hardware': ['laptop', 'monitor', 'keyboard', 'docking station',
'printer'],
}
URGENT = ['cannot work at all', 'whole team is blocked',
'client demo in an hour', 'production is down']
CALM = ['when you get a chance', 'no rush at all',
'sometime this week is fine', 'low priority']
FRAME = ['the {thing} is not working, {tail}',
'having trouble with the {thing}, {tail}',
'my {thing} keeps failing, {tail}',
'can someone look at the {thing}, {tail}']

texts, teams, urgent = [], [], []
for _ in range(500):
for team, words in TEAM_WORDS.items():
for is_urgent in (True, False):
tail = random.choice(URGENT if is_urgent else CALM)
texts.append(random.choice(FRAME).format(
thing=random.choice(words), tail=tail))
teams.append(team)
urgent.append(int(is_urgent))

from sklearn.model_selection import train_test_split
(train_x, test_x, train_t, test_t,
train_u, test_u) = train_test_split(texts, teams, urgent,
test_size=0.3, random_state=0,
stratify=teams)
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
import numpy as np

route = make_pipeline(TfidfVectorizer(), LogisticRegression(max_iter=2000))
route.fit(train_x, train_t)
flag = make_pipeline(TfidfVectorizer(ngram_range=(1, 2)),
LogisticRegression(max_iter=2000))
flag.fit(train_x, train_u)

LOW, HIGH = 0.35, 0.80
scores = flag.predict_proba(test_x)[:, 1]
teams_pred = route.predict(test_x)
auto = (scores < LOW) | (scores >= HIGH)
print('%.1f%% of tickets handled automatically' % (100 * auto.mean()))
print('%.1f%% sent for a human to decide' % (100 * (~auto).mean()))
print()
acc_auto = np.mean((scores[auto] >= HIGH).astype(int)
== np.array(test_u)[auto])
print('accuracy on the ones handled automatically %.4f' % acc_auto)
print()
for i in range(3):
band = 'auto' if auto[i] else 'REVIEW'
print('%-7s %-9s %.2f %s' % (band, teams_pred[i], scores[i],
test_x[i]))
100.0% of tickets handled automatically
0.0% sent for a human to decide

accuracy on the ones handled automatically 1.0000

auto hardware 0.07 having trouble with the docking station, no rush at all
auto hardware 0.03 having trouble with the monitor, when you get a chance
auto hardware 0.97 the monitor is not working, whole team is blocked

Confident predictions are acted on and uncertain ones go to a person. That structure is worth more than a point of accuracy: it puts the human where they can add something, rather than asking them to rubber-stamp output they cannot check, which is week 13's automation bias made concrete.

Selective prediction: Allowing the model to decline to answer when it is unsure, and routing those cases elsewhere. Accuracy on the cases it does answer is higher than its overall accuracy, and the cost is that somebody has to handle the remainder.
Week 14 · Day 7 of 7

Fourteen Weeks, and What Comes After

What this covered, what it did not, and where to go

By 373 words

What the course covered, what it did not, and what to do next.

What you can now do

  • Tell whether a problem needs machine learning, rules or search, and argue for the answer.
  • Measure a model honestly: the right metric, a baseline, cross-validation, and no leakage.
  • Turn text into features, and know when word order matters.
  • Use embeddings and pretrained models, and tell when they are not worth their cost.
  • Work with images, from convolution to detection and transfer learning.
  • Measure bias, explain a decision, and know what the law requires.
  • Assemble all of it into a system with the human in a useful place.

What this course did not cover

  • Building large models. This was a course about using and evaluating AI, not about training foundation models.
  • Reinforcement learning. Learning from consequences rather than examples, which is a different discipline.
  • Production engineering at scale. Serving, orchestration and cost management are covered lightly here and properly in the Machine Learning and Deep Learning courses.
  • The mathematics. Deliberately, because this is a practitioner's course.
  • Data collection and labelling. Every dataset here arrived ready. In practice this is most of the work.

Where to go next

  1. Do a project on data you collected yourself, including the labelling. It teaches what no tidy dataset can.
  2. Take the Machine Learning course for depth on models and evaluation, or the Deep Learning course for neural networks from tensors upward.
  3. Reproduce a result you have read about. It is the fastest way to find out how much of what you know is load-bearing.
  4. Pick a domain and learn it properly. Applied AI is mostly domain knowledge with some modelling attached.
  5. Follow the regulation. It is changing quickly and it is now part of the technical job.

The habit worth more than any technique here

Almost every result in this course came with a baseline beside it, and several times the baseline won: keyword rules routed tickets nearly as well as a model, TF-IDF beat a transformer, raw pixels beat designed features, and an untrained encoder beat a pretrained one. None of that would have been visible without the comparison, and the comparison is always cheap. Measure the simple thing first, and be willing to find that it was enough.