13 Applications for Machine Learning: Synthesis and Concept Generalization
13.1 Overview of Unit
13.1.1 Learning Objectives**
This week we will read a short “book” by Andrew Ng, a very well known computer scientist in the field of AI who also offers one of the best known MOOCs on machine learning at Coursera
This book is very application oriented and practical. The goals for reading it are:
To encourage generalization of your learning - some terminology and perspectives will be different here from the authors you have been reading. The concepts are all the same. We hope that seeing these ideas in a new context will help you recognize and generalize them.
The book is applied. It will help you consider carefully how to make best use of your data for model selection, evaluation, error analysis and other related tasks to figure out what to do when your models don’t perform well.
It is a “course in a book”, such that we hope this will encourage you to see the big picture of how all the pieces from our course fit together.
Finally, it is a quick read so don’t worry that it is a full book. Also, you have no lectures or application assigment this week. It is all about consolidation now!!!! You are in the home stretch.
13.1.2 Readings
Post questions to the readings channel in Slack
13.1.3 Lecture Videos
No lectures this week. Only lab and discussion section.
13.1.4 Application Assignment and Quiz
No assignment this week!
The unit quiz is due by 8 pm on Wednesday, April 24th
13.2 Discussion
13.2.1 Sets and iteration
train/val(dev)/test vs kfold/bootstrap with test vs nested cv
choosing sample sizes for train, val, test
Random vs non-random splits into train/val/test. what needs to be true? what is less important?
overfit val? Use test more than once? Do vs. don’t and risks
Review 12 Takeaways
13.2.2 Error analysis
why
start with baseline/basic system first- why?
eyeball and black box dev sets
size of eyeball sample?
risk of using all dev as eyeball sample
error analysus for regression?
review 19 takeaways
13.2.3 bias/variance
- identify by differences in train/val error
- 1/11
- 15/16
- 15/30
- 1/2
- 1/11
- role of optimal error rate
- avoidable bias and variance using train/val/ and optimal error
- how to fix avoidable bias (25)
- how to fix variance (27)
13.2.4 learning curves
- How and why
- how to calculate with small samples? issues with large samples?
- review figs (from book, saved in figs)
13.2.5 pipeline vs end to end
- when have we seen each?
- costs and benefits vs. possible?