Course Syllabus

Ma 140a: Probability 

Course Syllabus – Winter, 2021/22
Department of Mathematics, California Institute of Technology

MWF 14:00 - 14:55,  Aud CHL,  Hameetman

 

This syllabus is provided as a reference for prospective students. It's still subject to some modifications before the start of the semester. The definitive version of it will be discussed at the beginning of the course. 

 

Course Instructor

Juan Pablo Vigneaux (he/him)
Office hours: Wednesday 3:00PM-4:00PM
Office: 374 Linde Hall of Mathematics and Physics
Phone: x-6805
E-mail: vigneaux@caltech.edu

 

Teaching Assistant

Ismail Abouamal
Office Hours: Wednsdays 5:00PM-6:00PM.
E-mail: abouamal@caltech.edu.
Office: 259 Linde Hall of Mathematics and Physics

 

Course Welcome

Dear prospective student, welcome to Ma 140a! This course will introduce you to the basic ideas of modern probability theory from a mathematical viewpoint. It should give you a solid foundation to go into more advanced topics in probability theory (such as stochastic processes, stochastic calculus, etc., which are partly the subject of Ma 140b), or to explore the very rich applications of probabilistic ideas (such as queuing theory, statistical learning theory, random graph theory, statistical mechanics, mathematical statistics, information theory...).

The course will be reasonably self-contained but requires previous exposure to rigorous mathematical proofs and familiarity with modern analysis (at least with the so-called "naïve" set theory and with the formal definitions of limits of sequences and functions). Previous knowledge of Lebesgue integration, as taught in Ma 108b, is desirable but not necessary. 

The course will mix expository lessons with some active learning techniques, such as minute papers, group discussions, and collaborative problem solving. Since our time to work on problem sets during class is very limited, homework assignments will play a key role in mastering the contents. 

As an instructor, I am committed to creating a respectful, inclusive, and equitable learning environment. If you identify a problem in this respect that is affecting you or some of your colleagues, do not hesitate to contact me.

 

Course Description

Catalogue's description: Overview of measure theory. Random walks and the Strong law of large numbers via the theory of martingales and Markov chains. Characteristic functions and the central limit theorem. Poisson process and Brownian motion. Topics in statistics.

Additional information: This course covers the measure theoretic foundations of modern probability theory and some of its most fundamental results. We'll study some asymptotic "laws" that govern randomness, namely: the various laws of large numbers (LLNs), the classical central limit theorem (CLT), and basic large deviations theory. A discussion of different modes of convergence in probability theory is unavoidable. The main technical tools introduced in this course are conditional expectations, generating functions, and martingales. We'll take a "concrete" and computational approach to conditional expectations. As a refinement of the limiting laws, we’ll study concentration inequalities and their relationship with information measures. The course will finish with a brief exploration of stochastic processes (a vast topic) and some (very few!) research areas in probability theory.

 

Learning Outcomes

Upon successful completion of this course, you will be able to:

    • Use basic probabilistic concepts (such as event, random variable, probability, expectation, variance, independence, etc.) precisely and effectively both in mathematical and real-world applications. 
    • Recognize the main asymptotic results in probability theory (LLN, CLT), as well as some of their refinements, and be able to apply them in mathematical proofs and problem-solving. 
    • Manipulate conditional expectations and martingales, the defining concepts of modern probability theory, with ease.
    • Recognize and manipulate some basic stochastic models, such as random walks, branching processes, Markov chains, and Poisson processes. 
    • Have a synoptic view of certain subdomains and applications of probability theory, which might serve as a good foundation for further exploration. 

Moreover, the course should improve your skills at

    • Working in groups to solve problems effectively or refine previous solutions.  
    • Writing mathematical arguments clearly; in particular, assimilate the conventions of the probabilistic literature.  

 

Required Text 

As a general reference for the course, we’ll mainly use: 

David Williams, Probability with Martingales, Cambridge University Press, 1991. 

Williams is extremely good at making technical arguments accessible (it’s no coincidence that the book has been reprinted 25 times!). It reviews the parts of measure theory that are most useful for the course and develops in detail the theory of martingales (we won’t cover it in such detail, but the interested students are encouraged to read more about it).  

I’m complementing Williams’ book with 

Geoffrey Grimmett and David Stirzaker, Probability and Random Processes, Oxford University Press, 2001 (third edition). 

This book avoids most measure-theoretic technicalities, but in return it is extremely rich in examples and applications. It’ll be our reference for generating functions, Markov chains, and random processes.  

Most of the measure theory needed in this course is summarized in Williams’ book; for more details and explanations one must consult a textbook on the subject. My personal favorite is  

Donald L. Cohn, Measure Theory, Birkhäuser, 2013 (second edition), 

but for the most general and lean statements I have used 

E. Hewitt, and K. Stromberg,  Real and Abstract Analysis: A modern treatment of the theory of functions of a real variable. Springer Berlin Heidelberg, 1965. 

All these books can be found in the library and have been reserved for this course. Since there are not so many copies available, buying Williams' and/or Grimmett and Stirzaker's books might be a good idea, but this is not mandatory.

 

Coursework and grading

The course's going to have weekly graded homework assignments. They'll help you review and consolidate the concepts introduced each week.

The worst grade obtained in the assignments (except the last one) will be dropped. 

The last homework assignment will be longer and its grade cannot be dropped. 

Attendance and participation might be used by the instructor and TA to improve the final grade. 

Deadlines 

The following rules are in place from Wed, Feb. 2: 

  1. The first late submission (counting from that week) will be fine, provided it happens in the first 72 hours after the deadline. Otherwise, late submission penalties will be applied. 
  1. For a second late submission, the following penalties will be applied: We discount k points of the final grade if the submission took place between (k-1)*24 and k*24 hours after the official deadline. 
  1. For a third late submission, we’ll discount 2*k points of the final grade if the submission took place between (k-1)*24 and k*24 hours after the official deadline. 
  1. Further late submissions won’t be accepted. 

If you’re confronting a serious and recurrent problem (for instance, insomnia, depression, anxiety disorders, some other health condition, etc.), you’re encouraged to approach the Dean for Undergraduate Students, who may authorize further accommodations (see links below). The Dean’s office may also redirect you to other resources to help you with the specific difficulty that you’re facing, which is also very important.    

Mathematical writing

Homework solutions should be reasonably self-contained and clear (they should be clear enough to one of your classmates); formulas and computations must be accompanied by explanations in ordinary formal English. Typical rules of writing style also apply to mathematical texts! I encourage students to reflect about effective mathematical writing. The most celebrated book on this topic might be 

Norman Earl Steenrod, Paul R. Halmos, Menahem M. Schiffer and Jean R. Dieudonné, How to write mathematics, American Mathematical Society, 1973.  

which is a collection of essays authored by some of the finest mathematical writers of the XXth century (particularly Halmos and Dieudonné, which are archetypical representatives of the American and French schools, respectively). The book 

Donald E. Knuth, Tracy Larrabee, and Paul M. Roberts, Mathematical Writing, The Mathematical Association of America, 1996. 

is also a classic. You may already know that Knuth created TeX, so you can guess that he also took writing quite seriously.  

 

Attendance and Participation

I expect students to attend all lectures, or as many as possible, and to actively participate in class.  Please feel free to ask all sorts of questions; there are no "stupid questions".  

Lectures will be recorded; in case you miss one, please watch the recording as soon as possible. Each lecture will rely heavily on previous ones. 

Given that some people have time conflicts, attendance won't count towards the final grade. 

 

Wellness Policy

    • I want to clearly state that taking care of your health and well-being should be your number one priority. You cannot learn if you are unwell or under extreme duress. 
    • The course work should feel challenging in a positive way, but I do not want you to be overwhelmed by your work for this course. If you find yourself overwhelmed or encountering other personal challenges during the term, please reach out to me so we can develop a plan for you to pursue success in this course in a healthy way. 
    • I am available to chat, and you can always attend office hours for a non-academic conversation if necessary.
    • You can also visit the counseling center or talk to a Dean if you find you need help beyond the course staff. In addition, I encourage you to utilize the resources listed at the end of this syllabus.
    • Diversity, inclusion, and belonging are all core values of this course. All participants in this course must be treated with respect by others in accordance with the honor code. If you feel unwelcome or unsafe in any way, no matter how minor, I encourage you to talk to me or one of the Deans.

If you would like to ask about flexibility with coursework for a temporary or minor wellness issue, please contact the instructor directly. The Deans’ Office, Student Wellness Services (SWS) and Caltech Accessibility Services for Students (CASS) are available to help you with illness and health conditions that may impact your coursework:

    • Student Wellness Services will assess and treat illnesses and medical conditions, and communicate (with student’s permission) with the Deans’ Office if CASS, part of SWS, can recommend and provide for accommodations needed due to temporary or long-term disabilities. Policies about academic extensions for medical reasons can be found here.
    • The Deans’ Office may recommend academic exceptions in cases of significant family or personal emergencies, or moderate to severe illness or medical conditions that make it difficult to keep up with coursework. Please reach out to a Dean as soon as possible if you experience these conditions.

 

Covid

While COVID-19 remains a concern, all members of the Caltech community, including students and others, are required to promptly report to the Institute if they have become ill with COVID-like symptoms or have been exposed to someone who has tested positive for COVID-19. Furthermore, any individual, regardless of vaccination status, who is ill or has been exposed to COVID-19 should stay home or return home if they have already reported on-site (including not attending class or other meetings in person), and report their status through the Caltech COVID-19 Reporting Application. Individuals who have reported their status through the COVID-19 Reporting Application will receive personal follow up and guidance from Student Wellness Services on next steps. For additional information on the Institute’s COVID-19 preventative health measures and requirements, visit the Caltech Together website.

 

Students with Documented Disabilities

Students who may need an academic accommodation based on the impact of a disability must initiate the request with Caltech Accessibility Services for Students (CASS).  Professional staff will evaluate the request with required documentation, recommend reasonable accommodations, and prepare an Accommodation Letter for faculty dated in the current quarter in which the request is being made. Students should contact CASS as soon as possible, since timely notice is needed to coordinate accommodations. For more information: http://cass.caltech.edu/, cass@caltech.edu.

 

Academic Integrity

Caltech’s Honor Code: “No member of the Caltech community shall take unfair advantage of any other member of the Caltech community.”

Understanding and Avoiding Plagiarism: Plagiarism is the appropriation of another person's ideas, processes, results, or words without giving appropriate credit, and it violates the honor code in a fundamental way. You can find more information at: http://writing.caltech.edu/resources/plagiarism. 

All instances of plagiarism or other academic misconduct will be referred to the Board of Control for undergraduates. For graduate students, contact the Graduate Office.

 

Collaboration Policy

Full discussion of the HW problems and their solutions is allowed. This includes talking about the concepts relevant to the problem, as well as the details of the solution. However, each student should write down their own version of the solution, and explicitly cite their collaborators.   

Looking up the solutions in books, articles, or other written sources is not allowed. You may stumble upon a solution while reading a textbook to review the concepts; in this case, you must cite that book in your HW and anyway try to solve the problem by yourself once you have finished reading the book.  

As a guideline for the collaboration policy, you should be able to reproduce any solution you hand in without help from anyone else.

 

My Status as a “Responsible Employee”

As a faculty member, I am required to notify the Institute’s Equity and Title IX Office when I become aware of discrimination, sexual harassment, or sex- or gender-based misconduct involving our community members. If one of my students shares such an experience with me, I can help connect them to support resources but will not be able to keep that information confidential as part of fulfilling my responsibility to make sure my students are offered the opportunity to access information and support by the Institute. For more information, you can email equity@caltech.edu, go to equity.caltech.edu, or review the Institute’s Sex- and Gender-Based Misconduct Policy.

If you have experienced such prohibited conduct and would like confidential support, please feel to contact Student Wellness Services [626-395-8331; https://wellness.caltech.edu/counseling]; Taso Dimitriadis, Center for Inclusion and Diversity [626-395-8108; taso@caltech.edu]; or Teresa Mejia, Campus Sexual Violence Advocate [626-395-4770; teresam@caltech.edu].

 

Course Schedule

[W] and [G&S] refer to the main textbooks introduced above in the "Required text" section.

Each homework assignment (except the first) will be given on Friday of week n and will be due on Friday evening of week n+1, covering the material lectured on Friday of week n as well as Monday and Wednesday of week n+1. 

Week

Date

Lecture Topic

Associated Readings

Homework Due

1

Jan 3-7

Introduction. Origins of the concept of probability; “naive” approach. Kolmogorov's axiomatization. Basics of measure theory. Construction of measures. 

[W], Chapter 1.

Jan 7 (assignment zero, not graded)

2

Jan 10-14

Events. Borel-Cantelli lemmas. Measurable functions/random variables. Distributions. Independence.

[W], Chapters 2, 3 and 4.

Jan 14

3

Jan 19-21

Independence (continued). Komogorov's 0-1 law. Integration. 

[W], Chapters 4 and 5.

Jan 21

4

Jan 24-28

L^p spaces. Expectation, variance, and inequalities. Deviation estimates and a strong law of large numbers.

[W], Chapters 6 and 7

Jan 28

5

Jan 31 - Feb 4

Product spaces. Conditional expectations. Bayesianism.

[W], Chapters 8 and 9

Jan 5

6

Feb 7-11

Generating functions, the Central Limit Theorem, and Large deviations

[S&W], Chapter 5

Feb 11

7

Feb 14-18

Martingales. Stopping times. Martingale convergence theorems. Kolmogorov's strong law of large numbers. Applications of martingales.

[W], Chapters 10-12 (some sections)

Feb 18

8

Feb 23-25

Entropy and divergence. The asymptotic equipartition property. Concentration inequalities via the entropy method. Statistical learning theory. 

TBA

Feb 25

9

Feb 28 - Mar 4

Markov chains. Ergodic theorem. General Poisson process.

TBA

Feb 4

10

Mar 7 - Mar 9

Review. Some current applications and perspectives. 

TBA

TBA

 

Academic Resources for Students

 

Additional Resources for Students