As mentioned in my JuliaCon talk, Tim Wheeler and I have produced a draft of a textbook on optimization. It provides Julia code for the various algorithms. The publisher has agreed to allow us to post the draft behind a password protected website. If you are interested in providing feedback, please email me at mykel@stanford.edu and I’ll send you the login information. If you find errors in the textbook, you will be gratefully acknowledged.
55 Likes
I have a few questions:
- Are the algorithms implemented in pure Julia?
- Will the codes used in the text be made available online and under what license?
- Is there a table of contents available somewhere?
- All algorithms are implemented in pure Julia.
- The preface of the book states: “Permission is granted, free of charge, to use the code snippets associated with this book, subject to the condition that the source of the code is acknowledged.” For testing purposes, the code has been automatically extracted and posted on Tim’s github repo. Note his 99% test coverage on coveralls.
We’ll be working on a better way to make the source more accessible. - Yes. You can download the table of contents here.
11 Likes
Awesome!
This is great news, especially the fact that you provide Julia code. Can you please explain how the textbook is different from standard texts, eg Nocedal-Wright?
1 Like
Great question. There are many excellent texts on optimization, like Nocedal-Wright and Boyd-Vandenberghe. In Algorithms for Optimization, we try to cover a broader array of topics relevant to engineering, such as multiobjective optimization, multidisciplinary optimization, Gaussian process surrogate models, uncertainty propagation, expression optimization, etc. Some of the topics, of course, show up in Nocedal and Wright, such as trust region methods and linear programming. Since we were trying to constrain ourselves to about 500 pages and to make it so that it can be used in a one-semester or quarter course, we had to make some compromises. Instead of going in depth with theorems and proofs, we refer the reader to other resources using the Tufte-style sidenotes. We keep the focus on the algorithms and provide concrete implementations throughout. One fun aspect of the book is that there are zero external figures. We run lualatex and pythontex (which supports Julia), and it generates all the figures illustrating the algorithms in the text directly. 
20 Likes
From the table of contents, it looks like you cover quite an impressive range of topics! It’s very satisfying to see a text this comprehensive use Julia exclusively. I’m looking forward to seeing the final product!
1 Like
I will definitely buy this book
3 Likes
Thanks for your answer. Frequently, a problem I have with implementing algorithms from textbooks/papers is that while the main algorithm is well-specified, in practice its application requires some small heuristic or trick that is omitted from the discussion and can only be found in the code. Having a textbook which enforces a stronger correspondence between the discussion and the code would indeed be great. Looking forward to buying it!
4 Likes
Same here! And with executable code we can perform unit tests to help ensure correctness. The figures in the book are actually produced using the code you see.
2 Likes
It is a pity this book is apparently not going to be published in electronic form. I think it would be really cool to have the book contain links to the actual code. Imagine how much easier experimentation with the algorithms would be if you didn’t have to go searching for the code, but could just click to have it brought up in a Jupyter notebook…
I’ve tried experimenting with this (FEA book), but imagine how much further this could go if one was able and willing to code up for instance embedding the code within the PDF so that no external storage of files was required…
2 Likes
Our publisher will create an e-book version for Kindle and other platforms. We can discuss this with them; they have been quite open minded in the past. We also have the ability to provide ancillary material on the book website, including interactive Jupyter notebooks.
We’re also starting a new book titled Algorithms for Decision Making, which is a reworking of my Decision Making Under Uncertainty: Theory and Application book (MIT Press, 2015), which covers Bayesian networks, reinforcement learning, partially observable Markov decision processes, etc.—but in the style of the Algorithms for Optimization book with full Julia code. We’re aiming for 2020 with that book.
23 Likes
Oh, nice!
I’d think that that would help sell the book!
Given my previous experience with versions of textbooks for the Kindle: the typesetting of the math tends to be atrocious. Have you discussed how much control you will have over the conversion to the Kindle format? (Just curious, really…)
2 Likes
Yep, they would reproduce the book as it shows up in print without trying to reflow.
That’s good. It would be a shame to spoil the beautiful typesetting of the book.
Just wondering how the book is going - any rough expectations of when it might be published?
1 Like
It’s done! And it is so beautiful, primarily because of the amazing work of my coauthor Tim Wheeler in creating the figures (all done in TikZ and PGFPlots.jl). You can pre-order it now on Amazon and other bookstores. Amazon says that it will be released on March 12. Tim wrote a blog post about the experience creating this book, and I’m hoping he will be able to talk about it at the next JuliaCon.
23 Likes
Many congrats on the fantastic book! That’s also a great blog post.
5 Likes