SOLVED.
The course lecture doesn’t tell you, and the troubleshooting page doesn’t tell you, but the README file for the plotlyjs() does tell you:
You need to run jupyter labextension install jupyterlab-plotly from the command line.
After doing that (and restarting jupyter lab, and re-running the code), everything works. 