$$ %---- MACROS FOR SETS ----% \newcommand{\znz}[1]{\mathbb{Z} / #1 \mathbb{Z}} \newcommand{\twoheadrightarrowtail}{\mapsto\mathrel{\mspace{-15mu}}\rightarrow} % popular set names \newcommand{\N}{\mathbb{N}} \newcommand{\Z}{\mathbb{Z}} \newcommand{\Q}{\mathbb{Q}} \newcommand{\R}{\mathbb{R}} \newcommand{\C}{\mathbb{C}} \newcommand{\I}{\mathbb{I}} % popular vector space notation \newcommand{\V}{\mathbb{V}} \newcommand{\W}{\mathbb{W}} \newcommand{\B}{\mathbb{B}} \newcommand{\D}{\mathbb{D}} %---- MACROS FOR FUNCTIONS ----% % linear algebra \newcommand{\T}{\mathrm{T}} \renewcommand{\ker}{\mathrm{ker}} \newcommand{\range}{\mathrm{range}} \renewcommand{\span}{\mathrm{span}} \newcommand{\rref}{\mathrm{rref}} \renewcommand{\dim}{\mathrm{dim}} \newcommand{\col}{\mathrm{col}} \newcommand{\nullspace}{\mathrm{null}} \newcommand{\row}{\mathrm{row}} \newcommand{\rank}{\mathrm{rank}} \newcommand{\nullity}{\mathrm{nullity}} \renewcommand{\det}{\mathrm{det}} \newcommand{\proj}{\mathrm{proj}} \renewcommand{\H}{\mathrm{H}} \newcommand{\trace}{\mathrm{trace}} \newcommand{\diag}{\mathrm{diag}} \newcommand{\card}{\mathrm{card}} \newcommand\norm[1]{\left\lVert#1\right\rVert} % differential equations \newcommand{\laplace}[1]{\mathcal{L}\{#1\}} \newcommand{\F}{\mathrm{F}} % misc \newcommand{\sign}{\mathrm{sign}} \newcommand{\softmax}{\mathrm{softmax}} \renewcommand{\th}{\mathrm{th}} \newcommand{\adj}{\mathrm{adj}} \newcommand{\hyp}{\mathrm{hyp}} \renewcommand{\max}{\mathrm{max}} \renewcommand{\min}{\mathrm{min}} \newcommand{\where}{\mathrm{\ where\ }} \newcommand{\abs}[1]{\vert #1 \vert} \newcommand{\bigabs}[1]{\big\vert #1 \big\vert} \newcommand{\biggerabs}[1]{\Bigg\vert #1 \Bigg\vert} \newcommand{\equivalent}{\equiv} \newcommand{\cross}{\times} % statistics \newcommand{\cov}{\mathrm{cov}} \newcommand{\var}{\mathrm{var}} \newcommand{\bias}{\mathrm{bias}} \newcommand{\E}{\mathrm{E}} \newcommand{\prob}{\mathrm{prob}} \newcommand{\unif}{\mathrm{unif}} \newcommand{\invNorm}{\mathrm{invNorm}} \newcommand{\invT}{\mathrm{invT}} \newcommand{\P}{\text{P}} \newcommand{\pmf}{\text{pmf}} \newcommand{\pdf}{\text{pdf}} % real analysis \renewcommand{\sup}{\mathrm{sup}} \renewcommand{\inf}{\mathrm{inf}} %---- MACROS FOR ALIASES AND REFORMATTING ----% % logic \newcommand{\forevery}{\ \forall\ } \newcommand{\OR}{\lor} \newcommand{\AND}{\land} \newcommand{\then}{\implies} % set theory \newcommand{\impropersubset}{\subseteq} \newcommand{\notimpropersubset}{\nsubseteq} \newcommand{\propersubset}{\subset} \newcommand{\notpropersubset}{\not\subset} \newcommand{\union}{\cup} \newcommand{\Union}[2]{\bigcup\limits_{#1}^{#2}} \newcommand{\intersect}{\cap} \newcommand{\Intersect}[2]{\bigcap\limits_{#1}^{#2}} \newcommand{\intersection}[2]{\bigcap\limits_{#1}^{#2}} \newcommand{\Intersection}[2]{\bigcap\limits_{#1}^{#2}} \newcommand{\closure}{\overline} \newcommand{\compose}{\circ} % linear algebra \newcommand{\subspace}{\le} \newcommand{\angles}[1]{\langle #1 \rangle} \newcommand{\identity}{\mathbb{1}} \newcommand{\orthogonal}{\perp} \renewcommand{\parallel}[1]{#1^{||}} % calculus \newcommand{\integral}[2]{\int\limits_{#1}^{#2}} \newcommand{\limit}[1]{\lim\limits_{#1}} \newcommand{\approaches}{\rightarrow} \renewcommand{\to}{\rightarrow} \newcommand{\convergesto}{\rightarrow} % algebra \newcommand{\summation}[2]{\sum\nolimits_{#1}^{#2}} \newcommand{\product}[2]{\prod\limits_{#1}^{#2}} \newcommand{\by}{\times} \newcommand{\integral}[2]{\int_{#1}^{#2}} \newcommand{\ln}{\text{ln}} % exists commands \newcommand{\notexist}{\nexists\ } \newcommand{\existsatleastone}{\exists\ } \newcommand{\existsonlyone}{\exists!} \newcommand{\existsunique}{\exists!} \let\oldexists\exists \renewcommand{\exists}{\ \oldexists\ } % statistics \newcommand{\distributed}{\sim} \newcommand{\onetoonecorresp}{\sim} \newcommand{\independent}{\perp\!\!\!\perp} \newcommand{\conditionedon}{\ |\ } \newcommand{\given}{\ |\ } \newcommand{\notg}{\ngtr} \newcommand{\yhat}{\hat{y}} \newcommand{\betahat}{\hat{\beta}} \newcommand{\sigmahat}{\hat{\sigma}} \newcommand{\muhat}{\hat{\mu}} \newcommand{\transmatrix}{\mathrm{P}} \renewcommand{\choose}{\binom} % misc \newcommand{\infinity}{\infty} \renewcommand{\bold}{\textbf} \newcommand{\italics}{\textit} \newcommand{\step}{\text{step}} $$

Remote Computing with Jupyter Notebooks

Featured image

Awhile ago, I had AWS set up to provide me a unique URL that I could navigate to and use Jupyter Notebooks. I admired the convenience and the ability to just start a computation and close my laptop knowing full well my computations continued working away.

However, using an AWS P2 instance can get very costly depending on your usage, which for me would be around $600 per month. So, I figured I could just build a computer with that kind of money which could serve as a deep learning rig along with the occasional video gaming :).

This post describes the configuration setup once you have already built your computer and installed a flavor of Linux like Ubuntu. Turns out that the following configuration was way easier for me to setup than AWS, and with the help of aliases, remote computing with Jupyter Notebooks is easier than ever. Let’s get started!

Installation

So first you need to install the following on your Ubuntu server:

If you ever need to see the status of openssh-server or restart it, type the below into your terminal:

systemctl status ssh
sudo service ssh restart

Connecting locally to your server

To make sure most things are set up correctly, we first need to verify that you can connect to your server on your local network.

Ok! So on your server, open your sshd_config file located at /etc/ssh/sshd_config. To make changes to it, you’ll need sudo privileges. Once the file is open, you’ll need to specify what port you’ll want to use when connecting in. Whatever you choose, I highly advise not using the default port 22. Let’s say you decide to use port 22222 instead. There’s an entry in your sshd_config file called Port and you should edit it as such:

Port 22222

Under AllowUsers, put the username you use when logging into your server.

AllowUsers your_username

Next, set PasswordAuthentication to yes.

Lastly, to make sure Ubuntu won’t block incoming web traffic on port 8888, we need to adjust its iptables:

sudo iptables -I INPUT -p tcp -s 0.0.0.0/0 --dport 8888 -j ACCEPT
sudo iptables -I INPUT -p tcp -s 0.0.0.0/0 --dport 443 -j ACCEPT
sudo netfilter-persistent save

Now, take note of your server’s IP address. This can be found by typing ifconfig into your terminal and looking for something like: inet 192.168.1.112. Once you’ve identified your server’s IP address on your local network, it’s time to pick up your laptop and try to log into it:

ssh [email protected] -p 22222

If you get a terminal prompt, you’re in!

Connecting remotely to your server

Now the whole point of setting up remote computing is so that you can leave your house and remote into your server while on someone else’s network. To do this only requires a few changes (which require you to still be on your network):

Now try to remote into your server using the WAN address you found!

ssh [email protected]_wan_ip -p 22222

If you see a prompt, well done! One last thing is to set PasswordAuthentication to no in your sshd_config file since now you’re logging in with a ssh key; that way no one can try brute-forcing your password.

You can now access your server from outside your network, go grab yourself that Starbucks coffee :).

Starting a remote Jupyter Notebook

Now that all the hard work is done, you can easily use a remote Jupyter Notebook with the following steps:

  1. ssh into your server: ssh [email protected]_wan_ip -p 22222
  2. Start a new tmux session that you can easily detch from later: tmux new -s session-name
  3. Start jupyter-notebook without a browser: jupyter-notebook --no-browser --port=8889
  4. Now in a new terminal on your laptop, forward your server’s port traffic to your laptop’s local port: ssh -N -L localhost:8888:localhost:8889 [email protected]_wan_ip -p 22222
  5. In your web browser, navigate to localhost:8888/tree and you should see your Jupyter Notebooks!

Now you can easily use Jupyter Notebooks on your laptop, but instead using your server’s beefy resources to do the computing.

One last thing, after having figured out the steps above, I thought I’d make the process more simple by using aliases and functions. The below are the relevant lines I added to my laptop’s .bashrc file:

server-connect() {
  ssh [email protected]$1 -p 22222
}

jn-connect() {
  ssh -N -L localhost:8888:localhost:8889 [email protected]$1 -p 22222
}

And the lines I added to my server’s .bashrc file:

alias jn-remote="jupyter-notebook --ip='*' --no-browser --port=8889"

Now the 5 steps above become:

  1. server-connect server_wan_ip
  2. tmux new -s session-name
  3. jn-remote
  4. open new terminal and type jn-connect server_wan_ip
  5. navigate to localhost:8888/tree

(Photo by Lukas)

comments powered by Disqus