#+TITLE: How I Keep Track of My Servers
#+DATE: 2020-11-25T20:50:42-05:00
#+DRAFT: true
#+DESCRIPTION: The process I use to maintain my server list and ssh configuration
#+TAGS[]: emacs org-mode ssh
#+KEYWORDS[]: emacs org-mode ssh
#+SLUG:
#+SUMMARY:

#+ATTR_HTML: :title Emacs in the server room
#+ATTR_HTML: :alt Emacs in the server room
[[file:ox-ssh.png]]

I manage a lot of servers. Some are serving static content like this
blog, with others running services like [[https://nextcloud.com/][Nextcloud]], [[https://wiki.znc.in/ZNC][ZNC]], [[https://shadowsocks.org/en/index.html][Shadowsocks]],
or [[https://www.mumble.info/][Mumble]]. I also have one or two game servers to play with my family
and friends. These are spread across two providers for cost and
geographic reasons.

In addition, I have several machines running in my house, one running
[[https://www.freenas.org/][FreeNAS]] with some jails, another running [[https://www.proxmox.com/en/][Proxmox]] with several VMs and
containers. I also have a couple smaller single board PCs like
Raspberry Pis scattered around.

Needless to say, I've got a lot to keep track of. I tried using a
couple methods of keeping inventory of what I had running where, the
user names, IP addresses, and links, but found that none suited my
needs particularly well. I also had to worry about making sure my
=~/.ssh/config= file was always up to date with VMs and containers I
create.

I'd already been playing with the idea of using an [[https://orgmode.org/][org mode]] file to
keep track of servers with VMs and containers, as it seemed like that
would fit well with the hierarchical structure of org files.

What I came up was a system where each server location/provider gets a
heading, with the machines in that location as headings under it. If
the machine runs VMs or containers, I just put those as headings under
the host machine.

#+BEGIN_SRC org
,* Scaleway
,** example.com
,** example.org

,* Vultr
,** lambda.cx

,* Home
,** proxmox
,*** pi-hole
,*** openbsd-1
,** freenas
,*** web-jail
#+END_SRC

Each machine gets a bullet point list of what's running on it. For
services with web interfaces, I add a link with the name of the
service to the list, so I can click it to open it in my browser. I
also write details about services underneath their bullet points if I
there's more I want to remember.

#+BEGIN_SRC org
,* Vultr
,** example.com
   - Minecraft
     Save directory: =/home/minecraft/survival=
     Port 4587
   - [[https://example.com][nginx]]
   - [[https://example.com:8080/][znc]]
,** example.org
  - Shadowsocks
  - Mumble
  - [[https://example.net][nginx]]
#+END_SRC

From there I added either an =IP= or =Hostname= properties to each
heading, along with other information about the system like =OS=,
=SSH_USER=, etc. This allows me to use org-mode's [[https://orgmode.org/manual/Sparse-Trees.html][sparse trees]] to
search for, say, all VMs running OpenBSD. Using org mode also allows
me to manage servers like anything else in an org mode document;
adding [[https://orgmode.org/manual/Tags.html][tags]], [[https://orgmode.org/manual/TODO-Items.html#TODO-Items][TODO]] entries, [[https://orgmode.org/manual/Working-with-Source-Code.html#Working-with-Source-Code][code blocks]], [[https://orgmode.org/manual/Hyperlinks.html#Hyperlinks][hyperlinks]], [[https://orgmode.org/manual/Tables.html#Tables][tables]],
[[https://orgmode.org/manual/Attachments.html#Attachments][attachments]], putting details in [[https://orgmode.org/manual/Drawers.html#Drawers][drawers]], etc.

#+begin_src org
,* home
,** proxmox
   :PROPERTIES:
   :IP:       192.168.0.6
   :SSH_USER: dante
   :OS:       Proxmox-VE
   :END:
   - [[https://192.168.0.6:8006][WebUI]]

,*** VMs
,**** openindiana
     :PROPERTIES:
     :OS:       OpenIndiana
     :IP:       192.168.0.11
     :SSH_USER: dante
     :END:

,**** pfsense
     :PROPERTIES:
     :IP:       192.168.0.12
     :OS:       FreeBSD
     :SSH_USER: admin
     :END:
     - [[https://192.168.0.12][WebUI]]
     - DHCP
     - DNS
     - OSPF

,*** Containers
,**** pihole
     :PROPERTIES:
     :IP:       192.168.0.21
     :OS:       Debian
     :SSH_USER: pi
     :END:
     - [[http://192.168.0.23/admin/][WebUI]]

,**** ubuntu
     :PROPERTIES:
     :IP:       192.168.0.22
     :OS:       Ubuntu
     :SSH_USER: dante
     :END:
     - Prometheus
     - Shadowsocks
#+end_src

#+ATTR_HTML: :title The code above rendered in Emacs
#+ATTR_HTML: :alt The code above rendered in Emacs
#+caption: The code above in Emacs with all headings unfolded
[[file:example.png]]

Finally to keep my SSH config up to date, I wrote [[https://github.com/dantecatalfamo/ox-ssh][ox-ssh]]. A backend
for the org mode [[https://orgmode.org/manual/Exporting.html][export engine]] that lets me export my buffer as an SSH
configuration file. It takes the properties from the all headings with
either an =IP= or =HOSTNAME= property and turns them into entries in a
configuration file. It [[https://github.com/dantecatalfamo/ox-ssh#usage][supports]] every client configuration option
OpenSSH has, so I can maintain my entire SSH client list from within
my org mode file.

#+ATTR_HTML: :title An example ox-ssh export
#+ATTR_HTML: :alt An example ox-ssh export
#+caption: Property drawers are folded by default in org mode, but expanded here for context.
[[file:ox-ssh-example.jpg]]

For completeness, I also added a variable that lets me set a header to
the configuration when exporting. This lets me add options which apply
to all hosts, like keyring support for MacOS.

With this new setup, I just have a single version controlled
=Servers.org= file which I keep all the relevant information in.
Whenever I change any details related to a server, I simply press
@@html: <kbd><kbd>Ctrl</kbd> + <kbd>c</kbd></kbd>@@
@@html: <kbd><kbd>Ctrl</kbd> + <kbd>e</kbd></kbd>@@
to bring up the org export dispatch, then
@@html: <kbd>s</kbd>@@ to select =Export to SSH config=, and
@@html: <kbd>x</kbd>@@ to overwrite my existing ssh config file with
the newly generated one.