1---
  2template:
  3slug: building-forlater
  4title: How I built forlater.email
  5subtitle: A technical breakdown of my first big side-project
  6date: 2021-09-25
  7---
  8
  9Ever since I began browsing sites like Hacker News and Lobsters, coming
 10across new and exciting links to check out every day, I found it hard to
 11keep up. On most days, I just didn't. And that's fine -- [good,
 12even](/blog/dont-news). But oftentimes, I'd come across a genuinely
 13interesting link but no time to actually read it.
 14
 15I began using Pocket. It was alright -- the article view was very good;
 16but it stopped there. I didn't like nor use the other junk baked into
 17the app: discover, following/friends thing, etc. It's also proprietary,
 18and that irked me -- more so than the other "features".
 19
 20Thus, somewhat inspired by rss2email, I began building
 21[forlater.email](https://forlater.email) -- a bookmarking/read-later
 22service that works via email. Email is the perfect tool for this
 23use-case: works offline; you can organize it however you like; you own
 24your data.
 25
 26![forlater arch](https://cdn.icyphox.sh/JNAn4.png)
 27
 28Pictured above is how forlater works. Each component is explained below.
 29
 30## OpenSMTPD
 31
 32Mail containing links to be saved arrive here. OpenSMTPD is beautiful
 33software, and its configuration is stupid simple
 34([smtpd.conf(5)](https://man.openbsd.org/smtpd.conf)):
 35
 36```conf
 37table blocklist file:/etc/smtpd/blocklist
 38
 39action webhook mda "/home/icy/forlater/mdawh/mdawh"
 40match mail-from <blocklist> for any reject
 41match from any for rcpt-to "save@forlater.email" action webhook
 42```
 43
 44The `filter` and `listen` directives have been snipped for brevity. The
 45rest, in essence, simply sends all mail to `save@forlater.email` to an
 46MDA program, via stdin. Any mail from an address in the blocklist file
 47get rejected.
 48
 49[rspamd](https://rspamd.com) is used to prevent spam.
 50
 51## mdawh
 52
 53[mdawh](https://git.icyphox.sh/forlater/mdawh), or the MDA webhook tool.
 54A small Go program that processes mail coming from stdin and generates a
 55JSON payload that looks like so:
 56
 57```json
 58{
 59  "from": "foo@bar.com",
 60  "date": "Fri, 1 Jan 2010 00:00:00 UTC",
 61  "replyto": "...",
 62  "body": "...",
 63  "parts": {
 64    "text/plain": "...",
 65    "text/html": "...",
 66  }
 67}
 68```
 69
 70This is POSTed to a configured HTTP endpoint -- which in this case, is
 71navani.
 72
 73## navani
 74
 75[navani](https://git.icyphox.sh/forlater/navani) is forlater's primary
 76mail processing service[^1]. Listens for webhooks from mdawh, processes
 77them, and sends mail using a configured SMTP server. URLs are cached in
 78Redis along with the HTML content.
 79
 80For the readable HTML,
 81[go-readability](https://github.com/go-shiori/go-readability) is used;
 82the output of which is rendered into a minimal [HTML email
 83template](https://git.icyphox.sh/forlater/navani/tree/templates/html.tpl)
 84-- something that I never want to write again.
 85
 86The plaintext part is currently generated using `lynx -image_links -dump
 87-stdin`. The `-image_links` flag is handy because it generates footnote
 88links for images as well, instead of simply ignoring images altogether.
 89I plan to rewrite this; possibly using a blend of HTML-to-plaintext
 90libraries and handwritten rules.
 91
 92## future improvements
 93
 94I plan to implement some kind of `settings@` address to configure and
 95store user settings (dark theme? fonts?). However, this introduces state
 96in an otherwise mostly stateless system.
 97
 98The other thing I've been thinking of is making your own newsletter of
 99sorts. For example: save a bunch of links during the week, and have them
100all delivered over the weekend.
101
102Neither of these "features" are confirmed to happen, primarily because
103forlater is feature-complete for my use. That said, I'm happy to
104consider any improvements or suggestions that you might have -- please
105[email me](mailto:x@icyphox.sh).
106
107Finally, thanks to everyone who tossed a few bucks my way -- mighty kind
108of you.
109
110[^1]: Named after [Navani Kholin](https://coppermind.net/wiki/Navani_Kholin).