Hacky scripts post Signed-off-by: Anirudh Oppiliappan <x@icyphox.sh>
Anirudh Oppiliappan x@icyphox.sh
Thu, 24 Oct 2019 20:53:50 +0530
2 files changed,
330 insertions(+),
1 deletions(-)
M
pages/blog/feed.xml
→
pages/blog/feed.xml
@@ -11,7 +11,168 @@ <link>https://icyphox.sh/blog/</link>
</image> <language>en-us</language> <copyright>Creative Commons BY-NC-SA 4.0</copyright> - <item><title>Status update</title><description><![CDATA[<p>I’ve decided to drop the “Weekly” part of the status update posts, since + <item><title>Hacky scripts</title><description><![CDATA[<p>As a CS student, I see a lot of people around me doing courses online +to learn to code. Don’t get me wrong – it probably works for some. +Everyone learns differently. But that’s only going to get you so far. +Great you know the syntax, you can solve some competitive programming +problems, but that’s not quite enough, is it? The actual learning comes +from <em>applying</em> it in solving <em>actual</em> problems – not made up ones. +(<em>inb4 some seething CP bro comes at me</em>)</p> + +<p>Now, what’s an actual problem? Some might define it as real world +problems that people out there face, and solving it probably requires +building a product. This is what you see in hackathons, generally.</p> + +<p>If you ask me, however, I like to define it as problems that <em>you</em> yourself +face. This could be anything. Heck, it might not even be a “problem”. It +could just be an itch that you want to scratch. And this is where +<strong>hacky scripts</strong> come in. Unclear? Let me illustrate with a few +examples.</p> + +<h3 id="now-playing-status-in-my-bar">Now playing status in my bar</h3> + +<p>If you weren’t aware already – I rice my desktop. A lot. And a part of +this cohesive experience I try to create involves a status bar up at the +top of my screen, showing the time, date, volume and battery statuses etc.</p> + +<p>So here’s the “problem”. I wanted to have my currently playing song +(Spotify), show up on my bar. How did I approach this? A few ideas +popped up in my head:</p> + +<ul> +<li>Send <code>playerctl</code>’s STDOUT into my bar</li> +<li>Write a Python script to query Spotify’s API</li> +<li>Write a Python/shell script to query Last.fm’s API</li> +</ul> + +<p>The first approach bombed instantly. <code>playerctl</code> didn’t recognize my +Spotify client and whined about some <code>dbus</code> issues to top it off. +I spent a while in that rabbit hole but eventually gave up.</p> + +<p>My next avenue was the Spotify Web API. One look at the <a href="https://developer.spotify.com/documentation/web-api/">docs</a> and +I realize that I’ll have to make <em>more</em> than one request to fetch the +artist and track details. Nope, I need this to work fast.</p> + +<p>Last resort – Last.fm’s API. Spolier alert, this worked. Also, arguably +the best choice, since it shows the track status regardless of where +the music is being played. Here’s the script in its entirety:</p> + +<div class="codehilite"><pre><span></span><code><span class="ch">#!/usr/bin/env bash</span> +<span class="c1"># now playing</span> +<span class="c1"># requires the last.fm API key</span> + +<span class="nb">source</span> ~/.lastfm <span class="c1"># `export API_KEY="<key>"`</span> +<span class="nv">fg</span><span class="o">=</span><span class="s2">"</span><span class="k">$(</span>xres color15<span class="k">)</span><span class="s2">"</span> +<span class="nv">light</span><span class="o">=</span><span class="s2">"</span><span class="k">$(</span>xres color8<span class="k">)</span><span class="s2">"</span> + +<span class="nv">USER</span><span class="o">=</span><span class="s2">"icyphox"</span> +<span class="nv">URL</span><span class="o">=</span><span class="s2">"http://ws.audioscrobbler.com/2.0/?method=user.getrecenttracks"</span> +<span class="nv">URL</span><span class="o">+=</span><span class="s2">"&user=</span><span class="nv">$USER</span><span class="s2">&api_key=</span><span class="nv">$API_KEY</span><span class="s2">&format=json&limit=1&nowplaying=true"</span> +<span class="nv">NOTPLAYING</span><span class="o">=</span><span class="s2">" "</span> <span class="c1"># I like to have it show nothing</span> +<span class="nv">RES</span><span class="o">=</span><span class="k">$(</span>curl -s <span class="nv">$URL</span><span class="k">)</span> +<span class="nv">NOWPLAYING</span><span class="o">=</span><span class="k">$(</span>jq <span class="s1">'.recenttracks.track[0]."@attr".nowplaying'</span> <span class="o"><<<</span> <span class="s2">"</span><span class="nv">$RES</span><span class="s2">"</span> <span class="p">|</span> tr -d <span class="s1">'"'</span><span class="k">)</span> + + +<span class="k">if</span> <span class="o">[[</span> <span class="s2">"</span><span class="nv">$NOWPLAYING</span><span class="s2">"</span> <span class="o">=</span> <span class="s2">"true"</span> <span class="o">]]</span> +<span class="k">then</span> + <span class="nv">TRACK</span><span class="o">=</span><span class="k">$(</span>jq <span class="s1">'.recenttracks.track[0].name'</span> <span class="o"><<<</span> <span class="s2">"</span><span class="nv">$RES</span><span class="s2">"</span> <span class="p">|</span> tr -d <span class="s1">'"'</span><span class="k">)</span> + <span class="nv">ARTIST</span><span class="o">=</span><span class="k">$(</span>jq <span class="s1">'.recenttracks.track[0].artist."#text"'</span> <span class="o"><<<</span> <span class="s2">"</span><span class="nv">$RES</span><span class="s2">"</span> <span class="p">|</span> tr -d <span class="s1">'"'</span><span class="k">)</span> + <span class="nb">echo</span> -ne <span class="s2">"%{F</span><span class="nv">$light</span><span class="s2">}</span><span class="nv">$TRACK</span><span class="s2"> %{F</span><span class="nv">$fg</span><span class="s2">}by </span><span class="nv">$ARTIST</span><span class="s2">"</span> +<span class="k">else</span> + <span class="nb">echo</span> -ne <span class="s2">"</span><span class="nv">$NOTPLAYING</span><span class="s2">"</span> +<span class="k">fi</span> +</code></pre></div> + +<p>The <code>source</code> command is used to fetch the API key which I store at +<code>~/.lastfm</code>. The <code>fg</code> and <code>light</code> variables can be ignored, they’re only +for coloring output on my bar. The rest is fairly trivial and just +involves JSON parsing with <a href="https://stedolan.github.io/jq/"><code>jq</code></a>. +That’s it! It’s so small, but I learnt a ton. For those curious, here’s +what it looks like running:</p> + +<p><img src="/static/img/now_playing.png" alt="now playing status polybar" /></p> + +<h3 id="update-latest-post-on-the-index-page">Update latest post on the index page</h3> + +<p>This pertains to this very blog that you’re reading. I wanted a quick +way to update the “latest post” section in the home page and the +<a href="/blog">blog</a> listing, with a link to the latest post. This would require +editing the Markdown <a href="https://github.com/icyphox/site/tree/master/pages">source</a> +of both pages.</p> + +<p>This was a very +interesting challenge to me, primarily because it requires in-place +editing of the file, not just appending. Sure, I could’ve come up with +some <code>sed</code> one-liner, but that didn’t seem very fun. Also I hate +regexes. Did a lot of research (read: Googling) on in-place editing of +files in Python, sorting lists of files by modification time etc. and +this is what I ended up on, ultimately:</p> + +<div class="codehilite"><pre><span></span><code><span class="ch">#!/usr/bin/env python3</span> + +<span class="kn">from</span> <span class="nn">markdown2</span> <span class="kn">import</span> <span class="n">markdown_path</span> +<span class="kn">import</span> <span class="nn">os</span> +<span class="kn">import</span> <span class="nn">fileinput</span> +<span class="kn">import</span> <span class="nn">sys</span> + +<span class="c1"># change our cwd</span> +<span class="n">os</span><span class="o">.</span><span class="n">chdir</span><span class="p">(</span><span class="s2">"bin"</span><span class="p">)</span> + +<span class="n">blog</span> <span class="o">=</span> <span class="s2">"../pages/blog/"</span> + +<span class="c1"># get the most recently created file</span> +<span class="k">def</span> <span class="nf">getrecent</span><span class="p">(</span><span class="n">path</span><span class="p">):</span> + <span class="n">files</span> <span class="o">=</span> <span class="p">[</span><span class="n">path</span> <span class="o">+</span> <span class="n">f</span> <span class="k">for</span> <span class="n">f</span> <span class="ow">in</span> <span class="n">os</span><span class="o">.</span><span class="n">listdir</span><span class="p">(</span><span class="n">blog</span><span class="p">)</span> <span class="k">if</span> <span class="n">f</span> <span class="ow">not</span> <span class="ow">in</span> <span class="p">[</span><span class="s2">"_index.md"</span><span class="p">,</span> <span class="s2">"feed.xml"</span><span class="p">]]</span> + <span class="n">files</span><span class="o">.</span><span class="n">sort</span><span class="p">(</span><span class="n">key</span><span class="o">=</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">getmtime</span><span class="p">,</span> <span class="n">reverse</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span> + <span class="k">return</span> <span class="n">files</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> + +<span class="c1"># adding an entry to the markdown table</span> +<span class="k">def</span> <span class="nf">update_index</span><span class="p">(</span><span class="n">s</span><span class="p">):</span> + <span class="n">path</span> <span class="o">=</span> <span class="s2">"../pages/_index.md"</span> + <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span> + <span class="n">md</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">readlines</span><span class="p">()</span> + <span class="n">ruler</span> <span class="o">=</span> <span class="n">md</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="s2">"| --- | --: |</span><span class="se">\n</span><span class="s2">"</span><span class="p">)</span> + <span class="n">md</span><span class="p">[</span><span class="n">ruler</span> <span class="o">+</span> <span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">s</span> <span class="o">+</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span> + + <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span> + <span class="n">f</span><span class="o">.</span><span class="n">writelines</span><span class="p">(</span><span class="n">md</span><span class="p">)</span> + +<span class="c1"># editing the md source in-place</span> +<span class="k">def</span> <span class="nf">update_blog</span><span class="p">(</span><span class="n">s</span><span class="p">):</span> + <span class="n">path</span> <span class="o">=</span> <span class="s2">"../pages/blog/_index.md"</span> + <span class="n">s</span> <span class="o">=</span> <span class="n">s</span> <span class="o">+</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span> + <span class="k">for</span> <span class="n">l</span> <span class="ow">in</span> <span class="n">fileinput</span><span class="o">.</span><span class="n">FileInput</span><span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="n">inplace</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span> + <span class="k">if</span> <span class="s2">"--:"</span> <span class="ow">in</span> <span class="n">l</span><span class="p">:</span> + <span class="n">l</span> <span class="o">=</span> <span class="n">l</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">l</span><span class="p">,</span> <span class="n">l</span> <span class="o">+</span> <span class="n">s</span><span class="p">)</span> + <span class="k">print</span><span class="p">(</span><span class="n">l</span><span class="p">,</span> <span class="n">end</span><span class="o">=</span><span class="s2">""</span><span class="p">),</span> + + +<span class="c1"># fetch title and date</span> +<span class="n">meta</span> <span class="o">=</span> <span class="n">markdown_path</span><span class="p">(</span><span class="n">getrecent</span><span class="p">(</span><span class="n">blog</span><span class="p">),</span> <span class="n">extras</span><span class="o">=</span><span class="p">[</span><span class="s2">"metadata"</span><span class="p">])</span><span class="o">.</span><span class="n">metadata</span> +<span class="n">fname</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">basename</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">splitext</span><span class="p">(</span><span class="n">getrecent</span><span class="p">(</span><span class="n">blog</span><span class="p">))[</span><span class="mi">0</span><span class="p">])</span> +<span class="n">url</span> <span class="o">=</span> <span class="s2">"/blog/"</span> <span class="o">+</span> <span class="n">fname</span> +<span class="n">line</span> <span class="o">=</span> <span class="n">f</span><span class="s2">"| [{meta['title']}]({url}) | `{meta['date']}` |"</span> + +<span class="n">update_index</span><span class="p">(</span><span class="n">line</span><span class="p">)</span> +<span class="n">update_blog</span><span class="p">(</span><span class="n">line</span><span class="p">)</span> +</code></pre></div> + +<p>I’m going to skip explaining this one out, but in essence, it’s <strong>one +massive hack</strong>. And in the end, that’s my point exactly. It’s very +hacky, but the sheer amount I learnt by writing this ~50 +line script can’t be taught anywhere.</p> + +<p>This was partially how +<a href="https://github.com/icyphox/vite">vite</a> was born. It was originally +intended to be a script to build my site, but grew into a full-blown +Python package. I could’ve just +used an off-the-shelf static site generator +given that there are <a href="https://staticgen.com">so many</a> of them, but +I chose to write one myself.</p> + +<p>And that just about sums up what I wanted to say. The best and most fun +way to learn to code – write hacky scripts. You heard it here.</p> +]]></description><link>https://icyphox.sh/blog/hacky-scripts</link><pubDate>Thu, 24 Oct 2019 00:00:00 +0000</pubDate><guid>https://icyphox.sh/blog/hacky-scripts</guid></item><item><title>Status update</title><description>< and +I realize that I'll have to make _more_ than one request to fetch the +artist and track details. Nope, I need this to work fast. + +Last resort -- Last.fm's API. Spolier alert, this worked. Also, arguably +the best choice, since it shows the track status regardless of where +the music is being played. Here's the script in its entirety: + +```shell +#!/usr/bin/env bash +# now playing +# requires the last.fm API key + +source ~/.lastfm # `export API_KEY="<key>"` +fg="$(xres color15)" +light="$(xres color8)" + +USER="icyphox" +URL="http://ws.audioscrobbler.com/2.0/?method=user.getrecenttracks" +URL+="&user=$USER&api_key=$API_KEY&format=json&limit=1&nowplaying=true" +NOTPLAYING=" " # I like to have it show nothing +RES=$(curl -s $URL) +NOWPLAYING=$(jq '.recenttracks.track[0]."@attr".nowplaying' <<< "$RES" | tr -d '"') + + +if [[ "$NOWPLAYING" = "true" ]] +then + TRACK=$(jq '.recenttracks.track[0].name' <<< "$RES" | tr -d '"') + ARTIST=$(jq '.recenttracks.track[0].artist."#text"' <<< "$RES" | tr -d '"') + echo -ne "%{F$light}$TRACK %{F$fg}by $ARTIST" +else + echo -ne "$NOTPLAYING" +fi +``` + +The `source` command is used to fetch the API key which I store at +`~/.lastfm`. The `fg` and `light` variables can be ignored, they're only +for coloring output on my bar. The rest is fairly trivial and just +involves JSON parsing with [`jq`](https://stedolan.github.io/jq/). +That's it! It's so small, but I learnt a ton. For those curious, here's +what it looks like running: + + + +### Update latest post on the index page + +This pertains to this very blog that you're reading. I wanted a quick +way to update the "latest post" section in the home page and the +[blog](/blog) listing, with a link to the latest post. This would require +editing the Markdown [source](https://github.com/icyphox/site/tree/master/pages) +of both pages. + +This was a very +interesting challenge to me, primarily because it requires in-place +editing of the file, not just appending. Sure, I could've come up with +some `sed` one-liner, but that didn't seem very fun. Also I hate +regexes. Did a lot of research (read: Googling) on in-place editing of +files in Python, sorting lists of files by modification time etc. and +this is what I ended up on, ultimately: + +```python +#!/usr/bin/env python3 + +from markdown2 import markdown_path +import os +import fileinput +import sys + +# change our cwd +os.chdir("bin") + +blog = "../pages/blog/" + +# get the most recently created file +def getrecent(path): + files = [path + f for f in os.listdir(blog) if f not in ["_index.md", "feed.xml"]] + files.sort(key=os.path.getmtime, reverse=True) + return files[0] + +# adding an entry to the markdown table +def update_index(s): + path = "../pages/_index.md" + with open(path, "r") as f: + md = f.readlines() + ruler = md.index("| --- | --: |\n") + md[ruler + 1] = s + "\n" + + with open(path, "w") as f: + f.writelines(md) + +# editing the md source in-place +def update_blog(s): + path = "../pages/blog/_index.md" + s = s + "\n" + for l in fileinput.FileInput(path, inplace=1): + if "--:" in l: + l = l.replace(l, l + s) + print(l, end=""), + + +# fetch title and date +meta = markdown_path(getrecent(blog), extras=["metadata"]).metadata +fname = os.path.basename(os.path.splitext(getrecent(blog))[0]) +url = "/blog/" + fname +line = f"| [{meta['title']}]({url}) | `{meta['date']}` |" + +update_index(line) +update_blog(line) +``` + +I'm going to skip explaining this one out, but in essence, it's **one +massive hack**. And in the end, that's my point exactly. It's very +hacky, but the sheer amount I learnt by writing this ~50 +line script can't be taught anywhere. + +This was partially how +[vite](https://github.com/icyphox/vite) was born. It was originally +intended to be a script to build my site, but grew into a full-blown +Python package. I could've just +used an off-the-shelf static site generator +given that there are [so many](https://staticgen.com) of them, but +I chose to write one myself. + +And that just about sums up what I wanted to say. The best and most fun +way to learn to code -- write hacky scripts. You heard it here.