@bitspook's personal website: Org-Mode

@bitspook's personal website: Org-Mode - bitspook.inhttps://bitspook.in/tags/org-mode/feed.xml2024-09-08T12:36:45.718954ZCharanjit Singhhttps://bitspook.in/images/avatar.pngExtending org-mode to handle youtube links/blog/extending-org-mode-to-handle-youtube-links/index.html2022-09-26T00:00:00.000000ZCharanjit Singh2022-09-26T00:00:00.000000Z

<p> <iframe width="560" height="315" src="//youtube.com/embed/eaZUZCzaIgw" title="nil" frameborder="0" allowfullscreen=""></iframe> </p> <div id="outline-container-Requirement" class="outline-2"> <h2 id="Requirement">Requirement</h2> <div class="outline-text-2" id="text-Requirement"> <ol class="org-ol"> <li>I want to open youtube links in my notes in <code>mpv</code> instead of opening the browser</li> <li>When exporting to HTML, youtube links should become embedded videos instead of hyperlinks</li> </ol> </div> </div> <div id="outline-container-Implementation" class="outline-2"> <h2 id="Implementation">Implementation</h2> <div class="outline-text-2" id="text-Implementation"> <p> Org has a concept of 'link types'. We can add a new type of link and have it behave the way we want. </p> <ul class="org-ul"> <li><p> <code>org-link-parameters</code> </p> <p> This variable contains link types and how they behave on follow (i.e when someone opens the link with <code>C-c C-o</code>), export etc. A type of a link is determined by the string before first ":" in it. e.g "<a href="https://bitspook.in">https://bitspook.in</a>" has <code>https</code> type, "<a href="file:///etc">file:///etc</a>" has <code>file</code> type. </p> <p> Items in this list has 2 main components: </p> <ol class="org-ol"> <li>First element is string representing the link type</li> <li>Key-value pairs of behavior name (e.g <code>:follow</code>, <code>:export</code>) and function which handles that behavior</li> </ol> <p> For example: </p> <div class="org-src-container"> <pre class="src src-elisp">((<span style="color: #6aaf50;">"yt"</span> <span style="color: #7f7dca;">:follow</span> spook-org--follow-yt-link <span style="color: #7f7dca;">:export</span> spook-org--export-yt-link) (<span style="color: #6aaf50;">"eww"</span> <span style="color: #7f7dca;">:follow</span> org-eww-open <span style="color: #7f7dca;">:store</span> org-eww-store-link)) </pre> </div></li> <li><p> <code>org-link-set-parameters</code> </p> <p> This function is used to add new link-types (and also to add new behavior to existing ones). You can check the docs for <code>org-link-parameters</code> (with <code>C-h v org-link-parameters</code>) to see arguments provided to each type of callback. </p></li> </ul> <p> Here's the code I've added in <a href="https://github.com/bitspook/spookmax.d">my emacs config</a>: </p> <div class="org-src-container"> <pre class="src src-elisp">(<span style="color: #598bc1;">defun</span> <span style="color: #1d9a79;">spook-org--follow-yt-link</span> (path prefix) (<span style="color: #598bc1;">let*</span> ((url (format <span style="color: #6aaf50;">"https:%s"</span> path)) (proc-name (format <span style="color: #6aaf50;">"*yt://%s*"</span> url))) (<span style="color: #598bc1;">if</span> (<span style="color: #598bc1;">and</span> prefix (executable-find <span style="color: #6aaf50;">"mpv"</span>)) (browse-url url) (make-process <span style="color: #7f7dca;">:name</span> proc-name <span style="color: #7f7dca;">:buffer</span> proc-name <span style="color: #7f7dca;">:command</span> `(<span style="color: #6aaf50;">"mpv"</span> ,url)) (message <span style="color: #6aaf50;">"Launched mpv in buffer: %s"</span> proc-name)))) (<span style="color: #598bc1;">defun</span> <span style="color: #1d9a79;">spook-org--export-yt-link</span> (path desc backend) (<span style="color: #598bc1;">when</span> (eq backend 'html) (<span style="color: #598bc1;">let*</span> ((video-id (cadar (url-parse-query-string path))) (url (<span style="color: #598bc1;">if</span> (string-empty-p video-id) path (format <span style="color: #6aaf50;">"//youtube.com/embed/%s"</span> video-id)))) (format <span style="color: #6aaf50;">"<iframe width=\"560\" height=\"315\" src=\"%s\" title=\"%s\" frameborder=\"0\" allowfullscreen></iframe>"</span> url desc)))) (org-link-set-parameters <span style="color: #6aaf50;">"yt"</span> <span style="color: #7f7dca;">:follow</span> #'spook-org--follow-yt-link <span style="color: #7f7dca;">:export</span> #'spook-org--export-yt-link) </pre> </div> <p> <code>make-process</code> will create a background buffer named <code>*yt://<url>*</code> which allow monitoring and killing the mpv process. </p> <p> <i>Update</i>: Use <code>make-process</code> instead of <code>async-shell-command</code> for launching mpv. Thanks to <a href="https://www.reddit.com/r/emacs/comments/xls0ih/extending_orgmode_to_handle_youtube_links/iplpk57/">nv-elisp on /r/emacs</a> </p> </div> </div> <div id="outline-container-Result" class="outline-2"> <h2 id="Result">Result</h2> <div class="outline-text-2" id="text-Result"> <ol class="org-ol"> <li><code>C-c C-o</code> (i.e <code>org-open-at-point</code>) on a <code>yt://</code> link opens the youtube video in mpv</li> <li><code>C-u C-c C-o</code> (i.e <code>org-open-at-point</code> with an prefix argument) on a <code>yt://</code> link opens the video in browser</li> <li>When exported to HTML, all <code>yt://</code> links file are exported as embedded youtube videos</li> </ol> </div> </div> <div id="outline-container-Bonus%20gains%20%F0%9F%92%AA" class="outline-2"> <h2 id="Bonus%20gains%20%F0%9F%92%AA">Bonus gains 💪</h2> <div class="outline-text-2" id="text-Bonus%20gains%20%F0%9F%92%AA"> <p> Recently I've been building a tool (<a href="https://github.com/bitspook/cl-ownpress">cl-ownpress</a>) which to make blogging a zero-effort activity for me. Since I have already built a habit of judicious note taking, publishing a subset of my notes will enable maintaining an active blog without doing any "work". </p> <p> I've also been making thickly-accented-awkardly-narrated youtube videos. These are almost always <i>tldr;</i> videos for my blog posts. So they get embedded in my blog posts. </p> <p> 'Blog with notes + embed youtube videos' become easier to do with this little tinkering. Since <a href="https://github.com/bitspook/cl-ownpress">cl-ownpress</a> uses my running Emacs to publish my posts, I don't need to make any change in my blogging setup. I can embed my <i>tldr;</i> videos in my blog posts by just prefixing youtube links with <code>yt://</code> instead of <code>https://</code>. </p> <p> I am pretty happy that I got zero work blogging-setup, and zero work extending-the-blogging-setup as well. </p> </div> </div> Using org-mode as an SQL playground/blog/using-org-mode-as-an-sql-playground/index.html2022-02-03T00:00:00.000000ZCharanjit Singh2022-02-03T00:00:00.000000Z

<p> Showcase video: </p> <iframe width="560" height="315" src="https://www.youtube.com/embed/DY7t4T4TmUU" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe> <p> For every web app I work on, a database client is always present for exploring data and building complex queries. Recently, I have moved on from my PgAdmin to org-mode for this purpose, because why not. </p> <p> Org-mode allows executing code snippets and can show results in a nice tabular form. But org-mode isn't <a href="https://www.cvedetails.com/vulnerability-list/vendor_id-26/product_id-320/Microsoft-Office.html">written by Microsoft</a>. So it don't allow arbitrary code execution without our explicit consent. To tell org-mode that it is okay to execute SQL snippets, we need to run following elisp snippet: </p> <div class="org-src-container"> <pre class="src src-elisp">(<span style="color: #598bc1;">with-eval-after-load</span> 'org (org-babel-do-load-languages 'org-babel-load-languages '((sql . t)))) </pre> </div> <p> Org-mode has a number of <a href="https://www.orgmode.org/worg/org-contrib/babel/languages/ob-doc-sql.html#org7f0941b">header arguments for SQL blocks</a> which can be used to configure how a particular SQL source-code snippet is executed. Most important ones perhaps are: </p> <ul class="org-ul"> <li><code>:engine</code> which tells which database to run the SQL against</li> <li><code>:dbhost</code> to configure the database host, most likely <code>localhost</code> for a dev environment</li> <li><code>:dbport</code></li> <li><code>:dbuser</code> and <code>:dbpassword</code> for database authentication</li> <li><code>:database</code> for database name</li> </ul> <p> By default org-mode runs the code snippets in current OS environment. I use this fact to get the ability to run SQL against our postgres database without passing most of the above arguments. Postgres allows setting <a href="https://www.postgresql.org/docs/current/libpq-envars.html">some environment</a> variables, which it then use to for making connections by default. In my <a href="https://github.com/bitspook/entropy/blob/ae5ee52646c2c67ce713938548d67a722b73a294/flake.nix">flake.nix</a> file, I setup the required environment variables: </p> <div class="org-src-container"> <pre class="src src-sh"><span style="color: #b6b635;">PGDIR</span>=$<span style="color: #b6b635;">PWD</span>/storage/postgres <span style="color: #7f7dca;">export</span> <span style="color: #b6b635;">PGDATA</span>=$<span style="color: #b6b635;">PGDIR</span>/data <span style="color: #7f7dca;">export</span> <span style="color: #b6b635;">PGHOST</span>=$<span style="color: #b6b635;">PGDIR</span>/run <span style="color: #7f7dca;">export</span> <span style="color: #b6b635;">PGDATABASE</span>=entropy <span style="color: #7f7dca;">export</span> <span style="color: #b6b635;">DATABASE_URL</span>=<span style="color: #6aaf50;">"postgresql:///$PGDATABASE?host=$PGHOST"</span> <span style="color: #858585; font-style: italic;"># </span><span style="color: #858585; font-style: italic;">not required by postgres, but will be helpful</span> </pre> </div> <p> With this, only header option left to set is <code>:engine</code>, which I prefer to configure globally for my <a href="https://raw.githubusercontent.com/bitspook/entropy/96cc9c0485d972ed819996d61c3730cafb228660/sql-playground.org">sql-playground.org</a> since I usually have one database for my personal projects. You can check this <a href="https://github.com/bitspook/entropy/commit/96cc9c0485d972ed819996d61c3730cafb228660">commit on Entropy project</a> for how I have setup my sql-playground there. Following snippet on top of the sql-playground.org does the trick: </p> <div class="org-src-container"> <pre class="src src-org"><span style="color: #858585; font-style: italic;">#+PROPERTY: header-args:sql :engine postgres</span> </pre> </div> <p> Now any SQL snippets that we create can be executed against our development database right within the org file. For example, pressing <code>C-c C-c</code> in following SQL block produces output shown below that. </p> <div class="org-src-container"> <pre class="src src-org"><span style="color: #858585; font-style: italic;">#+name: events</span> <span style="color: #858585; font-style: italic; text-decoration: underline;">#+begin_src sql</span> <span style="color: #aab0ab;"> </span><span style="color: #598bc1;">SELECT</span><span style="color: #aab0ab;"> events.title,</span> <span style="color: #aab0ab;"> events.slug,</span> <span style="color: #aab0ab;"> start_time,</span> <span style="color: #aab0ab;"> events.id,</span> <span style="color: #aab0ab;"> events.</span><span style="color: #598bc1;">source</span><span style="color: #aab0ab;">,</span> <span style="color: #aab0ab;"> format(</span><span style="color: #6aaf50;">'%s mins'</span><span style="color: #aab0ab;">, </span><span style="color: #7f7dca;">extract</span><span style="color: #aab0ab;">(epoch </span><span style="color: #598bc1;">from</span><span style="color: #aab0ab;"> (events.end_time - events.start_time))/60)</span> <span style="color: #aab0ab;"> </span><span style="color: #598bc1;">FROM</span><span style="color: #aab0ab;"> events;</span> <span style="color: #858585; font-style: italic; text-decoration: overline;">#+end_src</span> <span style="color: #858585; font-style: italic;">#+RESULTS: events</span> <span style="color: #86b5e8;">| title | slug | start_time | end_time | source | format |</span> <span style="color: #86b5e8;">|--------------------+----------------------+---------------------+---------------------+--------+---------|</span> <span style="color: #86b5e8;">| Lol bro what | lol-rofl-lmao | 2021-11-29 12:30:00 | 2021-11-29 13:30:00 | local | 60 mins |</span> <span style="color: #86b5e8;">| Nested event title | nested-lol-rofl-lmao | 2021-10-29 12:30:00 | 2021-10-29 13:30:00 | local | 60 mins |</span> </pre> </div> <div id="outline-container-Utilizing%20LSP" class="outline-2"> <h2 id="Utilizing%20LSP">Utilizing LSP</h2> <div class="outline-text-2" id="text-Utilizing%20LSP"> <p> Building SQL queries is my primary use-case, and org-mode configured so far serves that purpose. But since I am putting all this effort in writing this blog post anyway, I thought let's go a step further and try configuring LSP mode for SQL snippets in my org file. To my surprise, it is possible. <a href="https://github.com/lighttiger2505/sqls/">SQLS</a> provides an LSP server for SQL files, which once configured can provide certain features like auto-completing table and column names. I have created a <a href="https://github.com/bitspook/entropy/blob/96cc9c0485d972ed819996d61c3730cafb228660/.dir-locals.el">.dir-locals</a> file for <a href="https://www.gnu.org/software/emacs/manual/html_node/emacs/Directory-Variables.html">adding directory local variables</a> to configure LSP to connect to correct database. </p> <div class="org-src-container"> <pre class="src src-elisp">((nil . ((eval (<span style="color: #598bc1;">lambda</span> () (<span style="color: #598bc1;">setq</span> lsp-sqls-connections `(((driver . <span style="color: #6aaf50;">"postgresql"</span>) (dataSourceName . ,(getenv <span style="color: #6aaf50;">"DATABASE_URL"</span>)))))))))) </pre> </div> <p> This snippet isn't actually the right way configuring directory local variables, becuase it sets <code>lsp-sqls-connections</code> globally. This is the result of me time-boxing the task of evaluating <code>(getenv "DATABASE_URL")</code>, so I won't need to duplicate my database configuration here. </p> <p> LSP for org-mode also need to provide a file to which it can tangle source blocks. So I added another snippet to top of my org file. </p> <div class="org-src-container"> <pre class="src src-org"><span style="color: #858585; font-style: italic;">#+PROPERTY: header-args:sql :tangle test.sql</span> </pre> </div> <p> I can now run <code>lsp-org</code> in an SQL block, and 8/10 times it has lsp configured properly. It isn't flawless though, but that is expected since <a href="https://emacs-lsp.github.io/lsp-mode/manual-language-docs/lsp-org/#">lsp support in org-snippets</a> is in alpha stage right now. </p> <p> I also tried setting up lsp support in edit buffer for SQL statements (which you get by pressing <code>C-c '</code> in org src blocks), but because of a <a href="https://github.com/emacs-lsp/lsp-mode/issues/2525">bug in lsp-mode</a>, that didn't work out too well. </p> <div class="org-src-container"> <pre class="src src-elisp">(<span style="color: #598bc1;">defun</span> <span style="color: #1d9a79;">org-babel-edit-prep:sql</span> (babel-info) (<span style="color: #598bc1;">setq-local</span> buffer-file-name (<span style="color: #598bc1;">->></span> babel-info caddr (alist-get <span style="color: #7f7dca;">:tangle</span>))) (<span style="color: #598bc1;">setq-local</span> lsp-buffer-uri (<span style="color: #598bc1;">->></span> babel-info caddr (alist-get <span style="color: #7f7dca;">:tangle</span>) lsp--path-to-uri)) (<span style="color: #598bc1;">setq-local</span> lsp-headerline-breadcrumb-enable nil) (lsp)) </pre> </div> </div> </div> <div id="outline-container-Problems%20with%20this%20setup" class="outline-2"> <h2 id="Problems%20with%20this%20setup">Problems with this setup</h2> <div class="outline-text-2" id="text-Problems%20with%20this%20setup"> <p> There are 2 main problems I face during my primary use-case. </p> <ol class="org-ol"> <li>Org tables don't work well if a column has a lot of data. e.g <code>description</code> of an <code>event</code> messes up the table real bad. lsp-mode allow executing queries which don't suffer from this.</li> <li>No LSP support in edit buffer is a major bummer.</li> </ol> </div> </div> Using org-roam as a CRM/blog/using-org-roam-as-a-crm/index.html2022-01-22T00:00:00.000000ZCharanjit Singh2022-01-22T00:00:00.000000Z