local DNS server via dnsmasq (#2168)

* Use dnsmasq to create virtual hosts and mock DNS management for custom domains

- dnsmasq docker image
- dnsmasq network bridge
- point *.sndev to 127.0.0.1
- set-dnsmasq script
- -- add/remove/list dns records in dnsmasq.conf
- add 'domains' to sndev
- 'sndev domains dns' referencing set-dnsmasq script

* restart dnsmasq if add/remove succeeded

* add domain to /etc/hosts; cleanup

* tell if the command needs sudo permission

* add directions for dnsmasq DNS server usage

* add --no-hosts flag to skip asking to edit /etc/hosts

* add domains command to README.md

* add dnsmasq instructions to README.md

* correct exit on usage function; final cleanup and comments

* portable bash; use default network for dnsmasq; set a version for dnsmasq image

* POSIX compliance, add env var to .env.development, adjust README

* ignore dnsmasq.conf edits, use template instead

* use extra configs for dnsmasq, more POSIX compliance

* fix --no-hosts flag recognition, light cleanup

* shift 4 only if the command has enough args; more error messages; adjust TXT type only on list

* different sed syntax for macOS

Author: Soxasora

.env.development

@@ -190,4 +190,9 @@ CPU_SHARES_IMPORTANT=1024
 CPU_SHARES_MODERATE=512
 CPU_SHARES_LOW=256
 
-NEXT_TELEMETRY_DISABLED=1
+NEXT_TELEMETRY_DISABLED=1
+
+# custom domains stuff
+# local DNS server for custom domain verification, by default it's dnsmasq.
+# reachable by containers on 172.30.0.2(:53), outside of docker with 0.0.0.0:5353
+DOMAINS_DNS_SERVER=172.30.0.2

.gitignore

@@ -70,4 +70,7 @@ scripts/twitter-link-extract.config.json
 scripts/twitter-links.db
 
 # pay-awards
-scripts/pay-awards.config.json
+scripts/pay-awards.config.json
+
+# dnsmasq
+docker/dnsmasq/dnsmasq.d/*

README.md

@@ -87,6 +87,9 @@ COMMANDS
     psql                   open psql on db
     prisma                 run prisma commands
 
+  domains:
+    domains                custom domains dev management
+
   dev:
     pr                     fetch and checkout a pr
     lint                   run linters
@@ -150,6 +153,17 @@ After `nlp-setup` is done, restart your containers to enable semantic search:
 > ./sndev restart
 ```
 
+#### Local DNS via dnsmasq
+
+To enable dnsmasq:
+
+- domains should be enabled in `COMPOSE_PROFILES`:
+
+    ```.env
+    COMPOSE_PROFILES=...,domains,...
+    ```
+
+To add/remove DNS records you can now use `./sndev domains dns`. More on this [here](#add-or-remove-dns-records-in-local).
 
 <br>
 
@@ -449,6 +463,25 @@ To enable Web Push locally, you will need to set the `VAPID_*` env vars. `VAPID_
 
 <br>
 
+## Custom domains
+
+### Add or remove DNS records in local
+
+A worker dedicated to verifying custom domains, checks, among other things, if a domain has the correct DNS records and values. This would normally require a real domain and access to its DNS configuration. Therefore we use dnsmasq to have local DNS, make sure you have [enabled it](#local-dns-via-dnsmasq).
+
+To add a DNS record the syntax is the following:
+
+`./sndev domains dns add|remove cname|txt <name/domain> <value>`
+
+For TXT records, you can also use `""` quoted strings on `value`.
+
+To list all DNS records present in the dnsmasq config: `./sndev domains dns list`
+
+#### Access a local custom domain added via dnsmasq
+sndev will use the dnsmasq DNS server by default, but chances are that you might want to access the domain via your browser.
+
+For every edit on dnsmasq, it will give you the option to either edit the `/etc/hosts` file or use the dnsmasq DNS server which can be reached on `127.0.0.1:5353`. You can avoid getting asked to edit the `/etc/hosts` file by adding the `--no-hosts` parameter.
+
 # Internals
 
 <br>

docker-compose.yml

@@ -119,6 +119,9 @@ services:
     command:
       - npm run worker:dev
     cpu_shares: "${CPU_SHARES_IMPORTANT}"
+    networks:
+      - default
+      - domains-network
   imgproxy:
     container_name: imgproxy
     image: darthsim/imgproxy:v3.23.0
@@ -806,6 +809,28 @@ services:
       CONNECT: "localhost:${LNBITS_WEB_PORT}"
       TORDIR: "/app/.tor"
     cpu_shares: "${CPU_SHARES_LOW}"
+  dnsmasq:
+    image: 4km3/dnsmasq:2.90-r3
+    profiles:
+      - domains
+    container_name: dnsmasq
+    restart: unless-stopped
+    ports:
+      - "5353:53/tcp"
+      - "5353:53/udp"
+    command:
+      - --no-daemon
+      - --address=/.sndev/127.0.0.1
+      - --conf-file=/etc/dnsmasq.conf
+      - --conf-dir=/etc/dnsmasq.d
+    volumes:
+      - ./docker/dnsmasq/dnsmasq.conf:/etc/dnsmasq.conf
+      - ./docker/dnsmasq/dnsmasq.d:/etc/dnsmasq.d
+    cpu_shares: "${CPU_SHARES_LOW}"
+    networks:
+      domains-network:
+        ipv4_address: 172.30.0.2
+
 volumes:
   db:
   os:
@@ -819,3 +844,13 @@ volumes:
   nwc_recv:
   tordata:
   eclair:
+  dnsmasq:
+
+networks:
+  default: {}
+  domains-network:
+    name: domains-network
+    driver: bridge
+    ipam:
+      config:
+        - subnet: 172.30.0.0/24

docker/dnsmasq/dnsmasq.conf

@@ -0,0 +1,12 @@
+server=1.1.1.1
+no-resolv
+bind-interfaces
+listen-address=0.0.0.0
+
+log-queries
+log-facility=/var/log/dnsmasq.log
+
+# example of cname and txt for custom domains verification
+# this is to be edited by sndev cli or manually
+cname=www.pizza.sndev,sn.sndev
+txt-record=_snverify.www.pizza.sndev,"EXAMPLE_TXT_VALUE"