Containerisation

[methanoid]

If RetroNAS is to be platform agnostic (ie ARM and x86) then a docker image would be awesome to have. I used to run unRAID for my local server before I went to cloud storage but being able to add a Docker to that with RetroNAS functionality would be a great option.

[danmons]

Docker is planned, however simply making the image isn't the challenge. Many people are going to want this to run on top of UnRAID/Synology/QNAP/TrueNAS/FreeNAS/etc. These systems already have modern SMB/CIFS stacks running, so a simple RetroNAS Docker container will conflict with that and not work.

I'll need to test various bridge/macvlan options in Docker primarily, but then also in each of those systems. Once that works without too many bugs, the Docker approach will definitely be happening.

As a short term workaround, if your NAS supports VMs, you can install a Debian 11 Bullseye VM and run RetroNAS in that. There's a rough guide on the wiki for VirtualBox on PC, which will be very similar to how any virtualisation system works. Just make sure you use a "bridge" network, and not a "NAT" network.

[TheMrChao]

Understand the legacy contradictions.

Looking forward to the container, for sure.

Might just run in ESXi in the meantime.

[birdybro]

Just checked with someone more knowledgeable than me to be sure I was right before chiming in... but for any IP or Hostname based file transfer protocol, docker should already be fine to use without any advanced configuration.

IPv4 forwarding is somewhat simple to setup --> https://docs.docker.com/network/bridge/#enable-forwarding-from-docker-containers-to-the-outside-world

Docker also creates a virtual MAC address for each container, so if you have ipv4 configured and have a very thin dhcp client on the container it should be relatively plug and play and widely compatible with any network, regardless of the docker host's configuration.

[miketweaver]

As a note for a docker image, I managed a PS3NetSRV image for Unraid and eventually had issues with it on Unraid.
aldostools/webMAN-MOD#333
https://forums.unraid.net/bug-reports/stable-releases/680-to-683-scandir-d_type-and-stat-errors-r942/

Just be aware this might be an issue that pops up eventually.

[sairuk]

I am sitting here trying to conceptualise this, what is the preference here?

• a single retronas container, where the retronas tool outputs and maintains an overarching docker-compose.yml
• service containers on a retronas net? where the retronas tool itself is capable of building and managing individual service based containers?

curious because arguably the latter fits with the current tool model moreso than the former.

[birdybro]

I think it's mainly for people that already have a NAS running something like unRAID who want to also add the functionality of RetroNAS (automatic generation of folders for MiSTer with the symlinking to a central roms folder, all the extra protocols for other legacy systems, etc...)

[sairuk]

I do get people want to incorporate it into an existing NAS to get the advantages retroNAS provides but the discussion has been pretty high level from what I've seen.

Let me kick the can further down the road as to where my thinking is with this,

note: I have some experience with Docker and containers and the hell that is docker networking but it is not day to day experience nor do I consider myself literate in the Docker space. I also note in theory the entire project could move to Docker if done correctly, not saying it should that is a much more broad conversation.

Noting my investment in docker is zero I have no interest yet in running it on a NAS for reasons.

• I really can't conceptualise the end product as it stands yet (so technically i'm curious)
• I don't trust QNAP nor Terramasters technical competence based on past experience, which are the two NAS i run.
• I don't hold massive collections of files anymore that dictate NAS usage

So with that in mind Docker integration for me docker may be used as a github actions pipeline scenario to validate commits and as such it would potentially be separated by service to be advantageous

Scope
What is in scope, what is RetroNAS actually responsible for here, as per Duplicated Services I don't think RetroNAS in docker is responsible for affecting NAS config

Infrastructure
How is it put together?

• Is the Dockerfile built live on the host after pulling the base layers from docker hub
• do we need to back this with a dockerhub account to build our own base images?

The kind of discussion I would like have from those asking is this, i.e. the stakeholders

Integration
multiple containers taking a per service approach at the container level with a backend net so you can selectively deploy services that make sense for your setup.

The installer that we currently know of would cover container deployment from a common base image but those with access to the different NAS product would need to advise who we integrate persistent services per platform

single container the end user will docker exec -it bash into the container to use the installer internally, the project itself requirements minimal change

• a base image can be pulled into an existing Docker instance from a dockerhub account where
• a base image can be built on debian:slim and incorporate retroNAS
• appropriate ipv4 forwarding is required or docker OR the docker net needs to be dropped into the trusted FW zone depending on end user setup to permit outgoing internet access

Duplicated Services
I would leave it up to the user to integrate duplicated services as they see fit based on their own environments as imho it should be outside of the scope of RetroNAS to do so.

Documentation guides with some basic approach sure but automated, no we can't assume anything as it could be breaking in some scenarios and if people are relying on automation here they may end up in an unacceptable outage scenrio they can't fix.

Resource Requirements
i wouldn't be looking to support anything but alpine based on the services there are in this project, even debian slim is huge in comparison but is that even feasible

[EmSixTeen]

I think your misgivings about Docker on peoples' NASs aren't really relevant. I and many others are definitely keen.

[sairuk]

No problems, I don't think I really have misgivings per se but we certainly need more technical insight from people experienced with each platform and work through what this will look like.

I watched some unraid and freenas overviews yesterday (I know nothing about them in particular) and unraid (unless it changed now) was said to assign ips for docker at a container level instead of host which in turn means you either need multiple ips available for each container or a proxy of some kind in front of it.

For retronas this may be problematic as a single container may not be an appropriate way forward.

Targeting dedicated pi hardware where we have free reign over all protocols and ports is one thing, integrating with docker on an existing system with duplicated services is whole new ball game. If we get this done there is no reason we couldn't move the pi solution to docker at the same time.

As I believe has been covered before we also need to revise our services approach if moving into docker, since systemd isn't a thing in that space, so that is one known roadblock currently

[sairuk]

We were having discussions around this internally a while back and made some notes on it but I don't think we linked it here, docker-aio

for completeness since I also can't remember if i linked these here,

• we have a hacky docker integration into the current project architecture here in a branch (which we have no intent of moving forward at this stage)
• a demo from @willoucom here

also I woke up to some movement from @natewalck in this space which looks interesting

[sairuk]

Please note: All further docker discussion will be moved to our dist repo here until the technical issues are resolved and a way forward is established. Then we will break the discussions up however they make sense.

[sairuk]

I"ve unlocked this thread as the retronas-dist repo needs to go bye bye at some stage

[sairuk]

Noting a couple of things, the entire ecosystem would be expected to be migrated to docker not just select services and we don't particularly want to take on the overhead/cost of a image repo, additionally in the spirit of opensource i'd rather people looked at OCI standards (Containerfile etc), rootful/rootless podman as applicable, etc.

[kfix]

I think a halfway point between VMming and app-containing (Docker, podman) could be reached by providing instructions to deploy retronas to a "system container" instance provided from a manager like Incus. Incus isn't available for most of the platforms mentioned in the 1st comment but its available from the stock Debian repositories so its already available to the current deployment targets for running retronas.

Incus containers are equivalent from an isolation perspective to docker/podman ones but aren't rigidly managed in terms of having a single app process that must stay alive - it watches a contained init manager (systemd, for Debian) which in turn runs all services that ansible would set up once its launched in the container.

Incus provides options for dedicating disks to the incus container and bind-mounting folder(s) from the host's storage.

As for networking, the choice is probably between converting the NAS's primary interface into a bridge which Incus can then add its containers to, or having Incus create & manage a macvlan with the ethernet interface as its parent. I don't think the downsides of macvlan (no networking between host and container) hurt the retronas usecases at all and an administrator can locally manage and import files from other storages using a shared mount and/or shelling into the incus container.

This setup means there is some service duplication between host & retronas, at least two Debian systemds, probably two sambas, but I think the different clients you have connecting to your host's modern samba vs the retronas warrants the additional memory usage in exchange for not having to keep both groups' samba configs meshed properly. Eventually we'll be at a point where we'll have to run a custom build of samba as upstream strips out legacy features....

[kfix]

So I finally tried this out and it seems to work! I'm just using SMB based services for the moment (OpenPS2Loader, recalbox)

the gist of my setup on a Debian Trixie homebuilt NAS:

apt install incus

# setup incus with its defaults (or however you want)
incus admin init

# add a macvlan network so your container can appear on your LAN as its own IP address
# (use the interface shown by `ip route show default` as parent=)
incus network create macvlan --type=macvlan parent=enp3s0

# create a container "instance" for retronas
# (I made it privileged so UID/GIDs carry over from the host OS)
incus launch images:debian/bookworm retronas --network=macvlan --config security.privileged=true

# bind mount some storage from your NAS to the instance. in my case, its from /tank-2025-1
incus config device add retronas nas-storage-area disk source=/tank-2025-1/retronas path=/data/retronas

# shell into the container
incus exec retronas bash

# precreate the retronas user
# (this will default to UID & GID 1000, which you might want to change match the primary uploader's userid from your NAS's OS)
adduser pi

# deploy retronas: https://github.com/retronas/retronas/wiki/Installing-RetroNAS

Now I have a virtual appliance that retronas can administer with some isolation from my NAS' own services (including its Samba)

Any modules based on the kernel's nfsd server are probably not going to work.