Skip to content

Commit d67e419

Browse files
committed
update to APT cacher
1 parent 4a5732d commit d67e419

File tree

2 files changed

+130
-16
lines changed

2 files changed

+130
-16
lines changed

_posts/draft-2025-03-17-APT-Cacher-NG.md

Lines changed: 130 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ pin: false
1010
toc: true
1111
---
1212

13-
# APT Cacher NG - About and "How To"
13+
# APT Cacher NG - About and over view of the
1414

1515
## Overview of APT cacher NG
1616
APT-Cacher-NG is designed to cache Ubuntu, Debian and other Linux distrobutions and packages locally. When one of your machines updates, those updates are stored here. The next machine that is the same or similar can pull from the cache locally, instead of going out to the internet again.
@@ -21,46 +21,160 @@ APT-Cacher-NG is designed to cache Ubuntu, Debian and other Linux distrobutions
2121
3. We are helping Linux, being a good steward of the internet. If we all do this, that will be less download bandwidth on these distro's. That will allow them to designate resources elsewhere than handling all the requests.
2222

2323

24-
[ANCHOR LINK TEST](https://vscode.dev/github/TheColetrain/thecoletrain.github.io/blob/main/_posts/draft-2025-03-17-APT-Cacher-NG.md#L24)
24+
[ANCHOR LINK TEST - UPDATE OR DELETE](https://vscode.dev/github/TheColetrain/thecoletrain.github.io/blob/main/_posts/draft-2025-03-17-APT-Cacher-NG.md#L24)
2525

26-
## Reccomendation. Should you do this on your server? (Or rasberry Pie, or whtever)
26+
## Reccomendation. Should you do this on your server?
2727
### Short answer is YES! (If you know "you are doing this" and need a guide, skip over the rest of this section and get to the "How To area".)
2828
- I will guide you through a "basic" method, this method will have you set up and running in minutes! Roughly only a few *copy pasta* steps to get APT Casher NG running! The basic method is so easy, even I can do it. So for the time spent to get anything done is a WIN. Amount Cached / Time Spent = Winner Winner Chicken dinner.
2929
- I will also then advise of the advanced options and "tweeks" if you want to mess with it further. If you want it to be as efficient as possible.
3030
### The long answer is maybe...
31-
- **Phase 1** If you do the short initial version that I will propose, your cache percentages will not be that significant. (My personal data will be below) i.e. you won't cache everything. The tradeoff is how EASY this first version is. So, you can set up a LXC that caches using scripts and copy/paste in minutes, and if it caches anyting at all... that's a win! The LXC will take very little resources and not that much storage, and you lost little time.... = win.
32-
- **Phase 2** If you go to the next phase, you can precache a little more, wont take any more CPU or RAM, but it will take a couple Gig's of storage. Not that much harder. Once more copy/paste, and press a GUI button.
31+
- **Phase 1** If you do the short initial version that I will propose, your cache percentages will not be that significant. (My personal data will be far below) i.e. you won't cache everything. The tradeoff is how EASY this first version is. So, you can set up a LXC that caches using scripts and copy/paste in minutes, and if it caches anything at all... that's a win! The LXC will take very little resources and not that much storage, and you lost little time.... = win.
32+
- **Phase 2** If you go to the next phase, you can precache a little more, wont take any more CPU or RAM, but it will take a couple Gig's of storage. Not that much harder. Once more, pretty easy, copy/paste, and press a GUI button.
3333
- **Phase 3** Phase three gets a little tricky. This is where you are "going for it" Trying to cache everything under the sun and satisfy all of your obsessive compulsive desires. I tried this level and got sick of it, turned it off. This involves having updates fail, and then going in to the settings and updating them to resolve. Quite honestly, I dont much know what I am doing, perhaps someone with a little knowlege would say *"It's so easy, you dont know what you are talking about,"* and they would be Right!
3434
### This wont cache HTTPS, it will only cache HTTP.
3535

3636

3737
## HOW TO DO IT
38-
- **Phase 1**
39-
1. I have proxmox. I like Proxmox. I like LXC's. I like "easy." Paste this script into your main PVE console.
38+
- ### **Phase 1**
39+
1. I have proxmox. I like Proxmox. I like LXC's. I like "easy." Paste this script into your **main PVE console.** (There is an `apt-get install` out there if you want to make an Ubuntu or Debain VM as well.) I wil be using the Proxmox LXC becasue it is lightweight and easy.
4040
```
4141
bash -c "$(wget -qLO - https://github.com/community-scripts/ProxmoxVE/raw/main/ct/apt-cacher-ng.sh)"
4242
```
43-
If you have not heard of Proxmox Helper Scripts, you should look around at that some more when you are done with this! (*A lot of these were created by, and Used to be run by *tteck*, if that helps ring a bell.*) [Proxmox Helper Scripts - APT Cacher NG](https://community-scripts.github.io/ProxmoxVE/scripts?id=apt-cacher-ng) *You can trust me, of course, but don't trust every random script on the internet!*
43+
- If you have not heard of Proxmox Helper Scripts, you should look around at that some more when you are done with this! (*A lot of these were created by, and used to be run by **tteck**, if that helps ring a bell.*)\
44+
[Proxmox Helper Scripts - APT Cacher NG](https://community-scripts.github.io/ProxmoxVE/scripts?id=apt-cacher-ng)
4445
:-)
4546
Cant get easier than **Copy Pasta** Right? Follow the prompts.
4647

4748
2. When that install finishes, you will get something like this.
49+
---
4850

4951

5052
🚀 Apt-Cacher-NG setup has been successfully initialized!
5153
💡 Access it using the following URL:
5254
🌐 http://192.168.1.215:3142/acng-report.html
5355

54-
Click the link, or type/paste in a web browser. This is the Apt Cache GUI. You won't have anything there yet. So just save it or note it for later.
55-
- Also, make sure you have set up a DHCP reservation, or go in and make a static IP, so the IP doesnt change on you.
56+
---
5657

57-
3. In the console of your newly created LXC, other than the GUI, (above) this will be where you go to update. You wont have to go thereunless you choose Phase 2 & 3.
58-
**`/etc/apt-cacher-ng/acng.conf`**
59-
Its just of note. The default config is 'PassThroughPattern: .*' This allows all HTTPS to pass. (No errors, updates just work. I.e. the update will update.)
58+
- Click the link, or type/paste in a web browser. This is the APT Cache GUI. You won't have anything there yet. So just save it or note it for later.
59+
- Also, make sure you have set up a DHCP reservation, or go in and make a static IP, so the IP doesnt change on you.
6060

61-
4. The server is set! I told you that was easy. The final step of **Phase 1**, is to update the client. Here is the easy way, and then I will explain.
62-
This is the command. paste this in the CLIENT.
61+
#### THE SERVER IS SET!
62+
3. I told you that was easy. The final step of **Phase 1**, is to update the client. Here is the easy way, and then I will explain.
63+
This is the command. paste this in the CLIENT. (Do a VM, or other linux box, LXC's are a tad different.) Replace the IP below with your Server IP.
64+
- This comand is telling the VM Ubuntu insance, to look ot the Server when it runs an update.
6365
```bash
6466
echo 'Acquire::http::Proxy "http://192.168.1.215:3142";' | sudo tee /etc/apt/apt.conf.d/02proxy
6567
```
66-
(above) Update the IP, to your server IP (step 2) But that is it. You are done with **Level 1 / Phase 1** You are caching. Run an update on the newly configured client, and then go back to your GUI and see how you did! `http://192.168.1.215:3142/acng-report.html`
68+
#### THE CLIENT IS SET
69+
- That is it. You are done with **Level 1 / Phase 1** You are caching. Run an update on the newly configured client, and then go back to your GUI and see how you did! `http://192.168.1.215:3142/acng-report.html`
70+
71+
72+
4. #### NOTES on what we did, and more on Phase 1.
73+
- The default config in the LXC Server is `PassThroughPattern: .*` located in **`/etc/apt-cacher-ng/acng.conf`**\
74+
I had things cache, using this config. Supposedly, it lets everything through, but I did not have that result. It cached things for me.\
75+
This allows all HTTPS to pass. (No errors, updates just work. I.e. the update will update without issues. But will not cache everything)
76+
- If you want to bump up the deafult a bit higher, and still stay pretty maintenance free, use this: `PassThroughPattern: ^(.*):443$` This is the config that I decided to stay with. More on this later if you choose to contunue to Phase 3, or if you want to note that, for when you mess up in phase 3.\
77+
78+
- **LXC's** - LXC Containers are a little different. You can see the similarity, and quite honestly, I am not sure what the extra does, but you can see how similar to VM's it is. (Replace the IP with your Server IP)
79+
```bash
80+
echo "Acquire::http::Proxy-Auto-Detect \"/usr/local/bin/apt-proxy-detect.sh\";" >/etc/apt/apt.conf.d/00aptproxy
81+
cat <<EOF >/usr/local/bin/apt-proxy-detect.sh
82+
#!/bin/bash
83+
if nc -w1 -z 192.168.1.215 3142; then
84+
echo -n "http://192.168.1.215:3142"
85+
else
86+
echo -n "DIRECT"
87+
fi
88+
EOF
89+
chmod +x /usr/local/bin/apt-proxy-detect.sh
90+
```
91+
- Got that from here: [tteck Github Proxmox discussion](https://github.com/tteck/Proxmox/discussions/2332#discussioncomment-8440318)
92+
93+
- ### **Phase 1 - Part #2**
94+
- *HOST commands* - I am calling this Phase 1 #2, because I wanted to show how easy Phase 1 could be (above,) and this adds a step. I wasnt lying, the above works, but I would move on and add this, its pretty easy. APT-Cacher-NG doesnt work on HTTPS. A lot of your repositories are already HTTP, so that is why the above works. But this converts more of them to HTTP. I am going to list three commands, you will see how simialar they are. Adapt this command to any other folders you want to try. Go ahead and `CD` on down in there, and see whats goign on, if you like. I personally have strung these three together with `&&` and paste one command. The third command will error if that folder doesnt exist, no harm though, and if the folder does exist, well great, it updates it.
95+
96+
```bash
97+
sudo sed -i 's#https://#http://HTTPS///#g' /etc/apt/sources.list.d/*.list
98+
```
99+
```bash
100+
sudo sed -i 's#https://#http://HTTPS///#g' /etc/apt/sources.list
101+
```
102+
```
103+
sudo sed -i 's#http://HTTPS///#https://#g' /etc/apt/sources.list.d/docker.list
104+
```
105+
106+
107+
- ### **Phase 2** Cache some repositories
108+
1. This is the one that takes up a little space. Not much really though.\
109+
Here are the two things you need (replace your IP's here)
110+
- http://192.168.1.215:3142/acng-report.html - GUI
111+
- **`/etc/apt-cacher-ng/acng.conf`** (On the SERVER)
112+
- First thing I would advise, is to paste any updates you are going to make on the top of this `acng.conf` file. I pasted mine down in the area where it is currently discussed by default and uncommented, and I did A LOT of scrolling while messing with it.
113+
- We are going to list repositories to Cache, adn then we are going to hit the GUI button to cache them.
114+
- In your `acng.conf` file paste any of the below you want to Cache. I utilized ChatGPT to put some of this together, seems legit to me though! Or, if you dont know either, copy and paste into yours.
115+
116+
```bash
117+
# Ubuntu Packages
118+
PrecacheFor: debrep/dists/ubuntu/*/binary-amd64/Packages*
119+
120+
# Ubuntu Backports
121+
PrecacheFor: debrep/dists/ubuntu-backports/*/binary-amd64/Packages*
122+
123+
# Debian Packages
124+
PrecacheFor: debrep/dists/debian/*/binary-amd64/Packages*
125+
126+
# Ubuntu Security Updates
127+
PrecacheFor: debrep/dists/ubuntu-security/*/binary-amd64/Packages*
128+
129+
# Debian Security Updates
130+
PrecacheFor: debrep/dists/debian-security/*/binary-amd64/Packages*
131+
132+
# Kernel Updates
133+
PrecacheFor: debrep/pool/main/l/linux-image-*
134+
PrecacheFor: debrep/pool/main/l/linux-headers-*
135+
136+
# Common Packages
137+
PrecacheFor: debrep/pool/main/a/apt
138+
PrecacheFor: debrep/pool/main/s/systemd
139+
PrecacheFor: debrep/pool/main/g/gcc
140+
PrecacheFor: debrep/pool/main/c/curl
141+
PrecacheFor: debrep/pool/main/o/openssl
142+
PrecacheFor: debrep/pool/main/s/ssh
143+
144+
# Additional Common Packages
145+
PrecacheFor: debrep/pool/main/p/python3
146+
PrecacheFor: debrep/pool/main/n/nginx
147+
PrecacheFor: debrep/pool/main/m/mysql-server
148+
PrecacheFor: debrep/pool/main/p/postgresql
149+
PrecacheFor: debrep/pool/main/r/redis
150+
151+
# Proxmox VE Packages
152+
Remap-proxmox: debrep/dists/pve/*/binary-amd64/Packages* ; http://download.proxmox.com/debian/pve
153+
PrecacheFor: debrep/dists/pve/*/binary-amd64/Packages*
154+
```
155+
2. On the GUI
156+
![alt text](image.png)
157+
158+
- The two box's that are checked, are checked by default. Click the `Start Mirroring` button and it will show you what it will cash. (Not actually cache it.)
159+
- When you are ready, check the `download package files` box, and then the `Start Mirroring` and that will actually do it!
160+
161+
- That wasnt bad, right? Done with **Phase 2**
162+
163+
- I will add this note here. At one point, I added this command to the `acng.conf.` I dont recall if it is still needed. I think I needed it when I was messing around with Phase 3. `AllowUserPorts: 80 443`
164+
165+
**Phase 3**
166+
- Full disclosure, I quit bothering with this. The trouble of trying to update, and getting an error, and then diving back into `acng.conf` file and adding something, then trying an update again, was more of a pain than I wanted to deal with. Perhaps, you have better luck. Ideally, if you resolve every error, by adding to this, then everything will be great. But just as I thought I had it figured out, I got another error, and said "To heck with it."
167+
168+
- The default is: `PassThroughPattern: .*` and that lets it all through.
169+
- I suggest `PassThroughPattern: ^(.*):443$` as that lets all HTTPs through.
170+
- but if you want to get granular and make sure nothing gets through that shouldnt. Uncomment the above and you could add somethin like this:\
171+
`PrecacheFor: debrep/dists/unstable/*/source/Sources* debrep/dists/unstable/*/binary-amd64/Packages*`\
172+
- adjust as needed
173+
- I got to this point, and gave up \
174+
`PassThroughPattern: ^(changelogs\.ubuntu\.com|download\.docker\.com|developer\.download\.nvidia\.com|apt\.grafana\.com|repos\.influxdata\.com|ppa\.launchpad\.net|p>`\
175+
Reverting back to:\
176+
`PassThroughPattern: ^(.*):443$`
177+
178+
179+
180+

_posts/image.png

18.6 KB
Loading

0 commit comments

Comments
 (0)