Skip to content

Commit 6ae9609

Browse files
Gaardsholtbrondum
andauthored
Multiple datastores (#3)
* in-memory works - still working on redis Signed-off-by: Gaardsholt <lasse.gaardsholt@bestseller.com> * redis seems to work now Signed-off-by: Gaardsholt <lasse.gaardsholt@bestseller.com> * first mock of redis and config test (#1) * changed so that the `SecretStore` will only handle storing the data and not encrypt/decrypt the data Signed-off-by: Gaardsholt <lasse.gaardsholt@bestseller.com> * go mod updates Signed-off-by: Gaardsholt <lasse.gaardsholt@bestseller.com> * Create CODEOWNERS * I have most likely forgotten something Signed-off-by: Gaardsholt <lasse.gaardsholt@bestseller.com> * Multiple datastores (#2) * first mock of redis and config test * added random to hash id to prevent duplicate IDs, even though it might only be theorectical possible * a bit of linting and a bit of error handling * making sure metrics dont block the functionality * added another redis test * added better error handling to crypto package * added test to crypto package Co-authored-by: Lasse Gaardsholt <lasse.gaardsholt@gmail.com> * fixing tests Signed-off-by: Gaardsholt <lasse.gaardsholt@bestseller.com> * fixing codeowners * updated readme Signed-off-by: Gaardsholt <lasse.gaardsholt@bestseller.com> * Never gonna give, never gonna give Signed-off-by: Gaardsholt <lasse.gaardsholt@bestseller.com> * hoo boy Signed-off-by: Gaardsholt <lasse.gaardsholt@bestseller.com> Co-authored-by: Peter Brøndum <34370407+brondum@users.noreply.github.com>
1 parent a1ef949 commit 6ae9609

File tree

24 files changed

+898
-287
lines changed

24 files changed

+898
-287
lines changed

.github/CODEOWNERS

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
* @Gaardsholt

.gitignore

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
2+
# Created by https://www.toptal.com/developers/gitignore/api/visualstudiocode,go
3+
# Edit at https://www.toptal.com/developers/gitignore?templates=visualstudiocode,go
4+
5+
### Go ###
6+
# Binaries for programs and plugins
7+
*.exe
8+
*.exe~
9+
*.dll
10+
*.so
11+
*.dylib
12+
pass-along
13+
14+
# Test binary, built with `go test -c`
15+
*.test
16+
17+
# Output of the go coverage tool, specifically when used with LiteIDE
18+
*.out
19+
20+
# Dependency directories (remove the comment below to include it)
21+
# vendor/
22+
23+
### Go Patch ###
24+
/vendor/
25+
/Godeps/
26+
27+
### VisualStudioCode ###
28+
.vscode/*
29+
!.vscode/settings.json
30+
!.vscode/tasks.json
31+
!.vscode/launch.json
32+
!.vscode/extensions.json
33+
*.code-workspace
34+
35+
# Local History for Visual Studio Code
36+
.history/
37+
38+
### VisualStudioCode Patch ###
39+
# Ignore all local history of files
40+
.history
41+
.ionide
42+
43+
# End of https://www.toptal.com/developers/gitignore/api/visualstudiocode,go

.vscode/launch.json

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,10 @@
1010
"request": "launch",
1111
"mode": "auto",
1212
"program": "${workspaceFolder}/main.go",
13-
"args": []
13+
"args": [],
14+
"env": {
15+
"DATABASETYPE": "redis"
16+
}
1417
}
1518
]
1619
}

README.md

Lines changed: 27 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,34 @@
1+
> :warning: Very much work in progress !
12
# Pass-along
23

3-
> :warning: Very much work in progress !
4+
The main application uses port `8080`.
5+
6+
`/healthz` and `/metrics` endpoints uses port `8888`.
7+
8+
9+
## Server config
10+
11+
The following config can be set via environment variables
12+
| Tables | Required | Default |
13+
| ----------------------------- | :------: | --------- |
14+
| [SERVERSALT](#SERVERSALT) | | |
15+
| [DATABASETYPE](#DATABASETYPE) | | in-memory |
16+
| [REDISSERVER](#REDISSERVER) | | localhost |
17+
| [REDISPORT](#REDISPORT) | | 6379 |
18+
19+
20+
### SERVERSALT
21+
For extra security you can add your own salt when encrypting the data.
22+
23+
### DATABASETYPE
24+
Can either be `in-memory` or `redis`.
25+
26+
### REDISSERVER
27+
Address to your redis server.
428

29+
### REDISPORT
30+
Used to specify the port your redis server is using.
531

6-
## TODO:
7-
* Add some server config
8-
* Security review?
932

1033
## Create a new secret
1134

api/api.go

Lines changed: 249 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,249 @@
1+
package api
2+
3+
import (
4+
"encoding/json"
5+
"fmt"
6+
"html/template"
7+
"mime"
8+
"net/http"
9+
"sync"
10+
"time"
11+
12+
"github.com/Gaardsholt/pass-along/config"
13+
"github.com/Gaardsholt/pass-along/datastore"
14+
"github.com/Gaardsholt/pass-along/memory"
15+
"github.com/Gaardsholt/pass-along/metrics"
16+
"github.com/Gaardsholt/pass-along/redis"
17+
"github.com/Gaardsholt/pass-along/types"
18+
"github.com/gorilla/mux"
19+
"github.com/prometheus/client_golang/prometheus"
20+
"github.com/prometheus/client_golang/prometheus/promhttp"
21+
"github.com/rs/zerolog/log"
22+
)
23+
24+
const (
25+
ErrServerShuttingDown = "http: Server closed"
26+
)
27+
28+
var pr *prometheus.Registry
29+
var secretStore datastore.SecretStore
30+
var startupTime time.Time
31+
var templates map[string]*template.Template
32+
var lock = sync.RWMutex{}
33+
34+
// StartServer starts the internal and external http server and initiates the secrets store
35+
func StartServer() (internalServer *http.Server, externalServer *http.Server) {
36+
startupTime = time.Now()
37+
38+
databaseType, err := config.Config.GetDatabaseType()
39+
if err != nil {
40+
log.Fatal().Err(err).Msgf("%s", err)
41+
}
42+
43+
switch databaseType {
44+
case "in-memory":
45+
secretStore, err = memory.New(&lock)
46+
case "redis":
47+
secretStore, err = redis.New()
48+
}
49+
50+
if err != nil {
51+
log.Fatal().Err(err).Msgf("%s", err)
52+
}
53+
54+
registerPrometheusMetrics()
55+
createTemplates()
56+
57+
internal := mux.NewRouter()
58+
external := mux.NewRouter()
59+
// Start of static stuff
60+
fs := http.FileServer(http.Dir("./static"))
61+
external.PathPrefix("/assets").Handler(http.StripPrefix("/assets", fs))
62+
external.PathPrefix("/robots.txt").Handler(fs)
63+
external.PathPrefix("/favicon.ico").Handler(fs)
64+
// End of static stuff
65+
66+
external.HandleFunc("/", IndexHandler).Methods("GET")
67+
external.HandleFunc("/", NewHandler).Methods("POST")
68+
external.HandleFunc("/{id}", GetHandler).Methods("GET")
69+
70+
internal.HandleFunc("/healthz", healthz)
71+
internal.Handle("/metrics", promhttp.HandlerFor(pr, promhttp.HandlerOpts{})).Methods("GET")
72+
73+
internalPort := 8888
74+
internalServer = &http.Server{
75+
Addr: fmt.Sprintf(":%d", internalPort),
76+
Handler: internal,
77+
}
78+
79+
go func() {
80+
err := internalServer.ListenAndServe()
81+
if err != nil && err.Error() != ErrServerShuttingDown {
82+
log.Fatal().Err(err).Msgf("Unable to run the internal server at port %d", internalPort)
83+
}
84+
}()
85+
86+
externalPort := 8080
87+
externalServer = &http.Server{
88+
Addr: fmt.Sprintf(":%d", externalPort),
89+
Handler: external,
90+
}
91+
go func() {
92+
err := externalServer.ListenAndServe()
93+
if err != nil && err.Error() != ErrServerShuttingDown {
94+
log.Fatal().Err(err).Msgf("Unable to run the external server at port %d", externalPort)
95+
}
96+
}()
97+
log.Info().Msgf("Starting server at port %d with %s as datastore", externalPort, databaseType)
98+
99+
go secretStore.DeleteExpiredSecrets()
100+
101+
return
102+
}
103+
104+
func IndexHandler(w http.ResponseWriter, r *http.Request) {
105+
templates["index"].Execute(w, types.Page{Startup: startupTime})
106+
}
107+
108+
// NewHandler creates a new secret in the secretstore
109+
func NewHandler(w http.ResponseWriter, r *http.Request) {
110+
var entry types.Entry
111+
err := json.NewDecoder(r.Body).Decode(&entry)
112+
if err != nil {
113+
w.WriteHeader(http.StatusBadRequest)
114+
return
115+
}
116+
117+
log.Debug().Msg("Creating a new secret")
118+
119+
expires := time.Now().Add(
120+
time.Hour*time.Duration(0) +
121+
time.Minute*time.Duration(0) +
122+
time.Second*time.Duration(entry.ExpiresIn),
123+
)
124+
125+
mySecret := types.Secret{
126+
Content: entry.Content,
127+
Expires: expires,
128+
TimeAdded: time.Now(),
129+
UnlimitedViews: entry.UnlimitedViews,
130+
}
131+
mySecret.UnlimitedViews = entry.UnlimitedViews
132+
id := mySecret.GenerateID()
133+
134+
encryptedSecret, err := mySecret.Encrypt(id)
135+
if err != nil {
136+
go metrics.SecretsCreatedWithError.Inc()
137+
return
138+
}
139+
140+
err = secretStore.Add(id, encryptedSecret, entry.ExpiresIn)
141+
if err != nil {
142+
http.Error(w, "failed to add secret, please try again", http.StatusInternalServerError)
143+
log.Error().Err(err).Msg("Unable to add secret")
144+
return
145+
}
146+
147+
w.WriteHeader(http.StatusCreated)
148+
fmt.Fprintf(w, "%s", id)
149+
}
150+
151+
// GetHandler retrieves a secret in the secret store
152+
func GetHandler(w http.ResponseWriter, r *http.Request) {
153+
vars := mux.Vars(r)
154+
155+
useHtml := false
156+
ctHeader := r.Header.Get("Content-Type")
157+
contentType, _, err := mime.ParseMediaType(ctHeader)
158+
if err != nil || contentType != "application/json" {
159+
useHtml = true
160+
}
161+
162+
if useHtml {
163+
newError := templates["read"].Execute(w, types.Page{Startup: startupTime})
164+
if newError != nil {
165+
fmt.Fprintf(w, "%s", newError)
166+
}
167+
return
168+
}
169+
170+
id := vars["id"]
171+
secretData, gotData := secretStore.Get(id)
172+
if !gotData {
173+
w.WriteHeader(http.StatusGone)
174+
fmt.Fprint(w, "secret not found")
175+
return
176+
}
177+
178+
s, err := types.Decrypt(secretData, id)
179+
if err != nil {
180+
log.Fatal().Err(err).Msg("Unable to decrypt secret")
181+
w.WriteHeader(http.StatusInternalServerError)
182+
fmt.Fprint(w, err)
183+
return
184+
}
185+
186+
decryptedSecret := ""
187+
188+
isNotExpired := s.Expires.UTC().After(time.Now().UTC())
189+
if isNotExpired {
190+
decryptedSecret = s.Content
191+
go metrics.SecretsRead.Inc()
192+
} else {
193+
gotData = false
194+
go metrics.ExpiredSecretsRead.Inc()
195+
}
196+
197+
if !isNotExpired || !s.UnlimitedViews {
198+
secretStore.Delete(id)
199+
}
200+
201+
log.Debug().Msg("Fetching a secret")
202+
203+
w.WriteHeader(http.StatusOK)
204+
fmt.Fprintf(w, "%s", decryptedSecret)
205+
}
206+
207+
// healthz is a liveness probe.
208+
func healthz(w http.ResponseWriter, _ *http.Request) {
209+
w.WriteHeader(http.StatusOK)
210+
}
211+
212+
func registerPrometheusMetrics() {
213+
pr = prometheus.NewRegistry()
214+
// pr.MustRegister(types.NewSecretsInCache(&secretStore))
215+
pr.MustRegister(metrics.SecretsRead)
216+
pr.MustRegister(metrics.ExpiredSecretsRead)
217+
pr.MustRegister(metrics.NonExistentSecretsRead)
218+
pr.MustRegister(metrics.SecretsCreated)
219+
pr.MustRegister(metrics.SecretsCreatedWithError)
220+
pr.MustRegister(metrics.SecretsDeleted)
221+
}
222+
223+
func createTemplates() {
224+
templates = make(map[string]*template.Template)
225+
templates["index"] = template.Must(template.ParseFiles("templates/base.html", "templates/index.html"))
226+
templates["read"] = template.Must(template.ParseFiles("templates/base.html", "templates/read.html"))
227+
}
228+
229+
// func secretCleaner() {
230+
// for {
231+
// time.Sleep(5 * time.Minute)
232+
// secretStore.Lock.RLock()
233+
// for k, v := range secretStore.Data {
234+
// s, err := types.Decrypt(v, k)
235+
// if err != nil {
236+
// continue
237+
// }
238+
239+
// isNotExpired := s.Expires.UTC().After(time.Now().UTC())
240+
// if !isNotExpired {
241+
// log.Debug().Msg("Found expired secret, deleting...")
242+
// secretStore.Lock.RUnlock()
243+
// secretStore.Delete(k)
244+
// secretStore.Lock.RLock()
245+
// }
246+
// }
247+
// secretStore.Lock.RUnlock()
248+
// }
249+
// }

0 commit comments

Comments
 (0)