devbennett7/goma-gateway
1
0
Fork 0
Code Pull Requests Activity

Merge pull request #83 from jkaninda/refactor

Browse Source 
Refactor
This commit is contained in:
Jonas Kaninda
2024-11-10 20:01:50 +01:00
committed by GitHub
parent ec9c8c0e71 14657bb83b
commit 99e2ee8a0f
19 changed files with 287 additions and 181 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
README.md
View File

@@ -44,6 +44,8 @@ It's designed to be straightforward and efficient, offering features, like:
- Custom Headers
- Backend Errors interceptor
- Logging
- Metrics
- Supports Load Balancing, round-robin algorithm
- Support TLS
- Block common exploits middleware
- Patterns to detect SQL injection attempts
@@ -58,11 +60,8 @@ It's designed to be straightforward and efficient, offering features, like:
- Limit HTTP methods allowed for a particular route.
### Todo:
- [x] Support Load Balancing, round-robin algorithm
- [ ] Load Balancing Healthcheck, disable unavailable servers
- [ ] Blocklist IP address middleware
- [ ] Tracing
- [ ] Metrics
- [ ] Distributed Rate Limiting for In-Memory client IP based across multiple instances using Redis
----
@@ -121,6 +120,13 @@ services:
- [x] MacOS
- [x] Windows
Please download the binary from the [release page](https://github.com/jkaninda/goma-gateway/releases).
To run
```shell
./goma server --config config.yml
```
## Deployment
- Docker

1
cmd/server.go
View File

@@ -41,6 +41,7 @@ var ServerCmd = &cobra.Command{
if err != nil {
logger.Fatal("Could not load configuration: %v", err)
}
gs.SetEnv()
if err := gs.Start(ctx); err != nil {
logger.Fatal("Could not start server: %v", err)

8
docs/install/docker.md
View File

@@ -21,6 +21,14 @@ docker run --rm --name goma-gateway \
-v "${PWD}/config:/etc/goma/" \
jkaninda/goma-gateway config init --output /etc/goma/config.yml
```
## 2. Check configuration
```shell
docker run --rm --name goma-gateway \
-v "${PWD}/config:/etc/goma/" \
-p 8080:8080 \
jkaninda/goma-gateway config config check --config /etc/goma/config.yml
```
### 3. Start server with a custom config
```shell

3
docs/quickstart/gateway.md
View File

@@ -16,9 +16,10 @@ gateway:
readTimeout: 15
idleTimeout: 30
# Rate limiting
rateLimiter: 0
rateLimit: 0
accessLog: /dev/Stdout
errorLog: /dev/stderr
logLevel: info
disableRouteHealthCheckError: false
disableDisplayRouteOnStart: false
disableKeepAlive: false

14
docs/quickstart/healthcheck.md
View File

@@ -10,6 +10,20 @@ nav_order: 5
Goma comes with routes healthcheck, that can be enabled and disabled.
```yaml
version: 1.0
gateway:
routes:
- path: /cart
name: example route
rewrite: /
methods: []
healthCheck:
path: "/health/live"
interval: 30 # in Seconds
timeout: 10 # in Seconds
healthyStatuses: [200,404] # Healthy statuses
```
- Goma Gateway healthcheck: `/health/live`
- Routes health check: `health/routes`

25
docs/quickstart/loadbalanging.md
View File

@@ -8,5 +8,30 @@ nav_order: 4
# Load Balancing
Goma Gateway supports rund robim load blancing
```yaml
version: 1.0
gateway:
routes:
- path: /
name: example route
hosts:
- example.com
- example.localhost
rewrite: /
methods: []
healthCheck:
path: "/"
interval: 0
timeout: 0
healthyStatuses: [200,404]
## destination: will be override by backends
destination: ""
backends:
- https://example.com
- https://example2.com
- https://example4.com
cors:
```

30
docs/quickstart/logging.md Normal file
View File

@@ -0,0 +1,30 @@
---
title: Logging
layout: default
parent: Quickstart
nav_order: 6
---
# Logging
### Set the log level to TRACE
Setting the log level to trace configures the server to trace-log all the headers given in forward auth requests.
This is helpful to confirm that certain required Headers are correctly forwarded from the reverse proxy.
### When using the environment variable
Set the Goma log level to TRACE:
Add the following block to your .env file:
```shell
GOMA_LOG_LEVEL=trace
```
### When using a configuration file
Edit the Goma settings and set `logLevel: trace`.

21
docs/quickstart/monitoring.md Normal file
View File

@@ -0,0 +1,21 @@
---
title: Monitoring
layout: default
parent: Quickstart
nav_order: 6
---
# Monitoring
Goma collects and exports metrics
To enable metrics, you need to set `enableMetrics` to `true` and the metrics are available at `/metrics`
```yaml
version: 1.0
gateway:
enableMetrics: true
...
```

250
docs/quickstart/route.md
View File

@@ -10,164 +10,142 @@ nav_order: 2
The Route allows you to match on HTTP traffic and direct it to the backend.
### Example of a route
### Simple route
```yaml
version: 1.0
gateway:
...
routes:
- name: Example
path: /store/cart
rewrite: /cart
destination: http://cart-service:8080
methods: [POST, PUT, GET]
healthCheck: ''
cors: {}
```
### Route with limited HTTP methods
The proxy will allow all HTTP methods if there's no defined method.
Example of route with limited HTTP methods allowed for a particular route.
```yaml
version: 1.0
gateway:
...
routes:
- name: Example
path: /store/cart
rewrite: /cart
destination: http://cart-service:8080
methods: [PATCH, GET]
cors: {}
middlewares:
- api-forbidden-paths
- jwt-auth
```
### Full example of routes and middlewares
### Route with healthcheck
Example of route with backend health check.
```yaml
# Goma Gateway configurations
version: 1.0
gateway:
# Proxy write timeout
writeTimeout: 15
# Proxy read timeout
readTimeout: 15
# Proxy idle timeout
idleTimeout: 60
## SSL Certificate file
sslCertFile: '' #cert.pem
## SSL Private Key file
sslKeyFile: ''#key.pem
# Proxy rate limit, it's In-Memory IP based
rateLimit: 0
accessLog: "/dev/Stdout"
errorLog: "/dev/stderr"
## Enable, disable routes health check
disableHealthCheckStatus: false
## Returns backend route healthcheck errors
disableRouteHealthCheckError: false
# Disable display routes on start
disableDisplayRouteOnStart: false
# disableKeepAlive allows enabling and disabling KeepALive server
disableKeepAlive: false
# interceptErrors intercepts backend errors based on defined the status codes
interceptErrors:
- 405
- 500
# - 400
# Proxy Global HTTP Cors
cors:
# Global routes cors for all routes
origins:
- http://localhost:8080
- https://example.com
# Global routes cors headers for all routes
headers:
Access-Control-Allow-Headers: 'Origin, Authorization, Accept, Content-Type, Access-Control-Allow-Headers, X-Client-Id, X-Session-Id'
Access-Control-Allow-Credentials: 'true'
Access-Control-Max-Age: 1728000
##### Define routes
...
routes:
# Example of a route | 1
- name: Public
# host Domain/host based request routing
host: "" # Host is optional
path: /public
## Rewrite a request path
# e.g rewrite: /store to /
rewrite: /
destination: https://example.com
methods: [GET]
#DisableHeaderXForward Disable X-forwarded header.
# [X-Forwarded-Host, X-Forwarded-For, Host, Scheme ]
# It will not match the backend route, by default, it's disabled
disableHeaderXForward: false
# Internal health check
healthCheck: '' #/internal/health/ready
# Route Cors, global cors will be overridden by route
cors:
# Route Origins Cors, route will override global cors origins
origins:
- https://dev.example.com
- http://localhost:3000
- https://example.com
# Route Cors headers, route will override global cors headers
headers:
Access-Control-Allow-Methods: 'GET'
Access-Control-Allow-Headers: 'Origin, Authorization, Accept, Content-Type, Access-Control-Allow-Headers, X-Client-Id, X-Session-Id'
Access-Control-Allow-Credentials: 'true'
Access-Control-Max-Age: 1728000
##### Apply middlewares to the route
## The name must be unique
## List of middleware name
middlewares:
- api-forbidden-paths
# Example of a route | 2
- name: Basic auth
path: /protected
rewrite: /
destination: https://example.com
- name: Example
path: /store/cart
rewrite: /cart
destination: http://cart-service:8080
methods: [PATCH, GET]
healthCheck:
path: "/health/live"
interval: 0
timeout: 0
healthyStatuses: [200,404]
cors: {}
```
### Route with middleware
Example of route with backend health check.
```yaml
version: 1.0
gateway:
...
routes:
- name: Example
path: /store/cart
rewrite: /cart
destination: http://cart-service:8080
methods: []
healthCheck:
path: "/health/live"
interval: 0
timeout: 0
healthyStatuses: [200,404]
cors: {}
## Middleware
middlewares:
- api-forbidden-paths
- basic-auth
- jwt-auth
```
### Route with backend errors interceptor
#Defines proxy middlewares
# middleware name must be unique
middlewares:
# Enable Basic auth authorization based
- name: basic-auth
# Authentication types | jwt, basic, OAuth
type: basic
paths:
- /user
- /admin
- /account
rule:
username: admin
password: admin
#Enables JWT authorization based on the result of a request and continues the request.
- name: google-auth
# Authentication types | jwt, basic, OAuth
# jwt authorization based on the result of backend's response and continue the request when the client is authorized
type: jwt
# Paths to protect
paths:
- /protected-access
- /example-of-jwt
#- /* or wildcard path
rule:
# This is an example URL
url: https://www.googleapis.com/auth/userinfo.email
# Required headers, if not present in the request, the proxy will return 403
requiredHeaders:
- Authorization
# You can also get headers from the authentication request result and inject them into the next request header or params.
# In case you want to get headers from the authentication service and inject them into the next request headers.
# Set the request variable to the given value after the authorization request completes.
# In case you want to get headers from the authentication service and inject them into the next request headers.
# Key is authentication request response header Key. Value is the next Request header Key.
headers:
userId: Auth-UserId
userCountryId: Auth-UserCountryId
# In case you want to get headers from the Authentication service and inject them to the next request params.
#Key is authentication request response header Key. Value is the next Request parameter Key.
params:
userCountryId: countryId
# The server will return 403
- name: api-forbidden-paths
type: access
## prevents access paths
paths:
- /swagger-ui/*
- /v2/swagger-ui/*
- /api-docs/*
- /internal/*
- /actuator/*
Example of route with backend errors interceptor.
```yaml
version: 1.0
gateway:
...
routes:
- name: Example
path: /store/cart
rewrite: /cart
destination: http://cart-service:8080
methods: []
healthCheck:
path: "/health/live"
interval: 0
timeout: 0
healthyStatuses: [200,404]
interceptErrors: [403,500]
blockCommonExploits: false
cors: {}
## Middleware
middlewares:
- api-forbidden-paths
- jwt-auth
```
### Route with enabled load balancing
Example of route with load balancing enabled.
```yaml
version: 1.0
gateway:
...
routes:
- name: Example
path: /store/cart
rewrite: /cart
## destination: will be override by backends
destination: ""
backends:
- https://example.com
- https://example2.com
- https://example4.com
methods: []
healthCheck:
path: "/health/live"
interval: 0
timeout: 0
healthyStatuses: [200,404]
interceptErrors: [403,500]
blockCommonExploits: false
cors: {}
## Middleware
middlewares:
- api-forbidden-paths
- jwt-auth
```

2
docs/quickstart/ssl.md
View File

@@ -2,7 +2,7 @@
title: SSL Certificate
layout: default
parent: Quickstart
nav_order: 4
nav_order: 5
---

16
internal/config.go
View File

@@ -102,9 +102,19 @@ func (GatewayServer) Config(configFile string) (*GatewayServer, error) {
middlewares: c.Middlewares,
}, nil
}
func GetConfigPaths() string {
return util.GetStringEnv("GOMAY_CONFIG_FILE", ConfigFile)
// SetEnv sets environment variables
func (gatewayServer GatewayServer) SetEnv() {
util.SetEnv("GOMA_LOG_LEVEL", gatewayServer.gateway.LogLevel)
util.SetEnv("GOMA_ERROR_LOG", gatewayServer.gateway.ErrorLog)
util.SetEnv("GOMA_ACCESS_LOG", gatewayServer.gateway.AccessLog)
}
func GetConfigPaths() string {
return util.GetStringEnv("GOMA_CONFIG_FILE", ConfigFile)
}
// InitConfig initializes configs
func InitConfig(cmd *cobra.Command) {
configFile, _ := cmd.Flags().GetString("output")
if configFile == "" {
@@ -114,6 +124,8 @@ func InitConfig(cmd *cobra.Command) {
return
}
// initConfig initializes configs
func initConfig(configFile string) {
if configFile == "" {
configFile = GetConfigPaths()

6
internal/handler.go
View File

@@ -65,7 +65,7 @@ func ProxyErrorHandler(w http.ResponseWriter, r *http.Request, err error) {
// HealthCheckHandler handles health check of routes
func (heathRoute HealthCheckRoute) HealthCheckHandler(w http.ResponseWriter, r *http.Request) {
logger.Info("%s %s %s %s", r.Method, r.RemoteAddr, r.URL, r.UserAgent())
logger.Debug("%s %s %s %s", r.Method, r.RemoteAddr, r.URL, r.UserAgent())
wg := sync.WaitGroup{}
wg.Add(len(heathRoute.Routes))
var routes []HealthCheckRouteResponse
@@ -80,11 +80,11 @@ func (heathRoute HealthCheckRoute) HealthCheckHandler(w http.ResponseWriter, r *
}
routes = append(routes, HealthCheckRouteResponse{Name: route.Name, Status: "unhealthy", Error: "Error: " + err.Error()})
} else {
logger.Info("Route %s is healthy", route.Name)
logger.Debug("Route %s is healthy", route.Name)
routes = append(routes, HealthCheckRouteResponse{Name: route.Name, Status: "healthy", Error: ""})
}
} else {
logger.Warn("Route %s's healthCheck is undefined", route.Name)
logger.Debug("Route %s's healthCheck is undefined", route.Name)
routes = append(routes, HealthCheckRouteResponse{Name: route.Name, Status: "undefined", Error: ""})
}
}()

2
internal/middleware/error-interceptor.go
View File

@@ -46,7 +46,7 @@ func (intercept InterceptErrors) ErrorInterceptor(next http.Handler) http.Handle
rec := newResponseRecorder(w)
next.ServeHTTP(rec, r)
if canIntercept(rec.statusCode, intercept.Errors) {
logger.Error("Backend error")
logger.Debug("Backend error")
logger.Error("An error occurred from the backend with the status code: %d", rec.statusCode)
w.Header().Set("Content-Type", "application/json")
//Update Origin Cors Headers

10
internal/middleware/middleware.go
View File

@@ -82,6 +82,7 @@ func (jwtAuth JwtAuth) AuthMiddleware(next http.Handler) http.Handler {
}
return
}
logger.Trace("JWT Auth response headers: %v", authReq.Header)
// Copy headers from the original request to the new request
for name, values := range r.Header {
for _, value := range values {
@@ -96,8 +97,8 @@ func (jwtAuth JwtAuth) AuthMiddleware(next http.Handler) http.Handler {
client := &http.Client{}
authResp, err := client.Do(authReq)
if err != nil || authResp.StatusCode != http.StatusOK {
logger.Info("%s %s %s %s", r.Method, getRealIP(r), r.URL, r.UserAgent())
logger.Warn("Proxy authentication error")
logger.Debug("%s %s %s %s", r.Method, getRealIP(r), r.URL, r.UserAgent())
logger.Debug("Proxy authentication error")
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusUnauthorized)
err = json.NewEncoder(w).Encode(ProxyResponseError{
@@ -139,10 +140,11 @@ func (jwtAuth JwtAuth) AuthMiddleware(next http.Handler) http.Handler {
// AuthMiddleware checks for the Authorization header and verifies the credentials
func (basicAuth AuthBasic) AuthMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
logger.Trace("Basic-Auth request headers: %v", r.Header)
// Get the Authorization header
authHeader := r.Header.Get("Authorization")
if authHeader == "" {
logger.Error("Proxy error, missing Authorization header")
logger.Debug("Proxy error, missing Authorization header")
w.Header().Set("WWW-Authenticate", `Basic realm="Restricted"`)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusUnauthorized)
@@ -175,7 +177,7 @@ func (basicAuth AuthBasic) AuthMiddleware(next http.Handler) http.Handler {
// Decode the base64 encoded username:password string
payload, err := base64.StdEncoding.DecodeString(authHeader[len("Basic "):])
if err != nil {
logger.Error("Proxy error, missing Basic Authorization header")
logger.Debug("Proxy error, missing Basic Authorization header")
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusUnauthorized)
err := json.NewEncoder(w).Encode(ProxyResponseError{

2
internal/middleware/rate-limit.go
View File

@@ -66,7 +66,7 @@ func (rl *RateLimiter) RateLimitMiddleware() mux.MiddlewareFunc {
rl.mu.Unlock()
if client.RequestCount > rl.Requests {
logger.Error("Too many requests from IP: %s %s %s", clientID, r.URL, r.UserAgent())
logger.Debug("Too many requests from IP: %s %s %s", clientID, r.URL, r.UserAgent())
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusTooManyRequests)
//Update Origin Cors Headers

2
internal/proxy.go
View File

@@ -30,6 +30,8 @@ import (
func (proxyRoute ProxyRoute) ProxyHandler() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
logger.Info("%s %s %s %s", r.Method, getRealIP(r), r.URL.Path, r.UserAgent())
logger.Trace("Request params: %s", r.URL.RawQuery)
logger.Trace("Request Headers: %s", r.Header)
// Check Method if is allowed
if len(proxyRoute.methods) > 0 {
if !slices.Contains(proxyRoute.methods, r.Method) {

1
internal/types.go
View File

@@ -188,6 +188,7 @@ type Gateway struct {
BlockCommonExploits bool `yaml:"blockCommonExploits"`
AccessLog string `yaml:"accessLog" env:"GOMA_ACCESS_LOG, overwrite"`
ErrorLog string `yaml:"errorLog" env:"GOMA_ERROR_LOG=, overwrite"`
LogLevel string `yaml:"logLevel" env:"GOMA_LOG_LEVEL, overwrite"`
// DisableHealthCheckStatus enable and disable routes health check
DisableHealthCheckStatus bool `yaml:"disableHealthCheckStatus"`
// DisableRouteHealthCheckError allows enabling and disabling backend healthcheck errors

55
pkg/logger/logger.go
View File

@@ -33,23 +33,15 @@ type Logger struct {
// Info returns info log
func Info(msg string, args ...interface{}) {
log.SetOutput(getStd(util.GetStringEnv("GOMA_ACCESS_LOG", "/dev/stdout")))
formattedMessage := fmt.Sprintf(msg, args...)
if len(args) == 0 {
log.Printf("INFO: %s\n", msg)
} else {
log.Printf("INFO: %s\n", formattedMessage)
}
logWithCaller("INFO", msg, args...)
}
// Warn returns warning log
func Warn(msg string, args ...interface{}) {
log.SetOutput(getStd(util.GetStringEnv("GOMA_ACCESS_LOG", "/dev/stdout")))
formattedMessage := fmt.Sprintf(msg, args...)
if len(args) == 0 {
log.Printf("WARN: %s\n", msg)
} else {
log.Printf("WARN: %s\n", formattedMessage)
}
logWithCaller("WARN", msg, args...)
}
// Error logs error messages
@@ -60,36 +52,47 @@ func Error(msg string, args ...interface{}) {
func Fatal(msg string, args ...interface{}) {
log.SetOutput(os.Stdout)
formattedMessage := fmt.Sprintf(msg, args...)
if len(args) == 0 {
log.Printf("ERROR: %s\n", msg)
} else {
log.Printf("ERROR: %s\n", formattedMessage)
}
logWithCaller("ERROR", msg, args...)
os.Exit(1)
}
func Debug(msg string, args ...interface{}) {
log.SetOutput(getStd(util.GetStringEnv("GOMA_ACCESS_LOG", "/dev/stdout")))
logWithCaller("DEBUG", msg, args...)
logLevel := util.GetStringEnv("GOMA_LOG_LEVEL", "")
if logLevel == "trace" || logLevel == "debug" {
logWithCaller("DEBUG", msg, args...)
}
}
func Trace(msg string, args ...interface{}) {
log.SetOutput(getStd(util.GetStringEnv("GOMA_ACCESS_LOG", "/dev/stdout")))
logLevel := util.GetStringEnv("GOMA_LOG_LEVEL", "")
if logLevel == "trace" {
logWithCaller("DEBUG", msg, args...)
}
}
// Helper function to format and log messages with file and line number
func logWithCaller(level, msg string, args ...interface{}) {
formattedMessage := fmt.Sprintf(msg, args...)
_, file, line, ok := runtime.Caller(2) // Get the caller's file and line number (skip 2 frames)
// Format message if there are additional arguments
formattedMessage := msg
if len(args) > 0 {
formattedMessage = fmt.Sprintf(msg, args...)
}
// Get the caller's file and line number (skip 2 frames)
_, file, line, ok := runtime.Caller(2)
if !ok {
file = "unknown"
line = 0
}
if len(args) == 0 {
log.Printf("%s: %s (File: %s, Line: %d)\n", level, msg, file, line)
} else {
// Log message with caller information if GOMA_LOG_LEVEL is trace
logLevel := util.GetStringEnv("GOMA_LOG_LEVEL", "")
if logLevel == "trace" {
log.Printf("%s: %s (File: %s, Line: %d)\n", level, formattedMessage, file, line)
} else {
log.Printf("%s: %s\n", level, formattedMessage)
}
}

8
util/helpers.go
View File

@@ -73,9 +73,11 @@ func GetBoolEnv(key string, defaultValue bool) bool {
// SetEnv Set env
func SetEnv(name, value string) {
err := os.Setenv(name, value)
if err != nil {
return
if len(value) != 0 {
err := os.Setenv(name, value)
if err != nil {
return
}
}
}