Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync with origin #1

Open
wants to merge 70 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
8ca3660
Add blog URL (#261)
n0vad3v Jul 24, 2023
9250d6f
Optimize README.md (#262)
n0vad3v Jul 26, 2023
5dba6bb
M1 should be arm64 (#263)
n0vad3v Jul 29, 2023
4003b03
Multiple backends support (#207)
bugfest Aug 2, 2023
6790ddd
Add support for Go 1.21, bump version, optimize code (#264)
n0vad3v Aug 9, 2023
ab43732
apt update before apt install (#265)
n0vad3v Aug 9, 2023
d9da7f3
Remove metadata when original not exist, update support for heic (#266)
n0vad3v Aug 17, 2023
523294e
fix image exist nil (#274)
BennyThink Sep 15, 2023
5d265dc
Parse flat before load config (#275)
n0vad3v Sep 15, 2023
c54b8be
Use compiled libwebp (#276)
n0vad3v Sep 15, 2023
51dde5e
Remove ARMv7 (#277)
n0vad3v Sep 15, 2023
0b0e5b1
Upgrade github.com/gofiber/fiber (#278)
n0vad3v Sep 18, 2023
655fb1d
Add env on startup (#283)
n0vad3v Oct 17, 2023
d98740c
ISSUE-281 Add NEF support by convert to jpg (#284)
llanc Oct 30, 2023
4a90b18
Fix bug that could remove original image (#287)
n0vad3v Nov 4, 2023
543af24
Better configuration (#288)
n0vad3v Nov 5, 2023
4f39a1c
Unify ENV (#289)
n0vad3v Nov 5, 2023
ace5642
Bump golang.org/x/image from 0.5.0 to 0.10.0 (#290)
dependabot[bot] Nov 11, 2023
cd0b87c
Bump github.com/gofiber/fiber/v2 from 2.50.0 to 2.51.0 (#294)
dependabot[bot] Nov 22, 2023
d7d4c37
Bump github.com/valyala/fasthttp from 1.50.0 to 1.51.0 (#293)
dependabot[bot] Nov 22, 2023
290b233
fix: incorrect cache key (#286)
HolgerHuo Nov 22, 2023
84bd4c6
Bump github.com/schollz/progressbar/v3 from 3.13.1 to 3.14.1 (#291)
dependabot[bot] Nov 22, 2023
cc26be0
Add healthz endpoint (#296)
n0vad3v Nov 23, 2023
0d4856c
fix: metadata not updated when remote file changes (#297)
HolgerHuo Nov 23, 2023
d67601a
Fix link in README.md and fix CI permission (#299)
n0vad3v Nov 24, 2023
c771160
Ignore GIF when converting to AVIF (#302)
n0vad3v Dec 1, 2023
d83f666
Refine convert part (#303)
n0vad3v Dec 16, 2023
484c876
Bump to 0.10.4 (#304)
n0vad3v Dec 18, 2023
d35e0e0
Lock on convert (#307)
n0vad3v Dec 29, 2023
1ec981b
Read all pages (#308)
n0vad3v Dec 29, 2023
0dc7a0f
Bump github.com/gofiber/fiber/v2 from 2.51.0 to 2.52.0 (#309)
dependabot[bot] Feb 10, 2024
12d4cc1
Upgrade to Go 1.22 (#311)
n0vad3v Feb 10, 2024
123c96d
add: cache for remote images (#310)
HolgerHuo Feb 11, 2024
9be1a6b
Fix tests, bump version (#312)
n0vad3v Feb 11, 2024
c921561
Bump github.com/valyala/fasthttp from 1.51.0 to 1.52.0 (#314)
dependabot[bot] Feb 20, 2024
eb2b6d1
Bump govips to 2.14 (#317)
n0vad3v Feb 26, 2024
55c04d2
Bump github.com/gofiber/fiber/v2 from 2.52.0 to 2.52.1 (#315)
dependabot[bot] Feb 27, 2024
61880e2
Bump github.com/schollz/progressbar/v3 from 3.14.1 to 3.14.2 (#316)
dependabot[bot] Feb 29, 2024
ca1050d
Optimized Dockerfile, bump version (#318)
n0vad3v Feb 29, 2024
5a9b4ed
Bump github.com/gofiber/fiber/v2 from 2.52.1 to 2.52.2 (#320)
dependabot[bot] Mar 4, 2024
ddcb532
Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#319)
dependabot[bot] Mar 4, 2024
43c275e
Adds JPEG XL support, max_height/max_width support (#321)
n0vad3v Mar 22, 2024
ac30605
Bump github.com/gofiber/fiber/v2 from 2.52.2 to 2.52.3 (#323)
dependabot[bot] Mar 26, 2024
8c15bf3
Bump github.com/gofiber/fiber/v2 from 2.52.3 to 2.52.4 (#324)
dependabot[bot] Apr 10, 2024
b5842b2
Allow jxl based on Accept header (#326)
n0vad3v Apr 10, 2024
c7bebfc
Add EXTRA_PARAMS_CROP_INTERESTING (#328)
n0vad3v Apr 19, 2024
be32328
Fix Malformed dir traversal (#331)
BennyThink Apr 23, 2024
e69c3a2
Bump golang.org/x/net from 0.21.0 to 0.23.0 (#329)
dependabot[bot] Apr 23, 2024
74c6aa6
--- (#334)
dependabot[bot] May 21, 2024
d9c542d
Bump github.com/davidbyttow/govips/v2 to 2.15.0 (#337)
syedcherry Jun 7, 2024
5b3e2f3
Bump golang.org/x/image from 0.17.0 to 0.18.0 (#340)
dependabot[bot] Jun 27, 2024
53487b5
Bump github.com/valyala/fasthttp from 1.52.0 to 1.55.0 (#339)
dependabot[bot] Jun 27, 2024
81c6e78
Bump github.com/schollz/progressbar/v3 from 3.14.3 to 3.14.4 (#338)
dependabot[bot] Jun 27, 2024
f207e3c
Update README (#343)
n0vad3v Jun 28, 2024
89ea0af
Add `MAX_CACHE_SIZE` config (#344)
n0vad3v Jun 29, 2024
d29959c
Bump to 0.12.0 (#345)
n0vad3v Jun 29, 2024
a0ee22a
Bump github.com/gofiber/fiber/v2 from 2.52.4 to 2.52.5 (#346)
dependabot[bot] Jul 1, 2024
80d6a04
Bump github.com/schollz/progressbar/v3 from 3.14.4 to 3.14.5 (#348)
dependabot[bot] Aug 4, 2024
d400f52
Bump github.com/schollz/progressbar/v3 from 3.14.5 to 3.14.6 (#350)
dependabot[bot] Aug 5, 2024
89aea97
Bump to Go 1.23 (#353)
n0vad3v Aug 16, 2024
ae0274c
Add iOS 18 support (#360)
n0vad3v Oct 30, 2024
a08b6e6
Use newer version of libaom (#363)
n0vad3v Nov 13, 2024
e696ee9
Add prefetch-foreground (#365)
n0vad3v Nov 25, 2024
8d24d70
Bump github.com/stretchr/testify from 1.9.0 to 1.10.0 (#366)
dependabot[bot] Nov 26, 2024
ec27023
Bump github.com/schollz/progressbar/v3 from 3.14.6 to 3.17.1 (#362)
dependabot[bot] Nov 26, 2024
3f732e8
WebP Server Go 0.13.0 (#367)
n0vad3v Jan 1, 2025
95b112e
Bump github.com/gofiber/fiber/v2 from 2.52.5 to 2.52.6 (#372)
dependabot[bot] Jan 10, 2025
686a2bf
Bump github.com/schollz/progressbar/v3 from 3.17.1 to 3.18.0 (#373)
dependabot[bot] Jan 11, 2025
a654b4f
Bump github.com/valyala/fasthttp from 1.55.0 to 1.58.0 (#368)
dependabot[bot] Jan 11, 2025
67f6467
Customize loglevel with `--verbosity` flag (#375)
n0vad3v Jan 15, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Unify ENV (webp-sh#289)
n0vad3v authored Nov 5, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 4f39a1c0b496f4739cc39ce3e6375500efc6ad3f
5 changes: 4 additions & 1 deletion config.json
Original file line number Diff line number Diff line change
@@ -7,5 +7,8 @@
"ALLOWED_TYPES": ["jpg","png","jpeg","bmp","gif","svg","heic"],
"IMG_MAP": {},
"ENABLE_AVIF": false,
"ENABLE_EXTRA_PARAMS": false
"ENABLE_EXTRA_PARAMS": false,
"READ_BUFFER_SIZE": 4096,
"CONCURRENCY": 262144,
"DISABLE_KEEPALIVE": false
}
55 changes: 36 additions & 19 deletions config/config.go
Original file line number Diff line number Diff line change
@@ -28,27 +28,13 @@ const (
"IMG_PATH": "./pics",
"EXHAUST_PATH": "./exhaust",
"IMG_MAP": {},
"ALLOWED_TYPES": ["jpg","png","jpeg","bmp","svg","nef"],
"ALLOWED_TYPES": ["jpg","png","jpeg","bmp","svg","heic","nef"],
"ENABLE_AVIF": false,
"ENABLE_EXTRA_PARAMS": false
"READ_BUFFER_SIZE": 4096,
"CONCURRENCY": 262144,
"DISABLE_KEEPALIVE": false
}`

SampleSystemd = `
[Unit]
Description=WebP Server Go
Documentation=https://github.com/webp-sh/webp_server_go
After=nginx.target

[Service]
Type=simple
StandardError=journal
WorkingDirectory=/opt/webps
ExecStart=/opt/webps/webp-server --config /opt/webps/config.json
Restart=always
RestartSec=3s

[Install]
WantedBy=multi-user.target`
)

var (
@@ -83,6 +69,9 @@ type WebpConfig struct {
ExhaustPath string `json:"EXHAUST_PATH"`
EnableAVIF bool `json:"ENABLE_AVIF"`
EnableExtraParams bool `json:"ENABLE_EXTRA_PARAMS"`
ReadBufferSize int `json:"READ_BUFFER_SIZE"`
Concurrency int `json:"CONCURRENCY"`
DisableKeepalive bool `json:"DISABLE_KEEPALIVE"`
}

func NewWebPConfig() *WebpConfig {
@@ -96,6 +85,9 @@ func NewWebPConfig() *WebpConfig {
ExhaustPath: "./exhaust",
EnableAVIF: false,
EnableExtraParams: false,
ReadBufferSize: 4096,
Concurrency: 262144,
DisableKeepalive: false,
}
}

@@ -104,7 +96,6 @@ func init() {
flag.BoolVar(&Prefetch, "prefetch", false, "Prefetch and convert image to WebP format.")
flag.IntVar(&Jobs, "jobs", runtime.NumCPU(), "Prefetch thread, default is all.")
flag.BoolVar(&DumpConfig, "dump-config", false, "Print sample config.json.")
flag.BoolVar(&DumpSystemd, "dump-systemd", false, "Print sample systemd service file.")
flag.BoolVar(&ShowVersion, "V", false, "Show version information.")
}

@@ -166,6 +157,32 @@ func LoadConfig() {
if os.Getenv("WEBP_IMG_MAP") != "" {
// TODO
}
if os.Getenv("WEBP_READ_BUFFER_SIZE") != "" {
readBufferSize, err := strconv.Atoi(os.Getenv("WEBP_READ_BUFFER_SIZE"))
if err != nil {
log.Warnf("WEBP_READ_BUFFER_SIZE is not a valid integer, using value in config.json %d", Config.ReadBufferSize)
} else {
Config.ReadBufferSize = readBufferSize
}
}
if os.Getenv("WEBP_CONCURRENCY") != "" {
concurrency, err := strconv.Atoi(os.Getenv("WEBP_CONCURRENCY"))
if err != nil {
log.Warnf("WEBP_CONCURRENCY is not a valid integer, using value in config.json %d", Config.Concurrency)
} else {
Config.Concurrency = concurrency
}
}
if os.Getenv("WEBP_DISABLE_KEEPALIVE") != "" {
disableKeepalive := os.Getenv("WEBP_DISABLE_KEEPALIVE")
if disableKeepalive == "true" {
Config.DisableKeepalive = true
} else if disableKeepalive == "false" {
Config.DisableKeepalive = false
} else {
log.Warnf("WEBP_DISABLE_KEEPALIVE is not a valid boolean, using value in config.json %t", Config.DisableKeepalive)
}
}

log.Debugln("Config init complete")
log.Debugln("Config", Config)
8 changes: 0 additions & 8 deletions helper/helper.go
Original file line number Diff line number Diff line change
@@ -191,11 +191,3 @@ func HashFile(filepath string) string {
buf, _ := os.ReadFile(filepath)
return fmt.Sprintf("%x", xxhash.Sum64(buf))
}

func GetEnv(key string, defaultVal ...string) string {
value := os.Getenv(key)
if value == "" && len(defaultVal) > 0 {
return defaultVal[0]
}
return value
}
21 changes: 3 additions & 18 deletions webp-server.go
Original file line number Diff line number Diff line change
@@ -5,11 +5,9 @@ import (
"fmt"
"os"
"runtime"
"strconv"
"webp_server_go/config"
"webp_server_go/encoder"
"webp_server_go/handler"
"webp_server_go/helper"

"github.com/gofiber/fiber/v2"
"github.com/gofiber/fiber/v2/middleware/etag"
@@ -18,24 +16,15 @@ import (
log "github.com/sirupsen/logrus"
)

var (
ReadBufferSizeStr = helper.GetEnv("READ_BUFFER_SIZE", "4096") // Default: 4096
ReadBufferSize, _ = strconv.Atoi(ReadBufferSizeStr)
ConcurrencyStr = helper.GetEnv("CONCURRENCY", "262144") // Default: 256 * 1024
Concurrency, _ = strconv.Atoi(ConcurrencyStr)
DisableKeepaliveStr = helper.GetEnv("DISABLE_KEEPALIVE", "false") // Default: false
DisableKeepalive, _ = strconv.ParseBool(DisableKeepaliveStr)
)

// https://docs.gofiber.io/api/fiber
var app = fiber.New(fiber.Config{
ServerHeader: "WebP Server Go",
AppName: "WebP Server Go",
DisableStartupMessage: true,
ProxyHeader: "X-Real-IP",
ReadBufferSize: ReadBufferSize, // per-connection buffer size for requests' reading. This also limits the maximum header size. Increase this buffer if your clients send multi-KB RequestURIs and/or multi-KB headers (for example, BIG cookies).
Concurrency: Concurrency, // Maximum number of concurrent connections.
DisableKeepalive: DisableKeepalive, // Disable keep-alive connections, the server will close incoming connections after sending the first response to the client
ReadBufferSize: config.Config.ReadBufferSize, // per-connection buffer size for requests' reading. This also limits the maximum header size. Increase this buffer if your clients send multi-KB RequestURIs and/or multi-KB headers (for example, BIG cookies).
Concurrency: config.Config.Concurrency, // Maximum number of concurrent connections.
DisableKeepalive: config.Config.DisableKeepalive, // Disable keep-alive connections, the server will close incoming connections after sending the first response to the client
})

func setupLogger() {
@@ -78,10 +67,6 @@ func init() {
fmt.Println(config.SampleConfig)
os.Exit(0)
}
if config.DumpSystemd {
fmt.Println(config.SampleSystemd)
os.Exit(0)
}
if config.ShowVersion {
fmt.Printf("\n %c[1;32m%s%c[0m\n\n", 0x1B, banner+"", 0x1B)
os.Exit(0)