klevze/UploadShied
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add CONFIG_REFERENCE.md and CONTRIBUTING.md; update INTEGRATION and README

Browse Source
This commit is contained in:
Gregor Klevže
2026-02-12 10:18:32 +01:00
parent 1768f61da1
commit d6a19929bf
5 changed files with 404 additions and 528 deletions
Download Patch File Download Diff File Expand all files Collapse all files

607
INTEGRATION.md
View File

@@ -1,464 +1,89 @@
## Integration
## Integration & Tuning
Example `upload-logger.json` (commented for easy copy/paste into your environment):
This document complements the installation steps in [docs/INSTALLATION.md](docs/INSTALLATION.md) by focusing on detector tuning, allowlists, and advanced integrations (log forwarding, Fail2Ban, etc.).
Example `upload-logger.json` (simplified):
```json
// {
// "modules": {
// "flood": true,
// "filename": true,
// "mime_sniff": true,
// "hashing": true,
// "base64_detection": true,
// "raw_peek": false,
// "archive_inspect": true,
// "quarantine": true
// },
// "paths": {
// "log_file": "logs/uploads.log",
// "quarantine_dir": "quarantine",
// "state_dir": "state",
// "allowlist_file": "allowlist.json"
// },
// "limits": {
// "max_size": 52428800,
// "raw_body_min": 512000,
// "sniff_max_bytes": 8192,
// "sniff_max_filesize": 2097152,
// "hash_max_filesize": 10485760,
// "archive_max_inspect_size": 52428800,
// "archive_max_entries": 200
// },
// "ops": {
// "quarantine_owner": "root",
// "quarantine_group": "www-data",
// "quarantine_dir_perms": "0700",
// "log_rotate": {
// "enabled": true,
// "size": 10485760,
// "keep": 7
// }
// },
// "allowlists": {
// "base64_uris": [
// "/api/uploads/avatars",
// "/api/v1/avatars",
// "/user/avatar",
// "/media/upload",
// "/api/media",
// "/api/uploads",
// "/api/v1/uploads",
// "/attachments/upload",
// "/upload",
// "#^/internal/webhook#",
// "#/hooks/(github|gitlab|stripe|slack)#",
// "/services/avatars",
// "/api/profile/photo"
// ],
// "ctypes": ["image/svg+xml","application/xml","text/xml"]
// }
// }
{
"modules": {
"flood": true,
"filename": true,
"mime_sniff": true,
"hashing": true,
"base64_detection": true,
"raw_peek": false,
"archive_inspect": true,
"quarantine": true
},
"paths": {
"log_file": "logs/uploads.log",
"quarantine_dir": "quarantine",
"state_dir": "state",
"allowlist_file": "allowlist.json"
},
"limits": {
"max_size": 52428800,
"raw_body_min": 512000,
"sniff_max_bytes": 8192,
"sniff_max_filesize": 2097152
},
"ops": {
"quarantine_owner": "root",
"quarantine_group": "www-data",
"quarantine_dir_perms": "0700",
"block_suspicious": false
},
"allowlists": {
"base64_uris": [
"/api/uploads/avatars",
"#/hooks/(github|gitlab|stripe|slack)#"
],
"ctypes": ["image/svg+xml","application/xml","text/xml"]
}
}
```
Notes:
- Remove the `//` comments if copying from examples. Use absolute paths in production where possible.
- Remove the leading `// ` when copying this into a real `upload-logger.json` file.
- Adjust paths, owners, and limits to match your environment and PHP-FPM worker permissions.
### Content detector tuning
ContentDetector tuning and false-positive guidance
- The `ContentDetector` performs a fast head-scan to detect PHP open-tags and common webshell indicators (e.g., `passthru`, `system`, `exec`, `base64_decode`, `eval`, `assert`).
- Tuning options (in `upload-logger.json`):
- `limits.sniff_max_bytes` (default 8192) — how many bytes to scan from the file head.
- `limits.sniff_max_filesize` (default 2097152) — only scan files up to this size.
- `detectors.content.allow_xml_eval` — relax `eval()` detection for XML/SVG when appropriate.
- The repository includes a `ContentDetector` that performs a fast head-scan of uploaded files to detect PHP open-tags and common webshell indicators (for example `passthru()`, `system()`, `exec()`, `shell_exec()`, `proc_open()`, `popen()`, `base64_decode()`, `eval()`, `assert()`). It intentionally limits the scan to a small number of bytes to reduce CPU/IO overhead.
False positives
- `eval(` appears in benign contexts (SVG/JS). To reduce false positives:
- Add trusted URIs to `allowlists.base64_uris`.
- Add trusted content-types to `allowlists.ctypes`.
- Tune scan size limits.
- Tuning options (place these in `upload-logger.json`):
- `limits.sniff_max_bytes` (integer): number of bytes to read from the file head for scanning. Default: `8192`.
- `limits.sniff_max_filesize` (integer): only perform head-scan on files with size <= this value. Default: `2097152` (2 MB).
- `allowlists.ctypes` (array): content-types that should be considered trusted for base64/raw payloads (for example `image/svg+xml`, `application/xml`, `text/xml`) and may relax some detections.
- `allowlists.base64_uris` (array): URI patterns that should be ignored for large base64 payloads (webhooks, avatar uploads, etc.).
### Allowlists
- False positives: `eval(` and other tokens commonly appear in client-side JS inside SVG files or in benign templating contexts. If you observe false positives:
- Add trusted URIs to `allowlists.base64_uris` for endpoints that legitimately accept encoded content.
- Add trusted content-types to `allowlists.ctypes` to relax detection for XML/SVG uploads.
- Tune `limits.sniff_max_bytes` and `limits.sniff_max_filesize` to increase or decrease sensitivity.
- `allowlists.base64_uris`: URI patterns (substring or PCRE when wrapped with `#`) that should bypass base64/raw detection.
- `allowlists.ctypes`: content-types to treat as permitted for encoded payloads (e.g., `image/svg+xml`).
- Suggested (example) detector tuning block (commented):
### Archive inspection & quarantine
```json
// "detectors": {
// "content": {
// "enabled": true,
// "sniff_max_bytes": 8192,
// "sniff_max_filesize": 2097152,
// "allow_xml_eval": false
// }
// }
```
- Archives uploaded are flagged and—if quarantine is enabled—moved to the quarantine directory for inspection.
- Quarantine should be owner `root`, group `www-data`, mode `0700`. Files inside should be `0600`.
Remove the leading `// ` when copying these example snippets into a real `upload-logger.json` file.
# 🔐 Per-Site PHP Upload Guard Integration Guide
## Fail2Ban integration (example)
This guide explains how to integrate a global PHP upload monitoring script
using `auto_prepend_file`, on a **per-site basis**, with isolated security
folders.
Create a Fail2Ban filter that matches suspicious JSON log lines and captures the IP as `<HOST>`.
---
## 📁 1. Recommended Folder Structure
Each website should contain its own hidden security directory:
```
/var/www/sites/example-site/
├── public/
├── app/
├── uploads/
├── .security/
│ ├── upload_guard.php
│ └── logs/
│ └── uploads.log
````
Benefits:
- Per-site isolation
- Easier debugging
- Independent log files
- Reduced attack surface
---
## 🔧 2. Create the Security Directory
From the site root:
```bash
cd /var/www/sites/example-site
mkdir .security
mkdir .security/logs
````
Set secure permissions:
- Set secure permissions:
```bash
chown -R root:www-data .security
chmod 750 .security
chmod 750 .security/logs
```
Quarantine hardening (important):
- Ensure the quarantine directory is owner `root`, group `www-data`, and mode `0700` so quarantined files are not accessible to other system users. Example provisioning script `scripts/provision_dirs.sh` now enforces these permissions and tightens existing files to `0600`.
- If using Ansible, the playbook `scripts/ansible/upload-logger-provision.yml` includes a task that sets any existing files in the quarantine directory to `0600` and enforces owner/group.
- Verify SELinux/AppArmor contexts after provisioning; the script attempts to register fcontext entries and calls `restorecon` when available.
---
## 📄 3. Install the Upload Guard Script
Create the script file:
```bash
nano .security/upload_guard.php
```
Paste your hardened upload monitoring script.
Inside the script, configure logging:
```php
$logFile = __DIR__ . '/logs/uploads.log';
```
Lock the script:
```bash
chown root:root .security/upload_guard.php
chmod 644 .security/upload_guard.php
```
---
## ⚙️ 4. Enable auto_prepend_file (Per Site)
### Option A — PHP-FPM Pool (Recommended)
Edit the site’s PHP-FPM pool configuration:
```bash
nano /etc/php/8.x/fpm/pool.d/example-site.conf
```
Add:
```ini
php_admin_value[auto_prepend_file] = /var/www/sites/example-site/.security/upload_guard.php
```
Reload PHP-FPM:
```bash
systemctl reload php8.x-fpm
```
# 🔐 Per-Site PHP Upload Guard Integration Guide
This guide explains how to integrate a global PHP upload monitoring script
using `auto_prepend_file`, on a **per-site basis**, with isolated security
folders.
---
## 📁 1. Recommended Folder Structure
Each website should contain its own hidden security directory:
```text
/var/www/sites/example-site/
├── public/
├── app/
├── uploads/
├── .security/
│ ├── upload-logger.php
│ └── logs/
│ └── uploads.log
```
Benefits:
- Per-site isolation
- Easier debugging
- Independent log files
- Reduced attack surface
---
## 🔧 2. Create the Security Directory
From the site root:
```bash
cd /var/www/sites/example-site
mkdir .security
mkdir .security/logs
```
Set secure permissions:
```bash
chown -R root:www-data .security
chmod 750 .security
chmod 750 .security/logs
```
---
## 📄 3. Install the Upload Guard Script
Create the script file:
```bash
nano .security/upload-logger.php
```
Paste your hardened upload monitoring script.
Inside the script, configure logging:
```php
$logFile = __DIR__ . '/logs/uploads.log';
```
Lock the script:
```bash
chown root:root .security/upload-logger.php
chmod 644 .security/upload-logger.php
```
---
## ⚙️ 4. Enable auto_prepend_file (Per Site)
### Option A — PHP-FPM Pool (Recommended)
Edit the site’s PHP-FPM pool configuration:
```bash
nano /etc/php/8.x/fpm/pool.d/example-site.conf
```
Add:
```ini
php_admin_value[auto_prepend_file] = /var/www/sites/example-site/.security/upload-logger.php
```
Reload PHP-FPM (adjust service name to match your PHP version):
```bash
systemctl reload php8.x-fpm
```
---
### Option B — Apache Virtual Host
If using a shared PHP-FPM pool, configure in the vHost:
```apache
<Directory /var/www/sites/example-site>
php_admin_value auto_prepend_file /var/www/sites/example-site/.security/upload-logger.php
</Directory>
```
Reload Apache:
```bash
systemctl reload apache2
```
---
## 🚫 5. Block Web Access to `.security`
Prevent direct HTTP access to the security folder.
In the vHost:
```apache
<Directory /var/www/sites/example-site/.security>
Require all denied
</Directory>
```
Or in `.htaccess` (if allowed):
```apache
Require all denied
```
---
## ✅ 6. Verify Installation
Create a temporary file:
```php
<?php phpinfo();
```
Open it in browser and search for:
```text
auto_prepend_file
```
Expected output:
```text
/var/www/sites/example-site/.security/upload_guard.php
```
Remove the test file after verification.
---
## 🧪 7. Test Upload Logging
Create a simple upload test:
```php
<form method="post" enctype="multipart/form-data">
<input type="file" name="testfile">
<button>Upload</button>
</form>
```
Upload any file and check logs:
```bash
cat .security/logs/uploads.log
```
You should see a new entry.
---
## 🔒 8. Disable PHP Execution in Uploads
Always block PHP execution in upload directories.
Example (Apache):
```apache
<Directory /var/www/sites/example-site/uploads>
php_admin_flag engine off
AllowOverride None
</Directory>
```
Reload Apache after changes.
---
## 🛡️ 9. Enable Blocking Mode (Optional)
After monitoring for some time, enable blocking.
Edit:
```php
$BLOCK_SUSPICIOUS = true;
```
Then reload PHP-FPM.
---
## 📊 10. (Optional) Fail2Ban Integration (JSON logs)
Create a JSON-aware filter that matches `event: "suspicious"` and extracts the IP address.
```bash
nano /etc/fail2ban/filter.d/php-upload.conf
```
Filter (`/etc/fail2ban/filter.d/php-upload.conf`):
```ini
[Definition]
# Match JSON lines where event == "suspicious" and capture the IPv4 address as <HOST>
failregex = ^.*"event"\s*:\s*"suspicious".*"ip"\s*:\s*"(?P<host>\d{1,3}(?:\.\d{1,3}){3})".*$
ignoreregex =
```
Create a jail that points to the per-site logs (or a central aggregated log):
```ini
[php-upload]
enabled = true
filter = php-upload
logpath = /var/www/sites/*/.security/logs/uploads.log
maxretry = 3
findtime = 600
bantime = 86400
action = iptables-multiport[name=php-upload, port="http,https", protocol=tcp]
```
Restart Fail2Ban:
```bash
systemctl restart fail2ban
```
### Fail2Ban action: nftables example
If your host uses nftables, prefer the `nftables` action so bans use the system firewall:
Jail example (adjust `logpath`):
```ini
[php-upload]
@@ -471,107 +96,35 @@ bantime = 86400
action = nftables[name=php-upload, port="http,https", protocol=tcp]
```
This uses Fail2Ban's `nftables` action (available on modern distributions). Adjust `port`/`protocol` to match your services.
Test with `fail2ban-regex` using a representative JSON log line.
### Central log aggregation (Filebeat / rsyslog)
## Central log aggregation (Filebeat / rsyslog)
Forwarding per-site JSON logs to a central collector simplifies alerts and Fail2Ban at scale. Two lightweight options:
- Filebeat prospector (send to Logstash/Elasticsearch):
Forward JSON logs to your aggregator to centralize alerts and analysis. Example Filebeat input:
```yaml
filebeat.inputs:
- type: log
paths:
- /var/www/sites/*/.security/logs/uploads.log
json.keys_under_root: true
json.add_error_key: true
fields:
source: php-upload-logger
paths:
- /var/www/sites/*/.security/logs/uploads.log
json.keys_under_root: true
json.add_error_key: true
fields:
source: php-upload-logger
output.logstash:
hosts: ["logserver:5044"]
```
- rsyslog `imfile` forwarding to remote syslog (central rsyslog/logstash):
## Logrotate & SELinux notes
Add to `/etc/rsyslog.d/10-upload-logger.conf`:
Per-site `logrotate` snippets are included in `examples/logrotate.d/upload-logger`. Use `copytruncate` or reload PHP-FPM after rotation.
```text
module(load="imfile" PollingInterval="10")
input(type="imfile" File="/var/www/sites/*/.security/logs/uploads.log" Tag="uploadlogger" Severity="info" Facility="local7")
*.* @@logserver:514
```
If SELinux is enabled, the provisioning script attempts to register fcontexts and run `restorecon`. Verify contexts manually as needed.
Both options keep JSON intact for downstream parsing and reduce per-host Fail2Ban complexity.
## Final notes
### Testing your Fail2Ban filter
- Use observe mode (`ops.block_suspicious: false`) while tuning.
- After tuning, enable blocking in a controlled rollout (canary hosts first).
- Keep `upload-logger.php` and `.security` owned by `root` and ensure logs and quarantine are not web-accessible.
Create a temporary file containing a representative JSON log line emitted by `upload-logger.php` and run `fail2ban-regex` against your filter to validate detection.
```bash
# create test file with a suspicious event
cat > /tmp/test_upload.log <<'JSON'
{"ts":"$(date -u +%Y-%m-%dT%H:%M:%SZ)","event":"suspicious","ip":"1.2.3.4","user":"guest","name":"evil.php.jpg","real_mime":"application/x-php","reasons":["bad_name","php_payload"]}
JSON
# test the filter (adjust path to filter if different)
fail2ban-regex /tmp/test_upload.log /etc/fail2ban/filter.d/php-upload.conf
```
`fail2ban-regex` will report how many matches were found and display sample matched groups (including the captured `<HOST>`). Use this to iterate on the `failregex` if it doesn't extract the IP as expected.
---
## 🏁 Final Architecture
```text
Client → Web Server → PHP (auto_prepend) → Application → Disk
↓
Log / Alert / Ban
```
This provides multi-layer upload monitoring and protection.
---
## 🗂️ Log rotation & SELinux/AppArmor notes
- Example `logrotate` snippet to rotate per-site logs weekly and keep 8 rotations:
```text
/var/www/sites/*/.security/logs/uploads.log {
weekly
rotate 8
compress
missingok
notifempty
create 0640 root adm
}
```
- If your host enforces SELinux or AppArmor, ensure the `.security` directory and log files have the correct context so PHP-FPM can read the script and write logs. For SELinux (RHEL/CentOS) you may need:
```bash
chcon -R -t httpd_sys_rw_content_t /var/www/sites/example-site/.security/logs
restorecon -R /var/www/sites/example-site/.security
```
Adjust commands to match your platform and policy. AppArmor profiles may require adding paths to the PHP-FPM profile.
## ⚠️ Security Notes
- Never use `777` permissions
- Keep `.security` owned by `root`
- Regularly review logs
- Update PHP and extensions
- Combine with OS-level auditing for best results
---
## 📌 Recommended Maintenance
Weekly:
```bash
grep ALERT .security/logs/uploads.log
```
For installation steps and per-site configuration, see `docs/INSTALLATION.md`.